Fleet runs and server tags

Server tags

Open a server page and use + tag in the header. Tags are lowercase, up to 40 characters of letters, digits and : _ . / -, with up to 20 per server. Any role can edit them, and the change is recorded in the audit log. Suggestions come from tags already used in the workspace, so prod, nginx or client:acme stay consistent.

On the fleet grid each server shows its tags as chips. Click a chip, or use the All tags filter, to show only matching servers. The grid then offers ▶ Run on "tag" (N), which opens the fleet-run composer with that tag selected; with no tag set it targets exactly the servers currently visible.

What a fleet run is

A fleet run is one action applied to many servers. Nothing new runs on the agent: every step on every server is a normal remote command, signed and recorded in that server’s history, linked back to the run. Open Automation → Fleet runs and click Run on servers. The composer has three kinds:

• •Command — one shell line, run on each server.
• •Runbook — pick a saved runbook; its steps become the run’s steps.
• •Push file — upload a file once and have each server fetch it (see below).

Who runs, who is skipped

Under Which servers pick tags, servers, or both. The composer shows a live preview of who will run and who will be skipped, with the reason beside each skipped server. A server is skipped, never silently included, when any of these is true:

• •Remote actions are off for this server
• •Server is offline
• •Kubernetes cluster agents have no command executor
• •Server was removed or archived

The same checks run again for each server at the moment it is reached, so a server that went offline mid-run is skipped rather than failed. A run can target up to 500 servers and have up to 50 steps.

How it rolls out

Under How, choose a Rollout: All at once, One server at a time, 25% at a time, 50% at a time, or Custom with a number. Servers in a wave run in parallel; on each server the steps run in order and stop at the first non-zero exit. Timeout per server (s) defaults to 120 seconds per command, up to 600.

Tick Stop on first failure and the run halts after any wave in which a server failed; the servers not yet reached are marked not reached. This only matters when rolling in waves. A confirmation dialog summarises what will run, where, and how before anything is dispatched.

Dry run

Click Dry run in the composer to see exactly what would happen without sending anything: each wave, each server in it, the exact commands it would run in order, and every skipped server with its reason. Nothing is dispatched. A file push can’t be dry-run, because it uploads the file first.

Run on a schedule

Under When, choose On a schedule instead of Now: Every day, Weekdays (Mon–Fri), On chosen days, or Every few hours (every 1, 2, 3, 4, 6, 8 or 12 hours, so the times line up the same way each day), at a time in the time zone you pick. The composer shows the next run times before you save. Saved schedules are listed under Schedules on the Fleet runs page, where you can pause or resume them.

• •At each run the servers are picked again from your tags and list, with the same checks, so a new server in the tag is included and an offline one is skipped.
• •A run that would start more than 10 minutes late, or while the previous run of the same schedule is still going, is skipped, never run late.
• •The schedule pauses itself, with the reason shown, if the person who created it is no longer an owner or admin, the workspace drops below the Growth plan, or its runbook was deleted.
• •A file push can run on a schedule too, for files up to 50 MB: Opservo keeps one copy of the file and pushes that same copy at every run. Deleting the schedule deletes the copy. Push bigger files with Now.

Results and stopping

Each run has a page with a status, a progress bar, the step list, and a per-server table: status (ok, failed, skipped, not reached), skip reason, steps completed, exit code and time taken. Expand a row to read the stdout and stderr of every step. Filter the table by Failed or Skipped / not reached.

While a run is live you have two buttons. Stop after current wave lets commands already running finish and starts nothing new. Stop now also cancels them: servers not yet started are not touched, and on servers running a command right now the command and everything it started are killed. A command killed half-way can leave a server half-changed, so check those servers afterwards. Servers on an older agent can’t be interrupted; their current command finishes and nothing more runs after it. The Fleet runs list keeps the history of every run.

Push a file to many servers

Choose Push file, select a file (up to 2 GB), set the Destination folder on each server and the file mode (default 0644). The file is staged once; each server pulls it over its own outbound HTTPS with a single-use link and installs it with sudo install. The step shown in the results contains {{pull_url}}, which is replaced with each server’s own link. Optionally add Then run, for example sudo nginx -t && sudo systemctl reload nginx, which runs as a second step on each server after the file lands.

From chat

With Let AI fix things on in AI chat, you can ask for a fleet run in plain words, for example “how full are the disks on the web servers?” or “restart php-fpm on every server tagged web”. The AI picks the servers by tag or name and starts an ordinary fleet run, with every check on this page applied to each server.

• •Read-only checks run straight away, for example looking at disk space or a service’s status.
• •Anything that changes a server never runs on the first ask. The AI shows who would run, who is skipped and why, and offers one exact answer, such as Yes, run on 6 servers (web · ref 3fa91c): sudo systemctl restart php-fpm. Only that answer starts the run. If the servers or the command change before you answer, even by one server, it asks again with a new ref.
• •Saved runbooks: ask it to run a runbook by name. The answer names the runbook and its number of steps, and a runbook edited in the meantime is asked about again.
• •Files: with act mode on, the chat box has a 📎 button. Attach a file (it is kept for 15 minutes), say where it goes, and the AI offers Yes, push site.conf to /etc/nginx/conf.d …, with an optional follow-up command. It always asks first.

The run shows as a card in the chat with live counts and a link to its results page, and it is listed under Fleet runs like any other. The same people can do it as on the page: owners and admins, Growth plan or higher.

Who can run one

Starting, stopping or pushing a file needs the owner or admin role, a 2FA code if your workspace requires one before server actions, and the Growth plan or higher. Any role can view runs and previews. Every dispatch and stop is written to the audit log, and each server keeps its own command record under its Automation tab.