Make a one-time change
Who acts: you, or your coding agent, in your MMX project on your
controller machine: the computer where you run mmx.
When to use this page: some work must happen once on each server and never again, such as moving data to a new directory, removing a program the team no longer uses, or restarting a service once.
| Your situation | Go to |
|---|---|
| You start a new piece of one-time work | Steps |
| The run failed on some servers, or a server was down | When the run fails on some servers |
| A new server joined the inventory after the work started | A server that joins later |
| The work is a plain command, a restart, or the removal of a program | Other kinds of one-time work |
| The project has an inventory for each environment | Several inventories |
What this page does: you write a one-time playbook that moves the
shop's uploaded files from /srv/shop/uploads to /srv/shop/files. You
run the one-time playbook, find the servers that still need the work, and
mark the one-time playbook as completed when no server needs the work.
Terms:
- A node is a server that MMX manages, listed in an inventory file
in
inventories/. - A playbook lists fleet stages. Each fleet stage picks nodes, and each of those nodes runs the fleet stage's node tasks, in order.
- A one-time playbook is a playbook in the directory
one_time_playbooks/whose file name starts with a date, such as2026-11-15_move_uploads_to_files_directory.yml. A one-time playbook runs only when a command names it. When no node needs the one-time playbook any more,mmx complete-one-time-playbookmoves its file toone_time_playbooks/completed/, which keeps the project's record of finished one-time work.
Before you start
- Your project has an inventory with the servers and a group, such as
web_serverswithweb01,web02, andweb03. Describe your servers shows how. - You know how to write a playbook and run it: Write a playbook and Preview and run changes.
- If the project is in Git, keep it there. The last step moves a file, and
you commit the move so that the rest of the team sees the work as
completed.
mmx initdoes not create a Git repository.
Steps
-
Create the one-time playbook. Give
mmx new-one-time-playbookwhat the work does, in lowercase words joined by underscores:mmx new-one-time-playbook move_uploads_to_files_directoryCreated one_time_playbooks/2026-11-15_move_uploads_to_files_directory.yml. Replace its example node task with the work, then preview it with mmx dryrun 2026-11-15_move_uploads_to_files_directory.MMX puts the controller machine's date before your words. The file name without
.ymlis the one-time playbook's name, which each command takes. The new file is a valid playbook that changes nothing:# A one-time playbook holds work that must happen once on every node it # selects and never again, such as removing a retired program. It runs only # when a command names it. # # Preview what it would change: mmx dryrun 2026-11-15_move_uploads_to_files_directory # Run it: mmx run 2026-11-15_move_uploads_to_files_directory # Check which nodes still need it: mmx check-one-time-playbooks # Complete it when no node needs it: mmx complete-one-time-playbook 2026-11-15_move_uploads_to_files_directory fleet_stages: - id: move_uploads_to_files_directory nodes: all node_tasks: # An example node task that changes nothing. Replace it with the work. - show: "Replace this example node task with the work of this one-time playbook."Check:
mmx validateprintsValid: mmx_example. Ifmmx new-one-time-playbookrefuses the words, see If something is different. -
Write the work. Replace the whole file with the following YAML:
# one_time_playbooks/2026-11-15_move_uploads_to_files_directory.yml # The shop now keeps uploaded files in /srv/shop/files instead of /srv/shop/uploads. fleet_stages: - id: move_uploads_to_files_directory nodes: web_servers # an inventory group node_tasks: - id: move_uploads shell: script: mv /srv/shop/uploads /srv/shop/files run_only_if_this_path_exists: /srv/shop/uploads # nothing to move: the server is done dryrun: ls /srv/shop/uploads # a read-only command for mmx dryrunReason: MMX finds out whether a server still needs the work by previewing the one-time playbook on the server, as
mmx dryrundoes. Ashellnode task's preview can tell only from what you give the node task.run_only_if_this_path_existsskips the move on a server without/srv/shop/uploads, so such a server counts as done.dryrunnames a command that changes nothing, and MMX runs thedryruncommand instead of the move during a preview. Without the guard, no server ever counts as done. Without thedryruncommand, a server that still needs the move is reported asnot known. How MMX knows which servers are done explains the reason further.Check:
mmx validateprintsValid: mmx_example. -
Preview the work.
mmx dryrun 2026-11-15_move_uploads_to_files_directory echo "exit code $?"move_uploads_to_files_directory / web01 / move_uploads: succeeded, would_change move_uploads_to_files_directory / web02 / move_uploads: succeeded, would_change move_uploads_to_files_directory / web03 / move_uploads: succeeded, would_change Run 0e84e15e: succeeded 3 finished Saved: /home/you/shop/.mmx/runs/20261115T053006.610880428-0e84e15e481b05c5 exit code 2Check: each server says
would_change, and the exit code is 2, which for a dryrun means "something would change" (exit codes). A server that saysskipped, unknown — the shell node task has no dryrun commandhas nodryruninput: add thedryruninput as in step 2. -
Run the work on one server first.
mmx run 2026-11-15_move_uploads_to_files_directory --limit web01move_uploads_to_files_directory / web01: succeeded move_uploads_to_files_directory / web01 / move_uploads: succeeded, changed Run fec80ec8: succeededCheck: the run ends with
Run ...: succeeded. Onweb01,ls /srv/shop/fileslists the uploaded files, and/srv/shop/uploadsno longer exists. Check the shop onweb01before you go on. -
See which servers still need the work.
mmx check-one-time-playbooks echo "exit code $?"Inventory: example (inventories/example.yml) 2026-11-15_move_uploads_to_files_directory done on 1 of 3 servers still needed on 2 servers: web02, web03 exit code 2mmx check-one-time-playbookspreviews each one-time playbook outsidecompleted/on the servers that the one-time playbook selects. The preview changes nothing that the one-time playbook manages. While each preview runs, standard error shows progress lines such asInspecting 2026-11-15_move_uploads_to_files_directory: 3 not started.Check:
web01counts as done. Each server is in one of three states:State Meaning What to do doneThe preview found nothing to change on the server. Nothing. still neededA node task would change the server. Run the one-time playbook. not knownMMX could not find out. The reason follows the server's name. Fix what the reason names, then check again. The exit code is 0 when every server is done, 2 when a server still needs the work, and 3 when the state of a server is not known.
--output nodesalso names the servers that are done, and the preview run that inspected the servers, such asinspected by dryrun 25e8601c; mmx results 25e8601c --diff shows what a server still needs. -
Run the work on every server.
mmx run 2026-11-15_move_uploads_to_files_directoryCheck: the run ends with
Run ...: succeeded.web02andweb03saysucceeded, changed.web01saysskipped, unchanged — node task skipped by a path condition for: /srv/shop/uploads, because the move onweb01is already done. If the run fails on a server, go to When the run fails on some servers. -
Check that no server needs the work.
mmx check-one-time-playbooks2026-11-15_move_uploads_to_files_directory done on all 3 serversCheck: the exit code is 0.
-
Complete the one-time playbook.
mmx complete-one-time-playbook 2026-11-15_move_uploads_to_files_directoryInventory: example (inventories/example.yml) 2026-11-15_move_uploads_to_files_directory done on all 3 servers Completed: moved one_time_playbooks/2026-11-15_move_uploads_to_files_directory.yml to one_time_playbooks/completed/2026-11-15_move_uploads_to_files_directory.yml.mmx complete-one-time-playbookchecks the servers again, as step 7 did, for this one-time playbook only. When every server is done, MMX moves the file toone_time_playbooks/completed/and adds a first line with the date:# Completed 2026-11-15: done on all 3 servers.MMX refuses to run, preview, or check a one-time playbook in
completed/.Check: the output ends with
Completed: moved ..., and the exit code is 0.mmx check-one-time-playbooksnow printsNo outstanding one-time playbook: one_time_playbooks/ holds none outside completed/.If MMX refuses, the output says which server still needs the work; see When the run fails on some servers. -
Commit the move. MMX changes only the files in your copy of the project and commits nothing.
git add one_time_playbooks git commit -m "Complete 2026-11-15_move_uploads_to_files_directory"Check:
git statusshows no change inone_time_playbooks/.
How MMX knows which servers are done
The check uses no record of earlier runs. Each check
asks the servers again with a preview, so the answer comes from each
server as the server is now. For example, a server restored from an old
backup has /srv/shop/uploads again. While the one-time playbook is not
completed, the next check reports that server as still needed.
So each node task must be able to tell, in a preview, whether the server still needs the work:
- Many modules tell by themselves. A
filetask withstate: absentreportsunchangedwhen the file is already gone. - A
shelltask tells only from its inputs. Give ashelltask a guard, such asrun_only_if_this_path_existsorskip_if_this_command_succeeds, and adryruncommand, as in step 2. - A command that has no check gets a marker file:
run_only_once_marker_filenames a file on the node that MMX creates after the command succeeded. While the marker file exists, the node task is done. A plain command shows a marker file.
Without a guard, a dryrun command, or a marker file, the check reports the server as not known with
the reason
the dryrun of fleet stage move_uploads_to_files_directory could not check node task move_uploads.
Each check is an ordinary dryrun, saved in the project's .mmx/runs/
like any other run. Like mmx dryrun, the check sets up MMX on a server
that does not have MMX yet.
When the run fails on some servers
A run of a one-time playbook can fail on one server and succeed on the
others, as any run can. In this example, web02 has a file in the way of
the move, and web03 is switched off:
move_uploads_to_files_directory / web01: succeeded
move_uploads_to_files_directory / web02: failed
move_uploads: exit status 1
move_uploads_to_files_directory / web03: failed
SSH 203.0.113.13: exit status 255: ssh: connect to host 203.0.113.13 port 22: Connection timed out
move_uploads_to_files_directory / web01 / move_uploads: skipped, unchanged — node task skipped by a path condition for: /srv/shop/uploads
move_uploads_to_files_directory / web02 / move_uploads: failed, changed — exit status 1
exit code: 1
mv: cannot overwrite non-directory '/srv/shop/files' with directory '/srv/shop/uploads'
Run a861675a: failed
1 finished · 2 failed
Saved: /home/you/shop/.mmx/runs/20261115T053018.266721649-a861675aada5231d
Inspect: mmx results a861675a
Resume unfinished work: mmx resume a861675a
Start a new run: mmx run 2026-11-15_move_uploads_to_files_directory
-
Fix the cause on each failed server. The line under each server says what went wrong. Here,
/srv/shop/filesonweb02is an old file that you look at and remove by hand, andweb03must be switched on.Check:
mmx check-one-time-playbookslists the failed servers asstill needed. A server that MMX cannot reach shows asnot knownwith the reason:2026-11-15_move_uploads_to_files_directory done on 1 of 3 servers still needed on 1 server: web02 not known on 1 server: web03 (MMX could not connect: SSH 203.0.113.13: exit status 255: ssh: connect to host 203.0.113.13 port 22: No route to host)When
not knownremains after the server is reachable again, the reason names what to fix, such as a node task that the preview could not check. -
Repeat the failed work.
mmx resume a861675ammx resumerepeats the failed and unfinished work of runa861675a: What a resume repeats. If you changed the one-time playbook to fix the failure, start a new run instead:mmx run 2026-11-15_move_uploads_to_files_directory.Check: the run ends with
Run a861675a: succeeded, andweb02andweb03saysucceeded, changed. -
Check, then complete. Do steps 7 to 9 of Steps.
Check: the output ends with
Completed: moved ....
Until every server is done, mmx complete-one-time-playbook prints the
check, moves nothing, and says why:
mmx: one-time playbook 2026-11-15_move_uploads_to_files_directory is not completed and stays in one_time_playbooks/: 1 node still needs the one-time playbook, and the state of 1 server is not known; run the one-time playbook with mmx run 2026-11-15_move_uploads_to_files_directory; mmx results 00eb1fe1 --diff shows what the check found on each server
The exit code is then 3 when the state of a server is not known, and 2
when the only cause is a server that still needs the work.
mmx results 00eb1fe1 --node web02 --diff shows what web02 still needs.
A server that joins later
A new server that joins the inventory, or joins the group that a one-time playbook selects, is checked like the others.
-
A one-time playbook that is not completed yet selects the new server. A new server often has nothing to change, such as no
/srv/shop/uploadsto move, so the check counts the new server as done. A command with only a marker file differs: the new server has no marker file, so the check reports the server asstill needed, and a run would run the command on the new server. Hereweb04joined after a version of2026-11-15_compress_old_logswithout the guard ran:2026-11-15_compress_old_logs done on 3 of 4 servers still needed on 1 server: web04If the new server needs the work, run the one-time playbook. If the new server does not need the work, add a guard that skips the work on the new server, such as
run_only_if_this_path_exists. The check then counts the new server as done. A plain command shows the whole file with the guard. A node task with no guard input, such as a restart, can leave the new server out withexcept_nodes: [web04]on the fleet stage instead. Several inventories has the limit ofexcept_nodesin a one-time playbook. -
A completed one-time playbook never runs on the new server. Set up a new server with your ordinary playbooks, which describe the server as the server is now. If the new server does need the work, move the file back from
one_time_playbooks/completed/toone_time_playbooks/, run the one-time playbook, and complete the one-time playbook again. MMX adds a second# Completedline.
Other kinds of one-time work
A program that the team no longer uses
Modules such as directory and file with state: absent report
unchanged when the file or directory is already gone, so these modules
need no guard. This
one-time playbook removes rbenv, an old tool that managed Ruby versions:
# one_time_playbooks/2026-11-15_remove_old_ruby_manager.yml
fleet_stages:
- id: remove_old_ruby_manager
nodes: web_servers
node_tasks:
- id: remove_rbenv_directory
directory:
path: /opt/rbenv
state: absent
also_delete_everything_inside: true
- id: remove_rbenv_profile_script
file:
path: /etc/profile.d/rbenv.sh
state: absent
Work of this kind belongs in a one-time playbook, not in an ordinary
playbook or a role. An ordinary playbook runs the deletion on every run for
as long as the lines stay, and nobody knows when the lines may go. A
one-time playbook runs only when you name it, and
mmx check-one-time-playbooks shows when the work is finished.
A plain command
A command that has no way to check whether it already ran gets a marker
file. This one-time playbook compresses old log files once. gzip has no
preview of what it would compress, so a preview cannot tell whether a
server still needs the work:
# one_time_playbooks/2026-11-15_compress_old_logs.yml
fleet_stages:
- id: compress_old_logs
nodes: web_servers
node_tasks:
- id: compress_logs
shell:
script: gzip --recursive /srv/shop/old_logs
run_only_if_this_path_exists: /srv/shop/old_logs # a new server has no old logs
run_only_once_marker_file: /var/lib/mmx/commands_run_once/2026-11-15_compress_old_logs
MMX creates the marker file on the node after gzip succeeded. A
preview first checks the guard: a server without /srv/shop/old_logs is
done. Otherwise a missing marker file means still needed, and an existing
marker file means done. A failed command writes no marker file,
so the next run tries again. Keep marker files in
/var/lib/mmx/commands_run_once/, where MMX creates the directory. To run
the command again on purpose, delete the marker file on the node.
The shell module has the
details.
A restart
A service that runs but no longer answers can get one restart. Like a plain command, the restart needs a marker file, because a restart changes nothing that a preview can see:
# one_time_playbooks/2026-11-15_restart_shop_worker.yml
fleet_stages:
- id: restart_shop_worker
nodes: web_servers
node_tasks:
- id: restart_worker
systemd_service:
name: shop-worker
state: running
restart_now: true
run_only_once_marker_file: /var/lib/mmx/commands_run_once/2026-11-15_restart_shop_worker
The run reports
succeeded, changed — restarted shop-worker.service once and wrote the marker file /var/lib/mmx/commands_run_once/2026-11-15_restart_shop_worker.
On a server without the service, the check reports not known with
the dryrun of fleet stage restart_shop_worker could not check node task restart_worker.
A cleanup of disk space
systemd_journal_vacuum,
docker_unused_images,
podman_unused_images,
apt_downloaded_package_files,
and delete_old_files report unchanged
in a preview when nothing is left to delete. So these modules need no
marker file, and the check counts a clean server as done. Logs and images
keep coming, so a server can need the cleanup again later. Complete the
one-time playbook when every server is done.
Several inventories
mmx check-one-time-playbooks and mmx complete-one-time-playbook check
the servers of one inventory: the project's default_inventory, or the
inventory that --inventory NAME names. In a project with an inventory
for each environment, check each inventory before you complete:
mmx check-one-time-playbooks --inventory staging
mmx check-one-time-playbooks --inventory example
mmx complete-one-time-playbook 2026-11-15_remove_old_ruby_manager --inventory example
mmx complete-one-time-playbook names the inventories that the command
did not check:
Note: MMX inspected nodes only in inventory example. MMX did not inspect nodes in another inventory, staging. Before completing a one-time playbook, check every inventory with mmx check-one-time-playbooks --inventory NAME.
If a node in another inventory still needs the completed one-time playbook, move the playbook file back to one_time_playbooks/ and run the playbook with the inventory that contains the node.
A one-time playbook must load with every inventory, because mmx validate
checks each one-time playbook with each inventory. A server name that only
one inventory has, such as except_nodes: [web04], makes the one-time
playbook fail with the other inventories:
2026-11-15_restart_shop_worker
inspected nothing: MMX cannot load the playbook with inventory staging: fleet stage restart_shop_worker: one_time_playbooks/2026-11-15_restart_shop_worker.yml:4:19 (fleet_stages[0].except_nodes): no node or inventory group named "web04"; a one-time playbook that no node needs any more goes to one_time_playbooks/completed/
Select servers by a group that each inventory has, such as web_servers,
or use a guard instead of a server's name.
Rules for one-time playbooks
- The name. The file name is
YYYY-MM-DD_what_it_does.yml: a real date, an underscore, and lowercase letters and digits joined by underscores.mmx new-one-time-playbookalso wants the words to start with a letter, because the words become the ID of the fleet stage.mmx new-one-time-playbookrefuses a name that already exists, with today's date and the same words, inone_time_playbooks/or incompleted/. - The directory.
one_time_playbooks/holds only one-time playbooks,completed/, and files whose names start with a dot, such as files that Git or a file manager leaves.mmx validateand the check report any other file inone_time_playbooks/as an error. - The commands that take the name.
validate,explain,nodes,dryrun,run, andmodule-usagetake a one-time playbook's name, as for any playbook.mmx validatewithout a name checks each one-time playbook outsidecompleted/with each inventory. - Command-line variables need a default. Each of a one-time
playbook's
command_line_varshas adefault, because the check previews the one-time playbook without--var. - The check has no
--limit.mmx check-one-time-playbookschecks every server that a one-time playbook selects, and takes no--limit,--only-fleet-stages, or--var. To look at a few servers, usemmx dryrun NAME --limit web02. - A server that a fleet stage's
whenskips has nothing to do in that fleet stage, and counts as done there. - Other flags.
mmx check-one-time-playbooksandmmx complete-one-time-playbooktake--inventory NAME,--connection ssh_onlyfor a controller machine that cannot reach the servers' HTTPS port 7443,--non-interactive,--json, and--vault-password-file. MMX asks for the vault passphrase once, also when the check previews several one-time playbooks. Keep secrets explains the vault. - JSON.
mmx check-one-time-playbooks --jsonprints, for each one-time playbook, itsname, the preview'srun_id, and the listsdone,still_needed, andnot_known, each with thenodeand areason.
Before you remove a server from the inventory
A one-time playbook that names a server which the inventory no longer has
cannot load, so mmx validate fails:
mmx: fleet stage remove_example_file_from_web03: one_time_playbooks/2026-11-15_remove_example_file_from_web03.yml:3:12 (fleet_stages[0].nodes): no node or inventory group named "web03"; a one-time playbook that no node needs any more goes to one_time_playbooks/completed/
Remove MMX from a server says what to do first, and what to do when the server is already gone.
If something is different
| What you see | What it means and what to do |
|---|---|
mmx: WHAT_IT_DOES must be lowercase words joined by underscores, such as remove_old_ruby_manager; got "Move-Uploads" |
Write the words in lowercase, joined by underscores. |
mmx: one_time_playbooks/completed/2026-11-15_move_uploads_to_files_directory.yml already exists; a one-time playbook name cannot be reused after completion; choose different words for WHAT_IT_DOES |
A completed one-time playbook has the same name. Choose other words. |
mmx: dryrun requires a playbook name; available playbooks: example; one-time playbooks: 2026-11-15_move_uploads_to_files_directory |
mmx dryrun and mmx run need the one-time playbook's full name, with the date. |
skipped, unknown — the shell node task has no dryrun command; set dryrun to a read-only command, or run_script_in_dryrun: true for a script that changes nothing, to check the task |
A preview cannot tell whether the shell task has work. Add a guard and a dryrun command, or a marker file. See How MMX knows which servers are done. |
not known on 1 server: web03 (MMX could not connect: ...) |
MMX could not reach the server. Fix the connection, then check again. |
not known on 1 server: web04 (the dryrun of fleet stage restart_shop_worker could not check node task restart_worker) |
A preview cannot tell whether the server needs the node task. The run's results say why: mmx results RUN_ID --node web04, with the run ID from --output nodes. |
not known with a run would be refused: this node refuses management because /etc/mmx/do_not_manage exists |
Someone works on the server by hand. Check again after that person removes the file. See Keep MMX off a server. |
mmx: one-time playbook 2026-11-15_move_uploads_to_files_directory is completed: ... move the file back to one_time_playbooks/ to make it runnable again |
The one-time playbook is in completed/. Move the file back if a server needs the work again. |
mmx: one_time_playbooks/completed/... already exists, so MMX cannot move one_time_playbooks/... into the completed directory |
A file of the same name is in both directories. MMX never replaces a file in completed/. Remove or rename one of the two files. |
inspected nothing: MMX cannot load the playbook with inventory staging: ... |
The one-time playbook names a node or a group that the inventory doesn't have. See Several inventories. mmx complete-one-time-playbook refuses such a one-time playbook with exit code 4. |
No outstanding one-time playbook: one_time_playbooks/ holds none outside completed/. |
Every one-time playbook is completed. Nothing to do. |
Where to go next
- Preview and run changes: results, exit
codes,
mmx resume, andmmx results. - Choose which servers a run touches:
--limit, groups,except_nodes, and an inventory for each environment. - Keep MMX off a server: stop MMX for a while, or remove MMX from a server.
- The
shellmodule: guards,dryruncommands, and marker files.
This page is docs/how-to/make-a-one-time-change.md in the MMX repository.