Set up the MMHotline-down alarm
Who acts: an administrator with a Cloudflare account and sudo on the
MMHotline server, or that person's coding agent.
When to use this page: so that someone is still woken when MMHotline itself is down: its server dies, or its location loses power or the internet.
What this page does: you give MMHotline a Cloudflare token, and it puts a small alarm program in your Cloudflare account that wakes the person on call when MMHotline stops checking in.
Terms:
- The MMHotline-down alarm is a Cloudflare Worker, a small program that Cloudflare runs in your team's own Cloudflare account, outside the MMHotline server. Setup is optional and needs no domain.
- MMHotline checks in with it every minute and hands it the Pushover notifications and automated voice calls to send if the check-ins stop, so a change of schedule or contact method reaches it within a minute.
Before you start
- MMHotline is installed, and you can change its settings file, as Change the settings says.
- The person on call and the backup have a Pushover notification or an automated voice call that gets SOS alerts, as Add contact methods says. Reason: only those two wake a sleeping person, so the alarm holds only those.
- A Cloudflare account; the free plan is enough.
Steps
-
Make a Cloudflare token. In the Cloudflare dashboard, copy your account ID from the account's home page. If you have never opened Workers & Pages, open it once, so the account gets its
workers.devaddress. Under Manage Account → Account API Tokens, create a custom token with one permission: Account › Workers Scripts › Edit.Make it an account API token, not a token under My Profile. Reason: MMHotline keeps the token to upload the new alarm after each upgrade, and Cloudflare stops a person's own tokens when that person leaves the account.
-
Add the alarm to the settings file, with your values:
mmhotline_down_alarm: cloudflare_account_id: 0123456789abcdef0123456789abcdef cloudflare_api_token: "YOUR_CLOUDFLARE_WORKERS_API_TOKEN" wake_the_person_on_call_after_minutes_without_a_check_in: 10 wake_the_backup_after_minutes_without_a_check_in: 20wake_the_person_on_call_after_minutes_without_a_check_in: how long MMHotline may be silent before the alarm wakes the person on call, 3 to 60 minutes.wake_the_backup_after_minutes_without_a_check_in: when it also wakes the backup, at least 5 minutes later. The backup is the one an unclaimed SOS escalates to: the named backup, or everyone else available withbackup_option: everyone_available.
A team whose deploy tool writes the settings file keeps the token in its vault.
-
Apply the file:
sudo mmhotline settings applyCheck: the line
mmhotline_down_alarm: applied.MMHotline uploaded the Worker, turned on itsworkers.devaddress, and checked in. A new address takes a few seconds to answer, so MMHotline tries the first check-in for up to 30 seconds. -
Check who would be woken:
sudo mmhotline down-alarm showCheck: it names the Worker and the last check-in, and lists each person, how, and after how long:
MMHotline-down alarm Worker mmhotline-down-alarm-ops-example-com-7db8a22e at https://mmhotline-down-alarm-ops-example-com-7db8a22e.example.workers.dev. Set up by the settings file /etc/mmhotline/mmhotline_settings.yml at 2026-11-15T07:46:30Z. Last check-in accepted at 2026-11-15T07:46:30Z. If MMHotline stops checking in: Pushover notification to Carlos Example (u...) after 10 minutes. If MMHotline stops checking in: Automated voice call to Carlos Example (+12025550124) after 10 minutes. If MMHotline stops checking in: Pushover notification to Barbara Example (u...) after 20 minutes.The dashboard's MMHotline-down alarm card shows the same.
-
Send a test:
sudo mmhotline down-alarm send-testCheck: each phone gets an emergency Pushover notification titled Test of the MMHotline-down alarm, or a call that says it is a test; acknowledge the Pushover test. The command lists each test and exits 0 when every service accepted it:
The MMHotline-down alarm sent a test alert for each held message, and each service accepted it.The dashboard card's Send a test now does the same.
What happens when MMHotline is down
- After the first wait, the alarm sends the held notifications to the person on call; after the second, to the backup.
- When MMHotline runs again, the people it reached get an Info alert, "MMHotline runs again: it was down, and the MMHotline-down alarm woke you", which lists what the alarm sent.
- If MMHotline can't reach the alarm for 10 minutes, such as during a
Cloudflare outage, the team gets a Warning, and the dashboard card says
Not checking in. Reason: without check-ins, the alarm sends its
urgent alerts once, then nothing more, even if MMHotline really goes
down later.
sudo mmhotline down-alarm showsays why check-ins fail. - After an upgrade, MMHotline uploads the new alarm by itself.
Change or remove the alarm
- Change the waits: change the two keys and apply.
- Turn it off: delete the
mmhotline_down_alarmsection and apply. The line saysmmhotline_down_alarm: not set up, because the settings file leaves it out.MMHotline first tells the alarm to hold nothing. Reason: stopping MMHotline without that would wake the person on call after the wait. The Worker stays in your Cloudflare account; delete it under Workers & Pages if you no longer need it. - Move it to another Cloudflare account: turn it off and apply, then add the section with the new account's ID and token and apply again. MMHotline refuses a direct change of account while the alarm is set up.
- Before you stop MMHotline on purpose, such as to remove it, turn the alarm off first, as Remove MMHotline says.
If something is different
- The test says
The MMHotline-down alarm sent a test alert for each held message; 3 of 3 were not accepted, so those people would not be woken., and each test line saysrefused by the service (HTTP 400), and the command exits 1: the alarm reached Pushover or Twilio, which refused the message, such as for a wrong user key or phone number. Check those contact methods withsudo mmhotline contact-methods send-test. - The test says
The MMHotline-down alarm holds no message, so it would wake nobody.and exits 1: nobody on call or as backup has a Pushover notification or an automated voice call that gets SOS alerts. - The apply says
mmhotline_down_alarm: did not apply: the MMHotline-down alarm's Worker was uploaded, but MMHotline's first check-in with the MMHotline-down alarm failed for 30 seconds: ...: the reason follows; the earlier setup stays, or none the first time. Apply again once the reason is fixed. showlists nobody, or the dashboard says the person on call can't be reached: the person has no Pushover notification or automated voice call that gets SOS alerts.- Turning it off fails with the
mmhotline_down_alarmpart not applied: the alarm could not be told. Apply again, or delete the Worker in the Cloudflare dashboard and apply again.
This page is docs/how-to/set-up-the-mmhotline-down-alarm.md in the MMHotline repository.