/ docs / heartbeats

Heartbeats: ping URLs, period and grace

How Fettle heartbeats work: create one, call its ping URL from a cron job, report start and failure, and set the period and grace that decide lateness.

Updated

How a heartbeat decides a run is late

A heartbeat is a URL your job calls when it finishes. Each successful ping sets the next deadline to the moment of the ping plus the period plus the grace. Fettle checks deadlines every 30 seconds. When one passes with no ping, the heartbeat goes DOWN, an incident opens with the cause HEARTBEAT_MISSED, and the alert goes out.

To alert after T seconds of silence from a job that runs every N seconds, set the period to N and the grace to T − N. A job that runs hourly and may take 20 minutes gets a period of 3600 and a grace of 1200.

A new heartbeat starts PENDING, and a pending heartbeat is never late: the clock starts at its first successful ping. Resuming a paused heartbeat puts it back to pending for the same reason, so a job that was off for a week does not alert the moment it is resumed.

Create a heartbeat

In the dashboard, under Heartbeats, or with a write-scoped API key:

Create a heartbeat
curl -s -X POST https://api.fettle.sh/v1/heartbeats \
  -H "Authorization: Bearer fettle_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{
        "name": "Nightly backup",
        "period_seconds": 86400,
        "grace_seconds": 3600
      }'

The response is the new heartbeat, with 201. Its ping_url field is the URL your job calls; the token at its end is the only credential a ping needs, so keep it out of public logs and repositories.

FieldAcceptsMeaning
name1–255 charactersWhat the alert calls the job.
period_seconds60 to 2,592,000 (30 days)How often the job runs.
grace_seconds0 to 2,592,000 (30 days)How long past the period to wait before the run counts as late.
notification_cooldown_seconds60 to 86,400; optional, 1,800 when omittedThe shortest gap between two alerts for this heartbeat.
channel_idsChannel ids; optionalWhere alerts go. Omitted or empty, every verified, active channel gets them. Email to the heartbeat's creator goes out either way.
tagsUp to 10 labels; optionalFilters for lists and bulk tag edits.

Heartbeats per organization: 3 on Free, 20 on Plus, 50 on Pro and 150 on Max. Past the limit, the create call answers 403 naming it.

The ping URL

Three paths share one token. Each accepts GET or POST, so plain curl or wget works:

  • ping_url — the run succeeded. The heartbeat goes UP and the next deadline is set. If it was down, the incident closes and the recovery alert goes out.
  • ping_url/start — the run began. The deadline moves to now plus period plus grace and the state stays as it is. The next success ping records how long the run took.
  • ping_url/fail — the run failed. The heartbeat goes DOWN at once, without waiting for the deadline, and an incident opens with the cause HEARTBEAT_FAILED. A heartbeat already down does not alert twice.

A ping needs no API key and no body; Fettle reads neither. Each answers {"ok":true}, or 404 for a token that does not exist. Pings are limited to 60 a minute per token, so a job stuck in a loop throttles only itself. A paused heartbeat ignores pings.

Ping after a successful run (crontab)
0 3 * * * /usr/local/bin/backup.sh && curl -fsS -m 10 --retry 3 https://api.fettle.sh/v1/heartbeat/<token> > /dev/null

The && sends the ping only when the job exits 0, so a failed run stays silent and turns into a missed deadline. To hear about a failure at once instead, and record the run’s duration, wrap the job:

Report start, success and failure (shell)
#!/bin/sh
PING="https://api.fettle.sh/v1/heartbeat/<token>"

curl -fsS -m 10 "$PING/start" > /dev/null
if /usr/local/bin/backup.sh; then
  curl -fsS -m 10 "$PING" > /dev/null
else
  curl -fsS -m 10 "$PING/fail" > /dev/null
fi

Planned downtime

Maintenance windows silence monitors, not heartbeats. Before turning a job off, pause its heartbeat; resume it when the job is back, and the clock restarts at the next success ping.

Where the alert goes

A heartbeat alerts on the same channels as a monitor. A webhook delivery names the heartbeat as its source with no target. What heartbeats include on each plan is on the cron job monitoring page; a step-by-step setup is in the cron job monitoring guide.