Skip to content

Runbook — On-call rotation

Honest status (2026-05-10): Puccha is a one-person engineering team. The “rotation” is a rotation of one. This document captures that reality so it’s planned-for, not pretended-around. When the team hits N≥2 engineers, the §3 rotation matrix gets filled in and this note gets deleted.

  • Primary on-call: founder (24/7, every day)
  • Escalation contact: none; escalation is to self
  • Acknowledgement clock for P0: 15 min. With one human this means P0 outside business hours has a hard floor at “founder is awake and checks Sentry within 15 min of the page firing.” Phone push notification is the minimum (ADR-0048 ships this).
  • Coverage gaps: travel, sleep, illness. Documented as known. The three CF-level mitigations that hold even when the on-call is unreachable:
    • CF WAF rate rules (Layer 1, ADR-0017 §4a) — auto-blocks abuse
    • KV per-session rate limit (Layer 2) — auto-blocks per-session abuse
    • AI Gateway monthly cost cap (Layer 4) — auto-blocks cost runaway
  • These don’t recover from a correctness outage. They cap the cost and blast radius until a human is available.

What pages the on-call:

Surface Trigger Notify via
Sentry alert error rate spike, new high-volume issue Email + ADR-0048 Web Push
CF Health Check /health returns non-200 ≥ 3 consecutive checks Email
Stripe webhook failures repeated webhook delivery failures Stripe dashboard email
External report tenant emails support, security researcher Email inbox
tenant_events quota_exceeded enterprise tenant burns daily quota Future: queue-driven email; today: visible in /admin

Not yet wired (future work, not blocking on N=1):

  • PagerDuty / Grafana OnCall / Discord webhook integration — tool selection deferred until N≥2 (ADR-0050 §“Non-goals”)
  • Phone call escalation
  • “Acknowledged but no progress in T minutes” auto-escalation

3. Rotation matrix (placeholder for N≥2)

Section titled “3. Rotation matrix (placeholder for N≥2)”

When the team grows past one engineer, populate this table:

Week of Primary Secondary Notes
YYYY-MM-DD name name first week of rotation; primary takes the comms role

Defaults to revisit at that point:

  • Rotation length: 1 week (industry standard for small teams; longer becomes a burnout risk)
  • Hand-off ritual: 15-min sync at week start covering open incidents, any acknowledged-not-resolved alerts, deploy windows in the coming week
  • Compensation: TBD with HR — at small scale, expect comp-time or a flat per-week stipend

Important to write down so the role doesn’t drift:

  • Build features. The on-call shift is reactive; feature work happens off-shift.
  • Polish runbooks. Improvements land as a normal PR against this directory, not as part of an active shift.
  • Tenant support tickets. Those go through the support inbox; the on-call only takes the ones flagged P0 / P1 by support.
  • Reach into another team member’s code without coordination — except during an active incident, where the incident-response runbook governs.

5. Hand-off when the on-call goes off-shift (N≥2 future)

Section titled “5. Hand-off when the on-call goes off-shift (N≥2 future)”
  • Open incidents: documented status in docs/security/incidents/YYYY-MM-DD-{slug}.md, owner reassigned with a note in the timeline
  • Acknowledged-but-not-resolved alerts: list in the hand-off sync
  • Pre-staged comms (drafts that haven’t been sent): linked from the incident timeline
  • Deploys planned for the next week: noted in the team channel so the incoming on-call isn’t surprised

For N=1, the hand-off is mental — no team to hand off to. Write it down anyway during multi-day incidents, future-you will thank present-you.

The on-call participates in the quarterly tabletop drill (ADR-0050 §5, incident-response.md §5). At N=1 the drill is a self-walk-through, not a team exercise; the output (notes on what was unclear) is still valuable.


When N≥2: delete the §1 honest-status note, populate §3, and add a “Last rotation reviewed: YYYY-MM-DD” line at the top.