Runbook — Chat Attachment Data Breach
- Severity tier: P0 (data exposure of visitor PII)
- First responder: on-call engineer
- Escalation: CTO + DPO within 60 minutes of breach confirmation
- Notification SLA: 24 hours to controller (tenant), 72 hours to PDPC / EU DPA
- Triggering events: see §1
1. Triggering events
Section titled “1. Triggering events”A breach is suspected when any of these are observed:
- Unexpected
403 → 200flip in/uploads/[...path]access logs (auth bypass) - R2 audit log shows reads from IPs / regions inconsistent with normal traffic
- Sentry alert on
chat.attachment.readaudit-row anomaly (volume / cross-tenant pattern) - External report (security researcher, customer, regulator) of attachment URL leakage
- Sub-processor breach disclosure (Cloudflare / Anthropic / Cohere / Sentry security notice)
- Internal discovery of
/uploads/URL in a public corpus (search engine, paste site, GitHub)
2. Containment (first 60 minutes)
Section titled “2. Containment (first 60 minutes)”Move fast on R2 and access controls before you write notification copy.
2.1 Stop the bleeding
Section titled “2.1 Stop the bleeding”- Disable the upload + read paths: deploy a hotfix that returns 503 from
/api/chat/uploadsand/uploads/[...path]for theattachments/prefix:Terminal window git checkout -b hotfix/disable-attachments# edit the two route handlers to return error(503, 'Maintenance')gh pr create --title 'hotfix: temporary disable chat attachments — incident' --body 'See incident-YYYY-MM-DD'gh pr merge --squash # bypass review for P0 - Verify rollout via
curl -I https://puccha-dev.hxlab.io/uploads/avatars/_test_andcurl -X POST https://puccha-dev.hxlab.io/c/_/api/chat/uploads. - If the leak is via an R2 URL pattern, rotate R2 bucket access. Wrangler can rebind to a fresh bucket; the old bucket goes private with a temporary CF API token revocation.
2.2 Forensic snapshot
Section titled “2.2 Forensic snapshot”- Snapshot the access log:
Terminal window pnpm exec wrangler d1 execute puccha-dev --remote --json \--command "SELECT * FROM audit_log WHERE event_type LIKE 'chat.attachment.%' AND at > datetime('now', '-7 days') ORDER BY at DESC" \> /tmp/incident-$(date -u +%FT%H%M%SZ)-attachment-audit.json - Snapshot R2 listing of affected prefix:
Terminal window pnpm exec wrangler r2 object list puccha-dev-uploads --prefix attachments/ --json \> /tmp/incident-$(date -u +%FT%H%M%SZ)-r2-listing.json - Save Sentry events from the relevant window (manually or via CLI).
- Preserve all of the above in a write-once incident folder — do NOT edit or delete during investigation.
2.3 Communication freeze
Section titled “2.3 Communication freeze”- All public communication routes through DPO + counsel. Engineering does not post status pages or Slack channels visible to customers without sign-off until §3 notification is dispatched.
3. Notification
Section titled “3. Notification”3.1 To the affected controller(s) — within 24 hours
Section titled “3.1 To the affected controller(s) — within 24 hours”Use this template, edit per incident:
Subject: [URGENT — Data Processing Notification] Puccha Chat Attachment Incident — {{tenant_id}}
We are writing to inform you of a data security incident that affected your account on {{date}}.
What happened: {{plain-language summary of the incident, no jargon, no defensiveness}}
What data was affected: {{specific categories — e.g., “between 10 and 250 image attachments uploaded by visitors of your widget between 2026-MM-DD and 2026-MM-DD”}}
What we have done: {{containment actions from §2}}
What you need to do:
- Notify your data subjects within your jurisdiction’s required SLA (PDPC: 72 hours; EU DPA: 72 hours; both from this notification)
- Determine if onward notification to data subjects is required under your privacy notice + applicable law
- We are available to assist you with regulator-facing technical detail
Our timeline for further information: a full incident report within 14 days.
Direct contact: {{name}}, DPO, {{email}}, {{phone}}.
3.2 To regulators — within 72 hours
Section titled “3.2 To regulators — within 72 hours”- Thailand PDPC: file via the PDPC online portal (
https://www.pdpc.or.th). Required content: nature of breach, categories + approximate count of affected subjects, contact for follow-up, mitigation taken, mitigation planned. - EU DPA(s): per controller’s home jurisdiction. Use Article 33 form templates; DPO holds the lead-DPA contact list.
- No regulator notification if breach is unlikely to result in risk to the rights and freedoms of data subjects (Art 33(1)) — but document the negative determination in writing with DPO sign-off.
3.3 To data subjects — only if required
Section titled “3.3 To data subjects — only if required”Direct notification to visitors is the controller’s responsibility, not ours. Where the controller asks for help drafting visitor-facing copy, DPO + counsel produce the localised text; engineering does not.
4. Investigation
Section titled “4. Investigation”Run in parallel with notification, not before. Outcome is the §5 incident report.
4.1 Root cause
Section titled “4.1 Root cause”Standard 5-whys on the access-log evidence. Common patterns:
- Auth gate misconfigured (e.g., visitor cookie comparison wrong)
- Prefix allowlist drift (e.g.,
attachments/accidentally globbed wider) - Sub-processor breach — investigation handoff to that sub-processor’s security team
- ATO / leaked agent session — credential rotation + MFA enforcement review
4.2 Scope determination
Section titled “4.2 Scope determination”- Number of affected visitors (
COUNT DISTINCT visitor_cookie_idfrom audit-log query) - Number of affected attachments
- Number of affected tenants
- Time window of exposure
- Confirmed external access (vs theoretical exposure)
4.3 Evidence chain
Section titled “4.3 Evidence chain”- All findings reference the §2.2 snapshots by file hash
- DPO + counsel have read-only access for the duration of the investigation
- Engineering does not alter audit log entries (ADR-0023 immutability is the gate)
5. Incident report (within 14 days of notification)
Section titled “5. Incident report (within 14 days of notification)”A report at docs/security/incidents/YYYY-MM-DD-attachments.md containing:
- Timeline (every relevant event in UTC)
- Root cause analysis
- Scope (numbers, categories, geographies)
- Mitigation actions taken
- Mitigation actions planned (with owners + dates)
- Permanent control changes (often a follow-up ADR)
- Sign-off from CTO + DPO
6. Post-incident
Section titled “6. Post-incident”6.1 Remediation
Section titled “6.1 Remediation”- Permanent fix lands as a normal PR with full code review
- Hotfix from §2.1 is reverted in the same PR (so the disable is not load-bearing)
- New regression test against the failure mode
6.2 Lessons learned
Section titled “6.2 Lessons learned”- Internal blameless retro within 7 days of close
- DPIA review triggered (per
docs/compliance/dpia-attachments.md§6) - ADR-0043 amended if architecture changed
7. Tooling references
Section titled “7. Tooling references”- Audit log:
audit_logtable in D1 - R2:
puccha-dev-uploadsbucket,attachments/prefix - Wrangler CLI for D1 + R2 reads from
platform/apps/app - Sentry: project
puccha-app, environmentprodanddev - DPO contact list:
docs/compliance/contacts.md(to be authored)
8. Drills
Section titled “8. Drills”- Tabletop exercise quarterly
- Live drill annually (synthesised breach against a staging tenant)
- Each drill produces a post-mortem at
docs/security/drills/YYYY-MM-DD.md
This runbook supersedes any verbal or chat-based incident process. Where ambiguity exists, the runbook wins.