Skip to content

Runbook — Chat Attachment Data Breach

  • Severity tier: P0 (data exposure of visitor PII)
  • First responder: on-call engineer
  • Escalation: CTO + DPO within 60 minutes of breach confirmation
  • Notification SLA: 24 hours to controller (tenant), 72 hours to PDPC / EU DPA
  • Triggering events: see §1

A breach is suspected when any of these are observed:

  • Unexpected 403 → 200 flip in /uploads/[...path] access logs (auth bypass)
  • R2 audit log shows reads from IPs / regions inconsistent with normal traffic
  • Sentry alert on chat.attachment.read audit-row anomaly (volume / cross-tenant pattern)
  • External report (security researcher, customer, regulator) of attachment URL leakage
  • Sub-processor breach disclosure (Cloudflare / Anthropic / Cohere / Sentry security notice)
  • Internal discovery of /uploads/ URL in a public corpus (search engine, paste site, GitHub)

Move fast on R2 and access controls before you write notification copy.

  1. Disable the upload + read paths: deploy a hotfix that returns 503 from /api/chat/uploads and /uploads/[...path] for the attachments/ prefix:
    Terminal window
    git checkout -b hotfix/disable-attachments
    # edit the two route handlers to return error(503, 'Maintenance')
    gh pr create --title 'hotfix: temporary disable chat attachments — incident' --body 'See incident-YYYY-MM-DD'
    gh pr merge --squash # bypass review for P0
  2. Verify rollout via curl -I https://puccha-dev.hxlab.io/uploads/avatars/_test_ and curl -X POST https://puccha-dev.hxlab.io/c/_/api/chat/uploads.
  3. If the leak is via an R2 URL pattern, rotate R2 bucket access. Wrangler can rebind to a fresh bucket; the old bucket goes private with a temporary CF API token revocation.
  1. Snapshot the access log:
    Terminal window
    pnpm exec wrangler d1 execute puccha-dev --remote --json \
    --command "SELECT * FROM audit_log WHERE event_type LIKE 'chat.attachment.%' AND at > datetime('now', '-7 days') ORDER BY at DESC" \
    > /tmp/incident-$(date -u +%FT%H%M%SZ)-attachment-audit.json
  2. Snapshot R2 listing of affected prefix:
    Terminal window
    pnpm exec wrangler r2 object list puccha-dev-uploads --prefix attachments/ --json \
    > /tmp/incident-$(date -u +%FT%H%M%SZ)-r2-listing.json
  3. Save Sentry events from the relevant window (manually or via CLI).
  4. Preserve all of the above in a write-once incident folder — do NOT edit or delete during investigation.
  • All public communication routes through DPO + counsel. Engineering does not post status pages or Slack channels visible to customers without sign-off until §3 notification is dispatched.

3.1 To the affected controller(s) — within 24 hours

Section titled “3.1 To the affected controller(s) — within 24 hours”

Use this template, edit per incident:

Subject: [URGENT — Data Processing Notification] Puccha Chat Attachment Incident — {{tenant_id}}

We are writing to inform you of a data security incident that affected your account on {{date}}.

What happened: {{plain-language summary of the incident, no jargon, no defensiveness}}

What data was affected: {{specific categories — e.g., “between 10 and 250 image attachments uploaded by visitors of your widget between 2026-MM-DD and 2026-MM-DD”}}

What we have done: {{containment actions from §2}}

What you need to do:

  • Notify your data subjects within your jurisdiction’s required SLA (PDPC: 72 hours; EU DPA: 72 hours; both from this notification)
  • Determine if onward notification to data subjects is required under your privacy notice + applicable law
  • We are available to assist you with regulator-facing technical detail

Our timeline for further information: a full incident report within 14 days.

Direct contact: {{name}}, DPO, {{email}}, {{phone}}.

  • Thailand PDPC: file via the PDPC online portal (https://www.pdpc.or.th). Required content: nature of breach, categories + approximate count of affected subjects, contact for follow-up, mitigation taken, mitigation planned.
  • EU DPA(s): per controller’s home jurisdiction. Use Article 33 form templates; DPO holds the lead-DPA contact list.
  • No regulator notification if breach is unlikely to result in risk to the rights and freedoms of data subjects (Art 33(1)) — but document the negative determination in writing with DPO sign-off.

Direct notification to visitors is the controller’s responsibility, not ours. Where the controller asks for help drafting visitor-facing copy, DPO + counsel produce the localised text; engineering does not.

Run in parallel with notification, not before. Outcome is the §5 incident report.

Standard 5-whys on the access-log evidence. Common patterns:

  • Auth gate misconfigured (e.g., visitor cookie comparison wrong)
  • Prefix allowlist drift (e.g., attachments/ accidentally globbed wider)
  • Sub-processor breach — investigation handoff to that sub-processor’s security team
  • ATO / leaked agent session — credential rotation + MFA enforcement review
  • Number of affected visitors (COUNT DISTINCT visitor_cookie_id from audit-log query)
  • Number of affected attachments
  • Number of affected tenants
  • Time window of exposure
  • Confirmed external access (vs theoretical exposure)
  • All findings reference the §2.2 snapshots by file hash
  • DPO + counsel have read-only access for the duration of the investigation
  • Engineering does not alter audit log entries (ADR-0023 immutability is the gate)

5. Incident report (within 14 days of notification)

Section titled “5. Incident report (within 14 days of notification)”

A report at docs/security/incidents/YYYY-MM-DD-attachments.md containing:

  • Timeline (every relevant event in UTC)
  • Root cause analysis
  • Scope (numbers, categories, geographies)
  • Mitigation actions taken
  • Mitigation actions planned (with owners + dates)
  • Permanent control changes (often a follow-up ADR)
  • Sign-off from CTO + DPO
  • Permanent fix lands as a normal PR with full code review
  • Hotfix from §2.1 is reverted in the same PR (so the disable is not load-bearing)
  • New regression test against the failure mode
  • Internal blameless retro within 7 days of close
  • DPIA review triggered (per docs/compliance/dpia-attachments.md §6)
  • ADR-0043 amended if architecture changed
  • Audit log: audit_log table in D1
  • R2: puccha-dev-uploads bucket, attachments/ prefix
  • Wrangler CLI for D1 + R2 reads from platform/apps/app
  • Sentry: project puccha-app, environment prod and dev
  • DPO contact list: docs/compliance/contacts.md (to be authored)
  • Tabletop exercise quarterly
  • Live drill annually (synthesised breach against a staging tenant)
  • Each drill produces a post-mortem at docs/security/drills/YYYY-MM-DD.md

This runbook supersedes any verbal or chat-based incident process. Where ambiguity exists, the runbook wins.