ADR-0044: `admin_audit_log` retention ceiling

Status: Accepted
Date: 2026-08-04
Deciders: Omar ElHawary

Context

admin_audit_log had no expiry. Nothing in src/memberintel/ deleted from it,
and no cascade reached it, so every row written since the table shipped
(2026-05-22) was retained by construction.

Seven of its 30 action types write a bare email address or operator free text
into the JSONB payload, and none of those strings is reachable by any existing
deletion path:

Action	Payload keys	FK reach
contact_submission.erased_by_email	email, rows_deleted, reason	none — no target_user_id by design (#790: erased rows may span zero or many accounts)
contact_submission.resolved	note (operator free text)	target_user_id, SET NULL — and this is the erasure-path table itself: the submission it describes is swept at 90 days (#790), so this row must not outlive it by a factor of four
user.email_changed	old_email, new_email, google_link_cleared	target_user_id, ON DELETE SET NULL — strings survive account deletion
user.email_change_notice_failed	old_email	same
user.invited	email	target_resource_id = invite id; an invitee may never become a user
user.account_status_changed	from_status, to_status, reason (operator free text)	target_user_id, SET NULL
escalation.resolved	note (operator free text)	target_user_id, SET NULL — an admin writing prose about one identified member’s chat

The sharp case is the first. Fulfilling a GDPR/CCPA erasure request deleted the
contact_submissions rows and then created a permanent record naming the
person who asked to be erased. #938 widened that record with an
operator-authored free-text reason — capped at 1000 chars, which bounds size,
not lifetime.

The counsel first-pass
(docs/superpowers/reviews/2026-07-31-counsel-erase-by-email-reason.md) listed
this as the branch’s real residual risk and as must-change-before-prod, and
rejected hashing the email as a fix: a hash of one known candidate address is
confirmable by whoever already suspects it, and it destroys the row’s
operational purpose.

Decision

admin_audit_log is an internal operations and security-forensics log. It is
not compliance evidence for rights-request fulfilment. If the org ever wants
that posture it gets a dedicated structured table with a verified request
reference — a materially larger build, deliberately not this one.

That framing is what makes a bounded ceiling defensible at all. Under a
record-of-processing framing the Art. 17(3)(b) argument pushes retention
longer — you cannot demonstrate you honored an erasure without keeping some
record of it — which is why “shorter is better” is not self-evident here. We do
not make that claim, so we do not need the record.

Two ceilings, by payload sensitivity, in
src/memberintel/api/admin/retention.py:

• RETENTION_AGE_DAYS = 365 — every ordinary action. Matches the 365-day
  retention already codified for the GCP audit-log sink (memberintel_audit
  BigQuery dataset, infra/modules/project_globals/bigquery.tf, per ADR-0010),
  so both audit trails share one number.
• SENSITIVE_RETENTION_AGE_DAYS = 90 — the seven actions above. Matches the
  #790 contact_submissions sweep that erased the underlying data, and the
  90-day analytics log sink. An audit row describing an erasure should not
  outlive the data it describes by a factor of four.

user.account_status_changed, contact_submission.resolved, and
escalation.resolved are sensitive on the strength of their free-text
reason/note fields, not because they store an address. None carries an
email by construction, but nothing stops an operator typing one. The latter
two were caught by the final whole-branch review, not the original spec: the
spec’s criterion (“an email address or operator free text”) already
covered them, and applying it only to user.account_status_changed while
leaving two structurally identical fields in the 365-day bucket was an
inconsistency, not a deliberate distinction.

The bucket is declared per action as the fourth element of _ACTION_MAP
(src/memberintel/api/admin/audit_view.py), not as a separate list.
tests/unit/test_audit_action_coverage.py already fails CI when a logged
action is missing from that map, so some bucket is mandatory for any new
action — but registration is not classification. The guard forces an author
to type a bucket; it cannot force the right one, and the two actions above
passed every gate (the coverage guard, this ADR’s own bucket test, mypy,
ruff) while misclassified. Catching that requires a human reading payloads
against the criterion, not just a green CI run.

A second, narrower gap: SENSITIVE_ACTIONS is matched by action string —
AdminAuditLog.action.in_(SENSITIVE_ACTIONS) / .notin_(...). If a sensitive
action is ever renamed (e.g. user.invited → user.invitation_sent), every
row already written under the old string silently reverts from the 90-day
bucket to the 365-day one on the very next sweep, because the NOT IN
predicate matches current strings, not row provenance. CI’s coverage guard
checks call sites, not table contents, so it cannot catch this. A rename that
needs to preserve the old retention would need a one-time backfill or a
translation table — not built here, and not needed unless a rename actually
happens.

A daily Cloud Scheduler job (07

UTC) calls
POST /internal/admin-audit-retention-sweep (OIDC, ADR-0022), which runs two
age-bounded DELETEs and returns per-bucket counts. Operator procedure:
docs/runbooks/20-admin-audit-retention.md.

This ADR is not legal sign-off. The counsel first-pass flagged the ceiling
as needing a human ruling from Allen or outside counsel. It is the written
rationale such a signer would countersign, and it is sufficient because the ADR
does not claim the log as compliance evidence — the question the ruling was
needed for.

Consequences

Positive:

• The largest open privacy gap on the erasure path closes: an erased person’s
  address leaves admin_audit_log 90 days after the erasure instead of
  never. It does not leave the system — the support-inbox notification for
  the original submission (src/memberintel/api/contact/router.py:157) is
  outside this table’s control, and account_status_history.reason is a
  separate free-text surface the spec explicitly scoped out. This ADR only
  ever claims the one table.
• One audit-retention number across the Postgres trail and the GCP sink.
• Bounded table growth, indefinitely.
• A future action cannot be added to _ACTION_MAP without some retention
  bucket — omission is impossible. Whether it’s the right bucket still
  needs a human to check the payload against the sensitivity criterion; see
  the coverage-guard limitation above.

Negative / costs:

• Forensic lookback on ordinary admin actions is now capped at 365 days. An
  incident discovered more than a year later has no trail.
• Erasure records are gone after 90 days, so the log cannot evidence an
  erasure older than that — acceptable only because this ADR states the log is
  not that evidence.
• _ACTION_MAP entries are now 4-tuples; every new action carries one more
  decision.

Mitigations:

• The GCP audit sink (365 days, IAM-level API calls) and Cloud Logging
  analytics sink (90 days) remain independent trails for incident work.
• Both ceilings are module-level constants with per-request overrides on the
  internal endpoint, so ops can sweep more aggressively without a deploy.
  Tightening needs no ADR change; loosening or removing does.

Known tension — global_brain.candidate_rejected stays in the 365-day
bucket on purpose, and that has a cost:

Its payload is {"reason": <operator free text>, "provenance": <pre-scrub source linkage>} (src/memberintel/api/admin/brain_candidates.py:142). By
this ADR’s own criterion the free-text reason argues for 90 days. But
ADR-0014’s three-roles model requires that source linkage never appear in
customer-facing global-brain content — approved entries carry only
non-identifying metadata (candidate_review.py:220-224), and a rejected
candidate’s content/provenance are scrubbed from the global_brain_candidates
row itself (candidate_review.py:247-249). This codebase’s way of keeping
that scrubbed provenance traceable for a reviewer without violating the
boundary is to hand the pre-scrub value to the caller so it can be logged —
per the code comment at brain_candidates.py:134-135, “the source linkage is
recorded in the audit row only (ADR-0014).” For a rejected candidate, this
audit row is therefore the only surviving copy of that linkage. A 90-day
sweep would destroy it, so this action stays “general” — the reason
free text rides along as a consequence of protecting the provenance, not
because the free text itself is judged safe to keep for a year. Decoupling
the two lifetimes — dropping reason from the payload and logging it
separately, or giving the provenance its own durable record outside
admin_audit_log — is a real fix but is deferred; this ADR just records that
the tension exists and was seen.

Known gap — the brain.* content-snapshot actions were never evaluated
against this ADR’s criterion, and that gap is deliberate but open:

brain.soul_edited, brain.bible_edited, brain.heartbeat_regenerated,
brain.memory_edited, and brain.memory_deleted
(src/memberintel/api/admin/brain.py:202,255,306,458,503) all log
{"before": <full text>, "after": <full text>} — full document snapshots,
not the bare identifiers or short free-text fields the sensitive bucket was
designed around. brain.memory_deleted is the sharpest case: deleting a
memory is the closest thing the product has to a per-memory erasure, and it
writes a verbatim, unexpiring copy of that member-derived content into
admin_audit_log at the general 365-day ceiling, outside whatever ADR-0018’s
memory limits and sanitization otherwise govern. None of these actions
change bucket here — an admin’s own tenant/brain content is a different risk
class from a third party’s email address, and #940 is scoped to personal
data on the erasure path, not customer-owned brain content. But this was not
considered by the original spec or this ADR when written, only surfaced by
the final whole-branch review, and whether brain-content snapshots need
their own retention ceiling is an open question this ADR does not answer.

Alternatives considered

• Hash the email in the payload — rejected by the counsel first-pass. A
  hash of one known candidate address is confirmable by anyone who already
  suspects it, so it is not meaningful de-identification, and it destroys the
  row’s purpose: a human reviewing the trail needs to read who it was about.
• Redact the payload at 90 days, keep the row to 365 — strip identifying
  keys, leave {rows_deleted, redacted_at}. Better privacy-per-forensic-value
  on paper, but under an ops-log framing a row whose identifying content is
  gone has no value, and it costs an UPDATE pass plus a redacted-row rendering
  path in _ACTION_MAP.
• One uniform ceiling for all 30 actions — simplest possible sweep, but a
  single number has to serve both a year of forensic lookback on
  user.admin_changed and not holding an erased person’s address for a year.
  Those pull apart; two buckets is where the real distinction is.
• Per-action ceilings for all 30 — thirty numbers to maintain for a
  distinction with two sides.
• Retain indefinitely, by design, documented — the honest alternative if
  the log were compliance evidence. It is not, so there is nothing to justify
  the retention.