Skip to content

fix(gateway): preserve expiration sweep coverage - #468

Merged
Menci merged 3 commits into
mainfrom
fix/expiration-sweep-integrity
Aug 12, 2026
Merged

fix(gateway): preserve expiration sweep coverage#468
Menci merged 3 commits into
mainfrom
fix/expiration-sweep-integrity

Conversation

@Menci

@Menci Menci commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Summary

D1 reports meta.changes from SQLite total_changes(), which includes writes performed by trigger programs. Dump and spilled Responses deletion therefore returned a trigger-amplified count rather than the number of source rows deleted. Inactive cleanup could mistake a full batch for a drained key and remove its expiration_sweeps row while records remained.

  • Count dump, Responses item, and snapshot deletions from DELETE ... RETURNING result rows.
  • Add migration 0082_expiration_sweep_integrity.sql.
  • Reject deletion of a sweep row while its dump or Responses domain still contains rows.
  • Make the sweep domain and key identity immutable so coverage cannot be reassigned.
  • Re-enqueue existing dump and Responses keys that are missing queue coverage without changing existing due times, revisions, or claims.
  • Keep the one-time legacy backfill for descriptor registration; no recurring full-table reconciliation is added.

The migration installs the deletion guard before repairing coverage, so the pre-deploy Worker cannot recreate the gap during a migrate-then-deploy rollout.

Test Plan

  • Trigger-amplified D1 change-count regressions for dump and spilled Responses deletion.
  • Migration repair preserves existing queue state and excludes empty keys.
  • Dump and Responses queue deletion is rejected until the final domain row is gone, and queue identity updates are rejected.
  • Repaired deleted and retention-disabled dump keys drain in bounded ticks.
  • Actual Node instance: migration repairs a missing queue; scheduled ticks drain 75 → 25 with the queue retained, then 25 → 0.
  • Actual workerd/local D1 instance: old code reproduces meta.changes = 150 for 50 dump rows and loses the queue; current code repairs and drains 70 → 20 → 0.
  • pnpm run verify — 544 test files and 5,654 tests passed, plus lint, typecheck, installer harness, generated assets, AGENTS validation, verification parity, and web build.

Menci added 3 commits August 12, 2026 07:05
Count target rows from DELETE RETURNING because D1 meta.changes includes trigger side effects and can make bounded cleanup report the wrong batch size.\n\nAdd a database invariant that rejects queue deletion while domain rows remain, then re-enqueue every existing dump and Responses key missing coverage. Cover trigger-amplified counts, migration repair, guarded completion, and bounded inactive cleanup.
Prevent queue identity updates from bypassing the source-row guard. Exercise each migration source and preservation path independently, calibrate D1 change-count instrumentation, and cover active and inactive trigger-amplified deletions.
Enforce queue identity as an unconditional database invariant, reject missing change-count instrumentation, and attach the complete D1 and SQLite evidence chain to exact deletion accounting.
@Menci
Menci merged commit 87ac6e0 into main Aug 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant