Repository navigation
perf(reporting-storage): drop the per-object advisory lock from deletes and self-recorded adoption - #3606
Merged
Conversation
…es and self-recorded adoption
Retirement, compaction and the write-intent sweep no longer take
adcp-reporting-object:{binding}:{objectRef}, and no longer hold a
transaction or pooled connection across the provider delete. A commit takes
the lock only for an adopted object its own intent did not record. The orphan
collector keeps it, so the pair still excludes each other.
Refs #3281
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…n the proof wording Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
bokelley
enabled auto-merge (squash)
November 3, 2026 02:12
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — removes the per-object advisory lock from reporting-storage delete/adopt paths while preserving it for the orphan collector and for commits adopting an object their own intent did not record.
What I checked:
- Lock-free delete at postgres.ts:357 verified safe: phase-1
pruningcommitted under the account lock orders concurrent retirements/compactions (fence 4), and unique{revision_key}keys ensure only the owning row set adopts via a writable binding (fence 1). deleteOrphanedObject(L398) correctly retains the every-binding lock since no record names an orphan.- Recorded-adopt skip (L493) still proves via
assertObjectsPresent(unchanged base). - Extracted
isRowObjectReferencedSQL is byte-identical to the prior inline query — faithful refactor, no wire/shape change. - Changeset present and correctly scoped (
@adcp/reporting-storage: patch); patch level justified — no API/schema/wire change, lock key text unchanged, rolling upgrade keeps the lock on both sides of every pair still needing mutual exclusion. - ~12 new integration tests cover each side of the contract.
No Critical/High/Medium findings. high_risk is false, gated_paths is false, no author team gate. Falls through to row 9 — approve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3281
What changed
The per-object advisory lock
adcp-reporting-object:{binding}:{objectRef}is removed from retirement, compaction and the write-intent sweep, and from commits that adopt an object their own intent recorded. It stays in two places that pair with each other, because nothing else orders them: the orphan collector's delete, and a commit that adopts an object its own intent did not record. Decision: partial removal, not full removal and not a close-with-analysis (removing it everywhere is unsafe; see case 3 below).What it buys: retirement, compaction and the sweep no longer run each object as a transaction holding a pooled connection and an advisory lock across the provider delete (the check is a plain query, then the version-pinned delete). A retry that adopts its own recorded object pays no lock round trip.
Proof, case by case
Fences that exist independent of the lock: (A) account lock held by the commit, retirement phase 1, compaction phase 1, intent takeover and sweep claim; (B)
rows_statemoves topruningunder the account lock before any delete, andpruningrow sets neither count as references nor can be shared; (C) the intent movesopen -> sweepingagainst the commit'sDELETE ... WHERE state = 'open', which is in the same transaction as the chunk-row INSERT (so a reader sees the open intent or the chunk rows, never neither); (D) deletes name an exact native version; (E) every writable key includes{revision_key}(sha256 of the row set ID) and{content_sha256}, so one row set alone can name a key.rows_shared_from_row_set_id)FOR SHAREs a predecessor that must belive; retirement/compaction mark itpruningfirst. The sharer commits first (its chunk rows make the check skip) or failsSTATE_UNAVAILABLEand re-uploads under its own key. The lock never covered this: the sharing path took none.sweepingintent.pruningbefore its check and counts only non-pruning row sets, so for any two at least one deletes (commit-order argument in the TSDoc ondeleteUnreferencedObjects). readOnly bindings never take commits, so nothing adopts through them.{revision_key}was requiredreadOnlybindings (writable ones are refused at config time), so no new write can collide with them (E needs a 256-bit coincidence).Known precondition, pre-existing and not changed by this PR: bindings that share one bucket and prefix must route a row set to the same binding across attempts. The old lock keys were per binding, so they did not order A's sweep against B's adopter either. Now documented in
REPORTING-LEDGER.md.Final contract (for the Python SDK, adcp-client-python#1492, to mirror)
live/unavailablerow set outside the operation (outside the obligation for retirement, outside the row set for compaction, none excluded for the sweep) references the same binding,object_refandnative_version_ref, and noopenintent in the binding lists it increated_objects(read bothobject_ref/native_version_refand the pre-alignmentobjectKey/nativeVersion); then delete that exact version.adcp-reporting-object:+ binding ID +:+ object ref, passed topg_advisory_xact_lock(hashtextextended(text, 0)).DELETE ... WHERE ... AND owner_token = $ AND state = 'open' RETURNING created_objects(zero rows means failSTATE_UNAVAILABLE); for each adopted object (sorted by object ref) whose(object_ref, native_version_ref)is not in the returnedcreated_objects, take the lock for its own binding; then prove every adopted object with a version-pinnedstat(full read fallback if the provider has nostat); then insert row set and chunk rows. Objects the intent recorded take no lock.write-unique-version).Tests
reporting-row-shared-object-pg.test.js: the old "a delete waits for the object lock" test is inverted. New: retirement, compaction and the sweep complete while the object lock is held elsewhere and hold no pooled connection across the delete; self-retry adoption (current keys and pre-alignment keys) takes no lock; adoption with a different native version, or of an object the intent does not name, still takes it; sweep vs an expired-intent writer; takeover renews against the sweep; a retry cannot take over asweepingintent; an orphan delete in flight blocks an unrecorded adopter, which fails cleanly; header-only sharer loses a race to compaction of its predecessor and re-uploads; two retirements of readOnly legacy row sets held at a barrier until both arepruningalways free the object. Mutation-checked: countingpruningrow sets in the check, and droppingrows_state = 'live'from the sharer'sFOR SHARE, each fail a test. Full reporting-storage suite (parallel, serial, chunked) green against PG; typecheck, lint, prettier, doc-links, package-size green.Expert review
Run read-only on the diff:
security-reviewer,code-reviewer-deep,nodejs-testing-expert, plusnpm run review:codex -- --all --base main(its sandbox cannot read the repo, so the diff was piped on stdin; verdicts: protocol "ships", code "no findings", security "safe under the stable-binding contract").Fixed: proof wording scoped to writable bindings and the replay-race nuance (a late writer can adopt a just-committed object but its commit inserts nothing); stale-deleter safety documented as depending on write-unique native versions;
retirePeriodcomment says no lock orders the shared-legacy case;RecordedIntentObjectTSDoc misplacement; onerecordedIntentObjectshelper for the two key spellings; a namedRowObjectQueryabletype; test hook reentrancy and leaks (afterEachreset, hook cleared before it is awaited); pending work awaited after a timeout; parallel readOnly retirement test now forces the worst interleaving with a barrier; test names andno longer liveassertion tightened; added the legacy-key and wrong-version adoption cases; "Cross-SDK contract" softened to the intended contract.Declined: the security reviewer and codex both flagged that the reference check is per binding, so two bindings sharing one bucket, prefix and template could alias a key if a row set is routed to different bindings across attempts. Pre-existing (the old per-binding lock keys did not order those two bindings either), widening the check to every binding would not close a check-then-delete race, and only a lock on every path would. Documented as a precondition instead; enforcing it (refusing two writable bindings that alias one key space) is a configuration-breaking change that belongs in its own PR.
Changeset: patch. No exported API, schema or wire behavior changes, and the lock key text is unchanged, so rolling upgrades between a version that locks everywhere and this one are safe (every pair that needs exclusion keeps the lock on both sides). No migration needed.
🤖 Generated with Claude Code