Skip to content

Feature/Fix: Diagnose Codex reset-epoch token collisions#2152

Closed
iam-brain wants to merge 13 commits into
steipete:mainfrom
iam-brain:iam-brain/lineage-reset-epoch-diagnostics
Closed

Feature/Fix: Diagnose Codex reset-epoch token collisions#2152
iam-brain wants to merge 13 commits into
steipete:mainfrom
iam-brain:iam-brain/lineage-reset-epoch-diagnostics

Conversation

@iam-brain

@iam-brain iam-brain commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Stack dependency

This is slice 13 of 15. It follows Feature/Fix: Validate lineage accounting on frozen local histories; review and merge that PR first.

Summary

  • add a pure, shadow-only analyzer for exact token snapshots repeated after strong cumulative-counter resets
  • separate same-document reset evidence from speculative cross-owner fork collisions
  • report only aggregate counts and UTC token estimates; the analyzer cannot emit accounting rows, cache data, or selector input
  • add an opt-in frozen-history replay that keeps private paths, rollout identities, models, and account totals out of output

Real-world result

This does not explain the remaining July 9 undercount. The frozen corpus produced zero reset-epoch candidate tokens on July 9.

Across the wider replay, 97 same-document repeats represented about 16.3M counterfactual tokens: roughly 7.3M on July 10 and 6.2M on July 11. Cross-owner detection produced 96,008 candidates and a clearly implausible estimate dominated by copied fork history, confirming that it is diagnostic evidence rather than a safe accounting rule.

Selected totals remain unchanged.

Tests

  • focused reset/shadow/local-validation suites (9 tests passed after review fixes)
  • make check
  • opt-in frozen replay over 819,779 observations
  • make test (648 selections in 54 groups; one group timed out under concurrent SwiftPM contention and all 12 isolated selections recovered; command passed)

Related investigation: steipete/CodexBar#2037.

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 14, 2026
@clawsweeper clawsweeper Bot added status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal priority bug or improvement with limited blast radius. labels Jul 14, 2026
@clawsweeper

clawsweeper Bot commented Jul 14, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 13, 2026, 9:21 PM ET / July 14, 2026, 01:21 UTC.

Summary
Adds experimental Codex lineage-accounting infrastructure, shadow-mode integration, a reset-epoch token-collision analyzer, aggregate replay validation, and focused tests.

Reproducibility: yes. for the diagnostic behavior: focused tests define the reset-and-repeat path, and the PR reports a frozen-history replay that found same-document candidates. That replay also shows this theory does not reproduce the remaining July 9 undercount.

Review metrics: 3 noteworthy metrics.

  • Current stack surface: 21 files, +4,957/-19. Most of the present diff is prerequisite lineage infrastructure, making dependency-order review important.
  • Reported replay corpus: 819,779 observations. The large aggregate-only replay materially supports the diagnostic behavior without exposing private rollout data.
  • Diagnostic result: 97 same-document repeats. The replay found meaningful reset-epoch evidence while finding zero candidates for the investigation's July 9 undercount.

Root-cause cluster
Relationship: partial_overlap
Canonical: #2037
Summary: The canonical issue tracks the observable Codex fork-accounting problem; this PR tests one collision hypothesis but reports that it does not explain the remaining undercount.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🦞 diamond lobster
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

Risk before merge

  • [P1] The current review surface is nearly 5,000 added lines because the predecessor has not landed; the reduced slice should be reviewed again after Feature/Fix: Validate lineage accounting on frozen local histories #2151 is resolved.
  • [P1] The diagnostic is safely shadow-only, but the combined stack also changes scanner parsing and introduces lineage accounting modes, so legacy totals, cache isolation, cancellation, and large-history resource bounds remain merge-critical validation surfaces.

Maintainer options:

  1. Decide the mitigation before merge
    Keep legacy accounting authoritative by default, land the lineage slices in dependency order, and retain this analyzer as aggregate-only shadow evidence unless later promotion work independently proves accounting correctness and upgrade safety.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] Review and land the dependency stack in order, then reassess the reduced slice; no narrow autonomous repair finding is established on this head.

Security
Cleared: The branch adds local parsing, hashing, diagnostics, and tests without new dependency sources, permissions, secret access, downloaded executables, or release automation changes.

Review details

Best possible solution:

Keep legacy accounting authoritative by default, land the lineage slices in dependency order, and retain this analyzer as aggregate-only shadow evidence unless later promotion work independently proves accounting correctness and upgrade safety.

Do we have a high-confidence way to reproduce the issue?

Yes for the diagnostic behavior: focused tests define the reset-and-repeat path, and the PR reports a frozen-history replay that found same-document candidates. That replay also shows this theory does not reproduce the remaining July 9 undercount.

Is this the best way to solve the issue?

Yes for diagnosis: a pure shadow analyzer with aggregate-only output is safer and narrower than changing selected totals from speculative collision evidence. Merge review should occur after the predecessor lands so the diagnostic-specific diff can be evaluated independently.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 0437067deaa8.

Label changes

Label changes:

  • add P2: This is a bounded diagnostic improvement for an active Codex accounting investigation, with meaningful but non-emergency user-impact relevance.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides copied after-change live output from an opt-in frozen-history replay over 819,779 observations, reports concrete aggregate results, and confirms selected totals remained unchanged.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body provides copied after-change live output from an opt-in frozen-history replay over 819,779 observations, reports concrete aggregate results, and confirms selected totals remained unchanged.

Label justifications:

  • P2: This is a bounded diagnostic improvement for an active Codex accounting investigation, with meaningful but non-emergency user-impact relevance.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The PR body provides copied after-change live output from an opt-in frozen-history replay over 819,779 observations, reports concrete aggregate results, and confirms selected totals remained unchanged.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides copied after-change live output from an opt-in frozen-history replay over 819,779 observations, reports concrete aggregate results, and confirms selected totals remained unchanged.
Evidence reviewed

What I checked:

Likely related people:

What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@iam-brain

Copy link
Copy Markdown
Contributor Author

Superseded by the consolidated four-PR review stack. This slice is now included in #2158; closing this draft to keep the upstream review surface manageable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant