Skip to content

feat: add offline adaptive-refresh replay tooling#2029

Merged
steipete merged 12 commits into
steipete:mainfrom
hhh2210:codex/adaptive-refresh-replay-upstream
Jul 12, 2026
Merged

feat: add offline adaptive-refresh replay tooling#2029
steipete merged 12 commits into
steipete:mainfrom
hhh2210:codex/adaptive-refresh-replay-upstream

Conversation

@hhh2210

@hhh2210 hhh2210 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

Add an offline AdaptiveReplayCLI for comparing refresh-timing policies against caller-supplied JSONL traces. Its supporting targets are package-internal and share the production adaptive decision table through the Foundation-only AdaptiveRefreshCore target.

Refs #1739 and #1861.

Scope

  • Reads an explicit trace path and writes table or JSON reports to stdout.
  • Compares the production adaptive policy, an activity-cap experiment, fixed interval baselines, and manual refresh.
  • Reports simulated cadence, menu-open staleness, constrained-state compliance, activity coverage, legacy-gap segmentation, and an independent audit of recorded schedule events.
  • Does not record app behavior, write trace files, scan session directories, call providers, change settings, or alter the production refresh policy.

Maintainer refinements

  • Extracted the production decision table into one shared core instead of maintaining a replay copy.
  • Restricted policy arguments to the seven documented names and reject invalid intervals before reading the trace.
  • Precomputed signal timestamp indexes to avoid repeated per-decision array allocation.
  • Made CRLF parsing, gap segmentation, schedule multiplicity, and boundary-censored staleness explicit and testable.
  • Anchored staleness to the later of the first recorded refresh and any simulated refresh.
  • Audited unequal serialized schedule dates as authoritative; the exact lead only disambiguates equal whole-second timestamps.

Validation

Maintainer validation at exact head b231503a53020acad599672717ef80e0207544fd:

  • 624 selections across 52 groups passed on the first attempt, with zero retries or timeouts.
  • Focused adaptive replay, replay segmentation, and recorded-schedule audit suites passed.
  • make check passed with zero SwiftLint violations across 1,402 files.
  • Maintainer autoreview is clean after two review/fix cycles (0.90 confidence).
  • Real CLI smoke: table and JSON output produced expected policy metrics; malformed JSONL exited 1 with its line number and content.

The contributor's private 1,780-record replay remains local and hash-pinned. It found 28 observed segments; current Adaptive scheduled 49.8% fewer simulated refreshes than fixed five minutes, while the activity candidate did not improve p95 staleness. This evidence does not support a default or production-policy change.

Limitations

  • Replay is counterfactual and assumes zero-duration refreshes.
  • Legacy-gap segmentation cannot identify sleep, reboot, long refreshes, or event-loop stalls.
  • Results do not measure battery use, elapsed user time, provider requests, or cost.
  • Account-authenticated or provider live testing is intentionally irrelevant to this offline tool.

@hhh2210 hhh2210 force-pushed the codex/adaptive-refresh-replay-upstream branch from a70eb5e to daef5ee Compare July 10, 2026 06:07
@hhh2210 hhh2210 marked this pull request as ready for review July 10, 2026 06:10
Copilot AI review requested due to automatic review settings July 10, 2026 06:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces an opt-in, local-only adaptive-refresh trace + replay harness intended to collect measured evidence about refresh policy behavior (including segmentation of unobserved gaps, replay against baseline/candidate policies, and auditing recorded schedule decisions) without changing production defaults or enabling any network/analytics path.

Changes:

  • Adds AdaptiveReplayKit (Foundation-only) with trace parsing, replay simulation, legacy gap segmentation, metrics, activity-coverage stats, and recorded-schedule auditing.
  • Adds AdaptiveReplayCLI as a thin wrapper to run replay/audit locally and emit tabular or JSON reports.
  • Adds an off-by-default in-app recorder (adaptiveRefreshTraceEnabled) plus a stat-only local coding-activity probe sampled only when tracing is enabled.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Tests/CodexBarTests/ReplayTraceSegmentationTests.swift Adds tests for legacy gap segmentation behavior and segmented replay rate math.
Tests/CodexBarTests/RecordedScheduleAuditTests.swift Adds tests for auditing live timer-advance evaluations/advances and linkage correctness.
Tests/CodexBarTests/CodingActivityReplayPolicyTests.swift Tests replay-only candidate policy that caps unconstrained active decisions at 5 minutes.
Tests/CodexBarTests/CodingActivityProbeTests.swift Tests stat-only activity probe over fake home directories (Codex + Claude).
Tests/CodexBarTests/ClaudeOAuthCredentialsStoreCLIStorageOwnershipTests.swift Adjusts test harness overrides wiring for Claude keychain/CLI read behavior.
Tests/CodexBarTests/AdaptiveReplayTraceParserTests.swift Adds trace JSONL parsing/compatibility tests (A-layer + B-layer fields, strict vs tolerant).
Tests/CodexBarTests/AdaptiveReplayPolicyMirrorTests.swift Ensures mirrored replay policy + in-app adapter match production policy decisions.
Tests/CodexBarTests/AdaptiveReplayEngineTests.swift Adds hand-computed replay engine metric tests and interaction-advance behavior tests.
Tests/CodexBarTests/AdaptiveRefreshTraceWriterTests.swift Tests writer byte-cap behavior and oversized-record skipping.
Tests/CodexBarTests/AdaptiveRefreshTraceRecordingTests.swift Anchors in-app recorder call sites via trace-content assertions (serialized suite).
Tests/CodexBarTests/ActivityCoverageStatsTests.swift Adds tests for informational activity-coverage stats computed from raw trace records.
Sources/CodexBar/UsageStore+AdaptiveRefresh.swift Records adaptive decisions (and optional activity sample) when tracing is enabled.
Sources/CodexBar/UsageStore.swift Records refresh completion; records timer-advance evaluations always and advances when accepted.
Sources/CodexBar/StatusItemController+Menu.swift Records menu-open events for the replay harness when tracing is enabled.
Sources/CodexBar/CodingActivityProbe.swift Implements bounded, stat-only activity probe for Codex/Claude transcript metadata.
Sources/CodexBar/AdaptiveRefreshTraceRecording.swift Adds opt-in recorder writing JSONL trace records into Application Support with byte cap.
Sources/CodexBar/AdaptiveRefreshPolicy+ReplayAdapter.swift Adapts production policy to ReplayPolicy for in-app replay tooling.
Sources/AdaptiveReplayKit/ReplayTraceSegmentation.swift Implements legacy gap segmentation + segmented replay wrapper for ReplayEngine.
Sources/AdaptiveReplayKit/ReplayPolicy.swift Defines replay policy interfaces and replay-friendly thermal/input types.
Sources/AdaptiveReplayKit/ReplayMetrics.swift Defines replay metrics and staleness stats computations.
Sources/AdaptiveReplayKit/ReplayEngine.swift Implements deterministic replay simulation including interaction-advance modeling.
Sources/AdaptiveReplayKit/RecordedScheduleAudit.swift Implements auditing of recorded timer-advance events independent of replay clock.
Sources/AdaptiveReplayKit/README.md Documents how tracing, parsing, replay, auditing, and activity telemetry fit together.
Sources/AdaptiveReplayKit/CandidatePolicies.swift Adds replay-only candidate policy using coding-activity signal to cap delays.
Sources/AdaptiveReplayKit/BaselinePolicies.swift Adds mirrored adaptive baseline plus fixed/manual baseline policies.
Sources/AdaptiveReplayKit/AdaptiveRefreshTraceWriter.swift Implements JSONL writer with async serialization and hard byte limit.
Sources/AdaptiveReplayKit/AdaptiveRefreshTraceParser.swift Implements strict-by-default JSONL parsing plus tolerant variant.
Sources/AdaptiveReplayKit/AdaptiveRefreshTrace.swift Defines trace record schema, kinds, and record constructors.
Sources/AdaptiveReplayKit/ActivityCoverageStats.swift Computes informational activity coverage fractions from raw decision records.
Sources/AdaptiveReplayCLI/main.swift Adds CLI for running replay + audit and emitting table/JSON reports.
Package.swift Adds AdaptiveReplayKit library and AdaptiveReplayCLI executable products/targets.
docs/predictive-refresh-policy.md Updates the decision record with implementation status and local replay follow-up evidence section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Sources/AdaptiveReplayKit/CandidatePolicies.swift
@hhh2210

hhh2210 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jul 10, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@hhh2210

hhh2210 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Proof update for head 2d43edb. This also addresses Copilot's policy-name comment.

  • Frozen replay input: 1,780 records, 345,642 bytes, SHA-256 b1e4aa33180b7c177293eb9ed16b45e24e026d259600fba2b1b67b931b904f0b.
  • Current Adaptive simulated 694 refreshes versus 1,383 for fixed 5m, or 49.8% fewer. Its p50/p95 menu staleness was 142s/1,093s versus 150s/281s.
  • Exact-head validation: 83/83 focused tests, 604 selections across 51/51 full-suite groups, make check, and an ad-hoc app bundle with strict codesign verification.
  • The frozen trace contains zero timer-advance evaluations. This proof does not claim accepted/rejected live-path coverage.
  • The raw trace remains local. The image below contains aggregate CLI output only.

Exact-head replay output:

AdaptiveReplayCLI exact-head aggregate output

@clawsweeper re-review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 32 out of 32 changed files in this pull request and generated 2 comments.

Comment thread Sources/AdaptiveReplayKit/AdaptiveRefreshTraceParser.swift Outdated
Comment thread Sources/AdaptiveReplayKit/AdaptiveRefreshTraceParser.swift Outdated
@hhh2210

hhh2210 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper What's your review status? Do you need to restart the review process?

@hhh2210

hhh2210 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper clawsweeper Bot added proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed July 11, 2026, 11:33 PM ET / July 12, 2026, 03:33 UTC.

Summary
The branch adds an offline JSONL adaptive-refresh replay CLI and analysis kit, extracts the production decision table into a shared package-internal core, and adds focused tests and decision-record documentation.

Reproducibility: not applicable. This PR adds an offline developer analysis capability rather than fixing a reproducible current-main defect. Its behavior is demonstrated through exact-head CLI runs and focused deterministic tests.

Review metrics: 3 noteworthy metrics.

  • Review surface: 28 files, +3,195/-146. The substantial change is mostly isolated in new replay targets and tests, with a narrow production policy adapter extraction.
  • Focused test files: 10 changed. Coverage spans CLI parsing, trace parsing, shared policy boundaries, adapters, replay math, segmentation, activity policy, and schedule auditing.
  • Policy ownership: 2 consumers, 1 decision table. The application and replay tooling now share one package-internal adaptive policy implementation.

Merge readiness
Overall: 🦞 diamond lobster
Proof: 🦞 diamond lobster
Patch quality: 🦞 diamond lobster
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Risk before merge

  • [P1] One exact-head x64 Linux test job is red while the matching arm64 Linux job passes; required checks should be rerun or diagnosed before merge, although the available evidence does not establish a patch defect.

Maintainer options:

  1. Decide the mitigation before merge
    Ship the maintainer-sponsored offline replay tool with its single canonical policy core after the remaining exact-head test gate is green or confirmed unrelated, while keeping runtime trace collection and production policy changes outside this PR.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] No repair task remains; required checks and ordinary maintainer merge controls should gate the exact sponsored head.

Security
Cleared: No concrete security or supply-chain concern was found: the patch adds no dependency, network, provider, credential, installer, downloaded-code, or runtime trace-collection path.

Review details

Best possible solution:

Ship the maintainer-sponsored offline replay tool with its single canonical policy core after the remaining exact-head test gate is green or confirmed unrelated, while keeping runtime trace collection and production policy changes outside this PR.

Do we have a high-confidence way to reproduce the issue?

Not applicable: this PR adds an offline developer analysis capability rather than fixing a reproducible current-main defect. Its behavior is demonstrated through exact-head CLI runs and focused deterministic tests.

Is this the best way to solve the issue?

Yes. Sharing the production decision table while accepting only an explicit caller-supplied trace is the narrowest maintainable design because it prevents replay drift without introducing runtime telemetry; normal exact-head checks should still gate merge.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 237fa70c97c7.

Label changes

Label justifications:

  • P3: This optional offline developer tool does not change user defaults, collect production data, or affect normal application workflows.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • feature: ✨ showcase: ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. The deterministic offline workflow enables evidence-based refresh-policy comparisons from private traces without shipping telemetry or changing user defaults.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The exact-head PR body documents real table and JSON CLI runs and line-specific malformed-input failure over a hash-pinned local trace, with privacy-preserving aggregate output in the discussion.
  • proof: sufficient: Contributor real behavior proof is sufficient. The exact-head PR body documents real table and JSON CLI runs and line-specific malformed-input failure over a hash-pinned local trace, with privacy-preserving aggregate output in the discussion.
Evidence reviewed

What I checked:

  • Canonical policy implementation: The app adapter and AdaptiveReplayPolicy both delegate to AdaptiveRefreshPolicyCore, resolving the prior duplicate-policy finding and preventing production/replay drift. (Sources/CodexBar/AdaptiveRefreshPolicy.swift:1, b231503a5302)
  • Offline product boundary: AdaptiveReplayCLI depends on AdaptiveReplayKit, whose only internal dependency is the Foundation-only AdaptiveRefreshCore; the implementation adds no provider, credential, browser, network, or production trace-collection path. (Package.swift:84, b231503a5302)
  • Real behavior proof: The PR records exact-head table and JSON CLI runs plus malformed-JSONL failure with line and content, supported by aggregate output from a hash-pinned 1,780-record local trace without publishing private trace data. (Sources/AdaptiveReplayCLI/main.swift:33, b231503a5302)
  • Focused validation coverage: Ten changed test files cover policy boundaries and adapters, CLI arguments, strict and tolerant parsing, replay metrics, activity policy, segmentation, and recorded-schedule auditing; earlier CRLF and interval-validation findings have regression coverage. (Tests/AdaptiveReplayCLITests/CLIArgumentsTests.swift:1, b231503a5302)
  • Maintainer sponsorship and provenance: The repository owner authored four final refinement commits, force-pushed the exact reviewed head, and renamed the item from an RFC to a feature, providing clear sponsorship for the tooling direction. (Sources/AdaptiveReplayKit/ReplayEngine.swift:1, b231503a5302)
  • Exact-head check state: The arm64 Linux job built, tested, and smoke-tested successfully; the x64 job completed its release build successfully but failed during Linux tests, leaving a required-check gate without a source-supported review finding. (b231503a5302)

Likely related people:

  • steipete: The repository owner authored the exact-head performance, staleness, documentation, and evidence-validation refinements and force-pushed the reviewed branch. (role: feature sponsor and recent implementation owner; confidence: high; commits: 4b91d701c325, f473e660a78c, f7b4d9c96f9a; files: Sources/AdaptiveReplayKit/ReplayEngine.swift, Sources/AdaptiveReplayKit/RecordedScheduleAudit.swift, Sources/AdaptiveReplayKit/ReplayTraceSegmentation.swift)
  • hhh2210: Merged history shows prior ownership of the adaptive-refresh decision record and production implementation that this replay tooling evaluates, in addition to foundational commits on this branch. (role: adaptive-refresh feature contributor; confidence: high; commits: 692da32ecbbb, 563f48d9ceda, 2b233e42c7b9; files: docs/predictive-refresh-policy.md, Sources/CodexBar/AdaptiveRefreshPolicy.swift, Sources/AdaptiveRefreshCore/AdaptiveRefreshPolicyCore.swift)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.
Review history (9 earlier review cycles; latest 8 shown)
  • reviewed 2026-07-11T11:32:36.456Z sha d394420 :: needs real behavior proof before merge. :: [P1] Reuse one canonical adaptive policy implementation
  • reviewed 2026-07-11T11:38:40.994Z sha cd96204 :: needs real behavior proof before merge. :: [P1] Use one canonical adaptive policy implementation
  • reviewed 2026-07-11T11:44:00.980Z sha cd96204 :: found issues before merge. :: [P1] Use one canonical adaptive policy implementation
  • reviewed 2026-07-11T12:23:47.220Z sha 4709bec :: needs maintainer review before merge. :: none
  • reviewed 2026-07-11T12:31:28.481Z sha 4709bec :: needs changes before merge. :: [P2] Reject invalid fixed policy intervals
  • reviewed 2026-07-11T12:51:56.096Z sha 41c6c0d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-11T12:58:10.270Z sha 41c6c0d :: needs maintainer review before merge. :: none
  • reviewed 2026-07-12T03:28:15.341Z sha b231503 :: needs maintainer review before merge. :: none

@hhh2210 hhh2210 force-pushed the codex/adaptive-refresh-replay-upstream branch from 2d43edb to d394420 Compare July 11, 2026 11:27
@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Jul 11, 2026
@hhh2210 hhh2210 force-pushed the codex/adaptive-refresh-replay-upstream branch from d394420 to cd96204 Compare July 11, 2026 11:34
@hhh2210

hhh2210 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Review update for exact head cd96204ac197ec0c7ce48e615a608358c01ba3fb:

  • Rebased onto current main; GitHub reports the PR as mergeable.
  • Split runtime collection out of this PR. The current diff has no Sources/CodexBar changes, app hooks, recorder, trace writer, transcript scanning, provider calls, or storage lifecycle.
  • Reduced the review surface from 32 files and +4,147/-55 to 21 files and +2,840/-7. Tests account for 1,224 additions.
  • Kept AdaptiveReplayKit package-internal. The only new product is AdaptiveReplayCLI.
  • Fixed and resolved both Copilot CRLF threads with Character.isNewline plus strict and tolerant multi-record regressions.

Exact-head checks passed: make check, swift package describe, swift build --product AdaptiveReplayCLI, and an author-run CLI replay over the hash-pinned local trace. The previous pushed revision, with identical source and test code, passed the full 610 selections in 51/51 groups; upstream CI is running the exact head.

Trace: 1,780 records, SHA-256 b1e4aa33180b7c177293eb9ed16b45e24e026d259600fba2b1b67b931b904f0b.

policy              refreshes  per24h  sim advances  active >5m  staleness p50/p95  constrained ok
adaptive            694        143.47  53            4/145       142s / 1093s       yes
adaptive-activity   696        143.88  53            0/145       139s / 1093s       yes
fixed-5m            1383       285.90  0             0/99        150s / 281s        NO (157)
activity telemetry: 462/733 sampled (63%), 185/462 active (40%)
recorded schedule: 46 advances, 0/0 evaluations, all mismatch counts 0
segmentation: 28 segments, 26.10h excluded, 300s grace

Runtime proof is no longer applicable because runtime collection is no longer in the diff. This remains single-machine, author-run evidence; it makes no battery, user-time, request-count, cost, or default-policy claim.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 11, 2026
@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. and removed merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 11, 2026
@hhh2210

hhh2210 commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Addressed fixed-interval validation in 41c6c0dc.

--policy now accepts only the seven names listed by --help. Generic fixed-<N>m parsing is gone, so fixed-0m, fixed--1m, fixed-3m, and fixed-9223372036854775807m are rejected before trace reading or policy construction. Each real CLI invocation exits 1 and prints the accepted names. FixedIntervalPolicy also converts to TimeInterval before multiplying, removing the integer-overflow path.

Validation on this head:

  • 3 CLI argument tests covering 12 cases, including all seven accepted names and the four invalid interval classes;
  • 67 replay/core tests across 8 suites and 4 app-adapter tests;
  • swift build --product AdaptiveReplayCLI, make check, and git diff --check;
  • the hash-pinned 1,780-record replay still produces the same metrics.

changes, lint, and GitGuardian are green on the new head. The Linux builds and macOS test shards are running. The mechanical finding is addressed; owner sign-off for the CLI remains the product decision.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 availability 🚨 Merging this PR could cause crashes, hangs, restart loops, stalls, or process outages. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Jul 11, 2026
hhh2210 and others added 12 commits July 12, 2026 04:11
Add local tooling for evaluating candidate refresh-timing policies against
real usage traces. The harness builds on the existing adaptive refresh policy
and interaction-advance behavior.

New targets:
- AdaptiveReplayKit (library, pure Foundation): JSONL trace schema
  (decision / menuOpen / refreshCompleted / timerAdvanced), strict-by-default
  parser with tolerant fallback, best-effort serialized writer, ReplayPolicy
  protocol with an advancesOnInteraction capability flag, deterministic
  ReplayEngine, ReplayMetrics (staleness percentiles, refresh cadence,
  constrained compliance, interaction-advance count), and baseline policies
  (mirrored adaptive table, fixed 2/5/15/30m, manual).
- AdaptiveReplayCLI (executable): thin shell that parses arguments, replays a
  trace against one or more policies, and prints a table or JSON.

App-side wiring is additive and remains off unless the
adaptiveRefreshTraceEnabled defaults key is set:
- AdaptiveRefreshTraceRecording writes local JSONL records containing only
  timestamps, signal values, reasons, delays, and event kinds.
- recordDecision, recordMenuOpen, recordRefreshCompleted, and
  recordTimerAdvanced cover the timer tick and interaction-advance paths.
- AdaptiveRefreshPolicy+ReplayAdapter lets replay tests compare the library
  mirror with the production policy.

Tests cover hand-computed replay metrics, interaction advances, constrained
policy boundaries, parser round trips, and each recorder call site.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires CodingActivityProbe into decision-event trace recording, gated on
adaptiveRefreshTraceEnabled: samples seconds-since-last-transcript-write
for Codex/Claude only while tracing is on, and records it as two optional
fields that are never fed into AdaptiveRefreshPolicy. Bounds the probe's
directory walk (today+yesterday for Codex, one level under Claude project
dirs) after measuring ~15-23ms unbounded vs ~5-7ms bounded against this
machine's real ~/.codex and ~/.claude trees. AdaptiveReplayCLI gains an
informational activity-coverage summary line.
Extends CodingActivitySample with three per-CLI fields read from the same
stat pass as the A-layer seconds-since-activity: session duration
(mtime - creationDate), transcript byte size, and a count of transcripts
modified in the last 5 minutes. Consolidates recordDecision's two flat
activity params into a single activitySample: CodingActivitySample?
parameter now that it carries eight fields instead of two. Adds a
README documenting the harness and why token-level (C layer) probing
of rate_limits/usage stays deferred until offline analysis justifies it.
@steipete steipete force-pushed the codex/adaptive-refresh-replay-upstream branch from 41c6c0d to b231503 Compare July 12, 2026 03:24
@steipete steipete changed the title RFC: add local adaptive-refresh replay tooling feat: add offline adaptive-refresh replay tooling Jul 12, 2026
@steipete

Copy link
Copy Markdown
Owner

Maintainer verification at exact head b231503a53020acad599672717ef80e0207544fd:

  • make test: 624 selections across 52 groups passed on the first attempt, with zero retries, recoveries, or timeouts.
  • Focused adaptive replay, replay segmentation, and recorded schedule audit suites passed.
  • make check: passed with zero SwiftLint violations across 1,402 files.
  • Maintainer autoreview: clean after two fix/review cycles (0.90 confidence).
  • Real CLI smoke produced expected table and JSON metrics; malformed JSONL exited 1 and identified line 2 plus its offending content.

The contributor's private trace was not copied or published. Provider/account live testing is intentionally out of scope for this offline, caller-supplied trace tool.

Hosted exact-head proof: lint, changes, Linux arm64, and Linux x64 are green. The first x64 attempt had one unrelated Claude planner availability race; its clean rerun passed the build, full Linux tests, and CLI smoke.

@steipete steipete merged commit fe75394 into steipete:main Jul 12, 2026
10 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: ✨ showcase ClawSweeper spotlight: unusually compelling feature idea for maintainer attention. P3 Low-risk cleanup, docs, polish, ergonomics, or speculative feature. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants