refactor(gateway): execute model refreshes in platform cells - #410
Open
Menci wants to merge 49 commits into
Open
refactor(gateway): execute model refreshes in platform cells#410Menci wants to merge 49 commits into
Menci wants to merge 49 commits into
Conversation
Require cold and arbitrarily stale catalog reads to return immediately while background refreshes populate the cache. Cover explicit force-fetch semantics, persistent failure cooldown, concurrent trigger coalescing, generation fencing, and obsolete cache replacement.
Serve cold and stale catalogs without awaiting upstream I/O, retain stale catalogs indefinitely, and move automatic refresh attempts behind an inline per-upstream claim with exponential failure cooldown and lease recovery. Keep explicit model fetches and post-save or OAuth warming synchronous while sharing generation fencing and in-flight coordination.
Discard the coordinator's internal fetched-or-skipped result at the scheduler boundary while preserving rejection for runtime observability.
Exercise memory and SQL repositories against the same atomic claim, exponential cooldown, forced bypass, abandoned-lease recovery, generation fence, and stale-completion contract.
Walk enabled upstreams on every scheduled maintenance tick, use the same nonblocking cache trigger and persisted cooldown as request access, and await the detached work inside the maintenance lifetime. Use the configured Node runtime location and a dedicated scheduled location tag where the runtime exposes no request colo.
Persist a failed first refresh as an empty immediately-stale catalog so later requests and the dashboard retain its error while automatic retries remain governed by backoff. Read cached error annotations during catalog assembly now that upstream failures happen outside the requesting lifecycle.
Warm direct-write app fixtures after fetch mocks are installed and before model-consuming requests, matching the synchronous create, update, and OAuth lifecycle. Keep provider catalog and resolution suites on persisted snapshots so their assertions remain about assembly and routing rather than cold-cache trigger timing.
Carry persisted catalog errors into request-time failed-upstream metadata without coupling request cancellation to detached refresh I/O. Update catalog and HTTP coverage for empty nonblocking listings when no last-known-good models exist, while retaining secret redaction and healthy sibling results.
Preserve the public list pagination fields when a cold catalog refresh fails and the request returns the currently empty snapshot.
Warm model-consuming app fixtures only while their upstream fetch mock is installed, avoiding real network work before validation-only requests. Keep unrelated scheduled-maintenance tests catalog-free and explicitly warm the standalone alpha-search app fixture.
Bring local Codex, WebSocket, addressable-listing, target-picker, and endpoint fixtures through the same persisted warm lifecycle before exercising their actual subject. Update SQL and Gemini expectations for persisted cold errors and immediate empty listings.
Await only catalog refreshes started by the fixture warm, so unrelated WebSocket and dump background work cannot hold the next test's setup open.
Document synchronous control-plane warming, nonblocking request and scheduled triggers, indefinite stale retention, inline claims, leases, failure cooldown, cold-error snapshots, and generation fencing. Align resolution comments and make the refresh fan-out test measure the actual background work.
Require the active persisted claim token to publish catalog or error results, and reject claims from either half of a stale timestamp/config generation. Compute the capped exponential retry schedule once in the coordinator, persist explicit outcomes in both repositories, and cover force/lease supersession plus the complete one-hour cap.
Add a blocking non-forced warm operation that joins in-flight work and respects persisted cooldown, while reserving forced claim bypass for the explicit Fetch Models action. Repair control-plane mocks and prove OAuth responses wait for their post-credential warm.
Use the credential-update path that actually persists and warms an existing upstream before asserting the OAuth response remains pending.
Submit scheduled model refresh promises to each runtime background scheduler without awaiting upstream model-list work, and run independent maintenance sweeps concurrently. Prove the scheduled trigger returns while a deferred upstream fetch remains owned by the runtime lifecycle.
Do not attribute a persisted refresh failure to a model id that the upstream prefix policy cannot address.
Keep requestApp transparent, move production-equivalent blocking warm behavior behind an explicitly named helper, and select it at model-dependent test imports. Add an HTTP regression proving a cold model listing returns before its triggered upstream fetch settles and later persists the result.
Remove stale request-time fetch and AbortError narratives, document durable refresh-error reporting and backoff-respecting warm behavior, and delete unreachable ProviderModelsUnavailableError listing branches.
Clear claim and backoff state whenever an upstream row is saved into a new generation, while preserving its last-known-good catalog unless the caller explicitly clears it.
Gate every mocked provider response until all sibling refreshes have started, replacing a load-sensitive elapsed-time assertion with an ordering proof.
Run model and image maintenance alongside the existing storage pipeline while keeping expiration processing ahead of spilled-file collection.
Classify persisted claim denials as active, backoff, or generation mismatch. Blocking warm operations now poll an active cross-runtime owner until it publishes or yields, while automatic triggers still return immediately. Cover durable warm joining and prove stale-token failure completion cannot reinstall cooldown.
Remove unowned model-cache repository writes so production and fixtures cannot bypass the persisted refresh protocol. Test seeding now acquires an explicit forced claim and publishes through the same owner-fenced methods.
Reserve fetch and round-trip terminology for background upstream I/O, describe blocking warm coordination accurately, and limit client-visible refresh-error reporting to resolution failures.
Track active owner tokens so cross-runtime warms observe durable completion without timestamp heuristics, let explicit force bypass local warm waiters, and avoid false backoff when claim release fails. Preserve cooldown across metadata generations while invalidating active owners, clear stale catalogs when fetch identity changes, expose warm side effects honestly in test calls, and cover the composed route behavior.
Publish a successful catalog and release its claim in one owner-fenced write; persist a failed error snapshot and backoff in one corresponding write. Use named claim inputs, durable active tokens, and unbounded forward-progress classification so runtime loss cannot split refresh outcome from coordination state.
Centralize upstream persistence around the provider model-fetch identity. Metadata-only saves retain cooldown and stale catalogs, while config, state, or proxy changes clear catalog and backoff before synchronous warm across PATCH, OAuth, and import flows.
When a forced owner steals a warm claim during upstream I/O, follow that durable owner through atomic finalization before returning the synchronous warm response.
…ite-swr # Conflicts: # AGENTS.md # docs/RESOLUTION.md # packages/gateway/src/data-plane/providers/resolution.ts # packages/gateway/src/repo/sql.ts
Resolve the new main-branch telemetry migration at 0077 by assigning model refresh coordination the next migration number.
Bypass the warm test wrapper so the merged client-disconnect regression continues to prove no cold catalog trigger dispatches after cancellation.
Keep scheduler-owned model refreshes on the raw upstream fetcher while retaining client-aware response lifetimes only for inference candidates. Cover disconnect after snapshot scheduling but before persisted claim dispatch.
Thread catalog and inference fetchers through named resolution context objects so their distinct background and client-disconnect lifecycles cannot be swapped positionally.
Carry one raw per-upstream fetcher through catalog resolution and model candidates. Apply client-disconnect retention only at inference dispatch boundaries, while scheduled catalog refreshes retain their independent background lifecycle.\n\nLock the contract by asserting that candidate enumeration preserves the supplied fetcher identity.
Replace overlapping cached/force/fetch modes with operation-shaped snapshot, explicit fetch, warm, and background scheduling boundaries. Explicit requests now join durable owners and bypass only cooldown, eliminating claim preemption and in-flight replacement races.\n\nSeparate provider-owned catalog identity from refresh-generation and operator-input identities. Fence durable writes against the full static fetch generation, preserve snapshots across safe credential rotation, reset cooldown for changed operator inputs, and apply control-plane replacements through an atomic compare-and-save operation.\n\nConsolidate save, warm, and readback across create, update, OAuth, and imports; reuse one transport catalog for batch imports. Make scheduled ownership and locationless egress explicit, isolate malformed upstreams, and constrain persisted refresh state.
Keep snapshot capture and freshness in a small cache facade, with durable/L1 ownership isolated in a refresh coordinator. Explicit fetches join successful work but retry through their own transport after background failure.\n\nMake catalog-aware creates insert-only, merge unrelated provider-state races during metadata replacement, fence failure counts, total the stale-owner transition, and retry uncertain finalization. Provider modules now own normalized catalog and refresh identities.\n\nFilter egress by runtime location before proxy parsing and cover cold inference convergence, remote-owner recovery, locationless scheduling, stale writers, and cross-owner lease recovery.
Reconcile uncertain finalization by releasing claims that never committed, retry explicit recovery exactly once across concurrent waiters, and make stale-owner and failure-count transitions total across SQL and memory.\n\nUse authoritative post-save rows for warming, preserve unrelated provider state during metadata writes, make catalog-aware inserts conflict-safe, and reject duplicate import ids before mutation. Normalize request identity per provider while excluding passive observation state.\n\nFilter proxy catalogs after location selection and update the merged Node repository fixture to the current refresh contract.
Replace provider-specific catalog/request identity projections with a persisted configVersion owned by the upstream repository. Split saved catalog refresh from draft preview so draft values can never publish a cache snapshot. Fence refresh coordination solely by the stored config generation while allowing runtime state and metadata writes to preserve it.
…ite-swr # Conflicts: # AGENTS.md
Keep catalog generations inside the gateway repository, unify provider-owned discovery projection, and separate saved refresh from draft preview through the dashboard. Warm only changed catalog inputs, preserve unrelated active owners, return cache status with saved refreshes, and make warm coordination side-effect-only. Fix abandoned claims so waiters reacquire ownership instead of treating release as successful completion.
Replace BroadcastDO with a protocol-driven ExecutionDO and migrate broadcast fan-out onto the platform execution-cell namespace. Route scheduled model refreshes through per-epoch cells; Cloudflare delegates database-owning work to a loopback WorkerEntrypoint while Node coalesces in process. Preserve the historical Durable Object class through a rename migration and enable ctx.exports.
Route explicit listing, save-time warmup, stale reads, and scheduled refreshes through the platform execution-cell namespace. Remove the isolate coordinator and SQLite claim/lease/token state; D1 now retains only retry backoff and config-version publication fencing. Replace ownership tests with execution coalescing, backoff, and generation-fence coverage.
Keep one execution cell per upstream/config/cache target across caller intent and runtime location, while preserving explicit discovery, proxy validation, retry, and automatic backoff semantics. Fence SQL completion by revision-aware cache epoch, advance epochs monotonically, collapse obsolete listing transport plumbing, and rewrite the unpublished refresh migration into its final shape. Add concurrency and error-path coverage from review-and-cleanup findings.
Menci
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ExecutionDOBroadcastDOwithExecutionDO, namespace broadcast cells separately, and preserve deployed objects through a Durable Object rename migrationconfigVersionand revision-aware cache epoch, with monotonically advancing publication epochsVerification
pnpm test— 520 files / 5454 tests passedpnpm run lint— passedpnpm run typecheck— passed across the workspacepnpm run check:agent-protocol— passed$review-and-cleanup— final correctness, comments, structure, and fallback rounds returned zero findings