Skip to content

Commit c844798

Browse files
Fix large->small model switch overflowing before recovery arms (#188)
Switching mid-session from a large-context model to a smaller one (e.g. 512k -> 272k) while carrying more history than the new model's window sent the first request oversized and let the provider reject it; recovery only armed on the NEXT pass from the error, so the user saw one hard 'Input exceeds context window'. Root cause was a model-resolution bug, not just missing recovery. On a switching turn the transform's message array ends with [..., OLD-model assistant, NEW user message]. OpenCode's schema is asymmetric: USER messages nest the selected model under info.model.{providerID,modelID} while ASSISTANT messages carry it flat as info.providerID/modelID, and the request resolves from the newest user's model (confirmed with the OpenCode maintainer). The detector compared liveModelBySession (already NEW, set by chat.message before transform) against findLastAssistantModel (still OLD on the switching turn), so it false-fired: it flipped the live model back to OLD and the whole pass (history budget, emergency-drop ceiling) resolved the old 512k limit. Worse, that false-fire was incidentally the only thing clearing stale per-model state on a live switch (hook-handlers.ts never clears on a live switch because chat.message updates the map first). Fix unifies the model-change trigger on the authoritative signal: the model that produced the last PERSISTED usage (lastObservedModelKey) vs the outgoing model (newest user message). When they differ the model genuinely changed since the last measured turn, so the old model's detected-limit / reasoning watermark / emergency state is cleared, while liveModelBySession is kept at the NEW model so the budget and ceiling resolve correctly. The persisted-usage snapshot is hoisted above the clear so the proactive arm still sees the old input. The arm then fires flag-only recovery when the last input (measured on a DIFFERENT model) exceeds the NEW model's catalog cap (getSdkContextLimit, not a possibly-stale detected limit), so the existing bump-to-95% compacts before the request is sent. A same-model 'input > limit' is treated as cache regression (#179), not overflow, so it never false-arms. Pi needs no change: its forward-pressure floor already trips the 95% band on a shrinking switch (getContextUsage().tokens is the forward estimate and .contextWindow reflects the new model immediately). Documented in PARITY.md #22. Reviewed across four adversarial Oracle passes; final verdict SHIP. +5 regression tests including the production-shaped live switch and the no-info.model fallback. Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>
1 parent c643ca1 commit c844798

3 files changed

Lines changed: 486 additions & 57 deletions

File tree

‎packages/pi-plugin/PARITY.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -595,6 +595,42 @@ navigation tools; Pi gets an equally safe, narrower built-in set.
595595

596596
---
597597

598+
## 22. Shrinking model-switch overflow: OpenCode arms proactively; Pi is covered by the forward-pressure floor
599+
600+
When a session switches mid-conversation from a large-context model to a smaller
601+
one (e.g. 512k → 272k) while carrying more history than the new model's window,
602+
the first request on the new model would otherwise be sent oversized and rejected
603+
("Input exceeds context window"), with recovery only arming on the next pass from
604+
the provider error (issue #188).
605+
606+
- **OpenCode** needs an explicit proactive arm. Its pressure on the first pass
607+
after a switch reads the OLD model's low ratio (the live `contextUsageMap` and
608+
persisted `lastContextPercentage` were measured against the larger window), so
609+
no band trips. The transform's overflow block therefore arms recovery
610+
(flag-only) when the last-measured input exceeds the CURRENT model's trusted
611+
limit, so the existing bump-to-95% compacts before sending.
612+
- **Pi** needs no such arm. Its `applyForwardPressureFloor` already computes
613+
`forwardTokens / (contextWindow * 0.85)` every pass, and on the first pass
614+
after a switch `getContextUsage()` returns `.tokens` ≈ the forward estimate
615+
(last OLD-model usage + trailing) while `.contextWindow` already reflects the
616+
NEW (smaller) model (set synchronously at `setModel`). So the floor computes
617+
~130% → trips the 95% emergency band → Pi compacts before sending. Verified
618+
against pi-mono source (`getContextUsage` → `estimateContextTokens`; `setModel`
619+
sets `state.model` synchronously).
620+
621+
Net behavior matches (both compact before the oversized request), via different
622+
mechanisms: OpenCode adds a targeted arm, Pi reuses its existing floor.
623+
624+
Narrow residual edge (not fixed, documented): if a user switches model in the
625+
window right AFTER a compaction but BEFORE any assistant response on the new
626+
model, Pi's `getContextUsage().tokens` is `null` (no post-compaction usage yet),
627+
so the floor falls back to the low ratio and would not trip. The reported #188
628+
scenario (substantial live history = many old-model assistant usages present) is
629+
firmly outside this edge. If it ever surfaces, the clean Pi fix is
630+
`tokens === null AND model-window-shrank-since-last-pass → arm`.
631+
632+
---
633+
598634
## Maintenance
599635

600636
Update this file whenever a deliberate Pi↔OpenCode divergence is introduced or

‎packages/plugin/src/hooks/magic-context/transform.test.ts‎

Lines changed: 275 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ import {
2222
getLastNudgeLevel,
2323
getLastNudgeUndropped,
2424
getOrCreateSessionMeta,
25+
getOverflowState,
2526
getPendingOps,
2627
getTagById,
2728
getTagsBySession,
@@ -39,7 +40,7 @@ import {
3940
import { createTagger } from "../../features/magic-context/tagger";
4041
import type { ContextUsage } from "../../features/magic-context/types";
4142
import type { PluginContext } from "../../plugin/types";
42-
import { clearModelsDevCache } from "../../shared/models-dev-cache";
43+
import { clearModelsDevCache, refreshModelLimitsFromApi } from "../../shared/models-dev-cache";
4344
import { Database } from "../../shared/sqlite";
4445
import { closeQuietly } from "../../shared/sqlite-helpers";
4546
import { createTransform } from "./transform";
@@ -2236,6 +2237,279 @@ describe("createTransform protected tail", () => {
22362237
});
22372238
});
22382239

2240+
describe("createTransform shrinking model-switch overflow pre-arm", () => {
2241+
const OLD_MODEL = { providerID: "anthropic", modelID: "claude-big-512k" };
2242+
const NEW_MODEL = { providerID: "openai", modelID: "gpt-5.5" };
2243+
const NEW_KEY = "openai/gpt-5.5";
2244+
const OLD_KEY = "anthropic/claude-big-512k";
2245+
2246+
async function seedNewModelLimit(inputCap: number): Promise<void> {
2247+
clearModelsDevCache();
2248+
await refreshModelLimitsFromApi({
2249+
config: {
2250+
providers: async () => ({
2251+
data: {
2252+
providers: [
2253+
{ id: "openai", models: { "gpt-5.5": { limit: { input: inputCap } } } },
2254+
],
2255+
},
2256+
}),
2257+
},
2258+
});
2259+
}
2260+
2261+
function makeTransform(
2262+
db: ReturnType<typeof openDatabase>,
2263+
sessionId: string,
2264+
liveModel: { providerID: string; modelID: string },
2265+
usage: { percentage: number; inputTokens: number },
2266+
) {
2267+
const abort = mock(async () => ({}));
2268+
const prompt = mock(async () => ({}));
2269+
const liveModelBySession = new Map([[sessionId, liveModel]]);
2270+
const transform = createTransform({
2271+
tagger: createTagger(),
2272+
scheduler: { shouldExecute: mock(() => "defer" as const) },
2273+
contextUsageMap: new Map([[sessionId, { usage, updatedAt: Date.now() }]]),
2274+
db,
2275+
historyRefreshSessions: new Set<string>(),
2276+
pendingMaterializationSessions: new Set<string>(),
2277+
lastHeuristicsTurnId: new Map<string, string>(),
2278+
clearReasoningAge: 50,
2279+
protectedTags: 0,
2280+
liveModelBySession,
2281+
// Mirror production: getModelKey derives from the live model map.
2282+
getModelKey: (id: string) => {
2283+
const m = liveModelBySession.get(id);
2284+
return m ? `${m.providerID}/${m.modelID}` : undefined;
2285+
},
2286+
client: { session: { abort, prompt } } as unknown as PluginContext["client"],
2287+
directory: "/tmp",
2288+
});
2289+
return { transform, abort, prompt };
2290+
}
2291+
2292+
// Production shape on a live switch: the array still ends with the OLD
2293+
// model's assistant response (flat info.providerID/modelID), then the NEW
2294+
// user message carrying the just-selected model nested under info.model.
2295+
// liveModelBySession was already flipped to NEW by chat.message.
2296+
function switchTurnMessages(sessionId: string) {
2297+
return [
2298+
{
2299+
info: { id: "m-user-0", role: "user", sessionID: sessionId },
2300+
parts: [{ type: "text", text: "earlier" }],
2301+
},
2302+
{
2303+
info: {
2304+
id: "m-assistant-old",
2305+
role: "assistant",
2306+
sessionID: sessionId,
2307+
providerID: OLD_MODEL.providerID,
2308+
modelID: OLD_MODEL.modelID,
2309+
},
2310+
parts: [{ type: "text", text: "old model reply" }],
2311+
},
2312+
{
2313+
info: {
2314+
id: "m-user-1",
2315+
role: "user",
2316+
sessionID: sessionId,
2317+
model: { providerID: NEW_MODEL.providerID, modelID: NEW_MODEL.modelID },
2318+
},
2319+
parts: [{ type: "text", text: "continue" }],
2320+
},
2321+
];
2322+
}
2323+
2324+
it("arms recovery (flag-only) when last input on the OLD model exceeds the NEW model's cap", async () => {
2325+
useTempDataHome("transform-shrink-switch-arm-");
2326+
createOpenCodeDbForTransform("ses-shrink", [
2327+
{ id: "m-raw-1", role: "user", text: "recent 1" },
2328+
{ id: "m-raw-2", role: "assistant", text: "recent 2" },
2329+
]);
2330+
const db = openDatabase();
2331+
// Persisted usage from the PREVIOUS (large, 512k) model: 300k input,
2332+
// ~58%, well under any threshold. lastObservedModelKey = OLD model.
2333+
updateSessionMeta(db, "ses-shrink", {
2334+
lastContextPercentage: 58,
2335+
lastInputTokens: 300_000,
2336+
lastObservedModelKey: OLD_KEY,
2337+
lastUsageContextLimit: 512_000,
2338+
});
2339+
await seedNewModelLimit(272_000);
2340+
2341+
const { transform } = makeTransform(db, "ses-shrink", NEW_MODEL, {
2342+
percentage: 58,
2343+
inputTokens: 300_000,
2344+
});
2345+
await transform({}, { messages: switchTurnMessages("ses-shrink") });
2346+
await new Promise((resolve) => setTimeout(resolve, 0));
2347+
2348+
const overflow = getOverflowState(db, "ses-shrink");
2349+
// Armed THIS pass so the existing bump-to-95% compacts before the
2350+
// oversized request is sent (not one pass late from the provider error).
2351+
expect(overflow.needsEmergencyRecovery).toBe(true);
2352+
// Flag-only: no catalog/auth limit pinned into detected_context_limit.
2353+
expect(overflow.detectedContextLimit).toBe(0);
2354+
});
2355+
2356+
it("does not arm on a normal SAME-model turn whose input fits the model", async () => {
2357+
useTempDataHome("transform-shrink-switch-noarm-");
2358+
createOpenCodeDbForTransform("ses-fits", [
2359+
{ id: "m-raw-1", role: "user", text: "recent 1" },
2360+
]);
2361+
const db = openDatabase();
2362+
// Same (new) model, last input under the 272k cap.
2363+
updateSessionMeta(db, "ses-fits", {
2364+
lastContextPercentage: 73,
2365+
lastInputTokens: 200_000,
2366+
lastObservedModelKey: NEW_KEY,
2367+
lastUsageContextLimit: 272_000,
2368+
});
2369+
await seedNewModelLimit(272_000);
2370+
2371+
const { transform } = makeTransform(db, "ses-fits", NEW_MODEL, {
2372+
percentage: 73,
2373+
inputTokens: 200_000,
2374+
});
2375+
await transform(
2376+
{},
2377+
{
2378+
messages: [
2379+
{
2380+
info: {
2381+
id: "m-user-1",
2382+
role: "user",
2383+
sessionID: "ses-fits",
2384+
model: NEW_MODEL,
2385+
},
2386+
parts: [{ type: "text", text: "continue" }],
2387+
},
2388+
],
2389+
},
2390+
);
2391+
await new Promise((resolve) => setTimeout(resolve, 0));
2392+
2393+
expect(getOverflowState(db, "ses-fits").needsEmergencyRecovery).toBe(false);
2394+
});
2395+
2396+
it("does NOT arm when last input exceeds the limit but on the SAME model (cache regression, not overflow)", async () => {
2397+
useTempDataHome("transform-shrink-switch-samemodel-");
2398+
createOpenCodeDbForTransform("ses-same", [
2399+
{ id: "m-raw-1", role: "user", text: "recent 1" },
2400+
]);
2401+
const db = openDatabase();
2402+
// The last input (300k) was ACCEPTED by this same model, so a now-lower
2403+
// catalog reading (272k) is a stale/regressed limit, not a real overflow.
2404+
// Arming here would cause a gratuitous compaction + cache bust.
2405+
updateSessionMeta(db, "ses-same", {
2406+
lastContextPercentage: 90,
2407+
lastInputTokens: 300_000,
2408+
lastObservedModelKey: NEW_KEY,
2409+
lastUsageContextLimit: 400_000,
2410+
});
2411+
await seedNewModelLimit(272_000);
2412+
2413+
const { transform } = makeTransform(db, "ses-same", NEW_MODEL, {
2414+
percentage: 90,
2415+
inputTokens: 300_000,
2416+
});
2417+
await transform(
2418+
{},
2419+
{
2420+
messages: [
2421+
{
2422+
info: {
2423+
id: "m-user-1",
2424+
role: "user",
2425+
sessionID: "ses-same",
2426+
model: NEW_MODEL,
2427+
},
2428+
parts: [{ type: "text", text: "continue" }],
2429+
},
2430+
],
2431+
},
2432+
);
2433+
await new Promise((resolve) => setTimeout(resolve, 0));
2434+
2435+
expect(getOverflowState(db, "ses-same").needsEmergencyRecovery).toBe(false);
2436+
});
2437+
2438+
it("does not arm for a subagent session", async () => {
2439+
useTempDataHome("transform-shrink-switch-subagent-");
2440+
createOpenCodeDbForTransform("ses-shrink-sub", [
2441+
{ id: "m-raw-1", role: "user", text: "recent 1" },
2442+
]);
2443+
const db = openDatabase();
2444+
updateSessionMeta(db, "ses-shrink-sub", {
2445+
isSubagent: true,
2446+
lastContextPercentage: 58,
2447+
lastInputTokens: 300_000,
2448+
lastObservedModelKey: OLD_KEY,
2449+
lastUsageContextLimit: 512_000,
2450+
});
2451+
await seedNewModelLimit(272_000);
2452+
2453+
const { transform } = makeTransform(db, "ses-shrink-sub", NEW_MODEL, {
2454+
percentage: 58,
2455+
inputTokens: 300_000,
2456+
});
2457+
await transform({}, { messages: switchTurnMessages("ses-shrink-sub") });
2458+
await new Promise((resolve) => setTimeout(resolve, 0));
2459+
2460+
expect(getOverflowState(db, "ses-shrink-sub").needsEmergencyRecovery).toBe(false);
2461+
});
2462+
2463+
it("still arms when the newest user message lacks info.model (falls back to the live map, not the OLD assistant)", async () => {
2464+
useTempDataHome("transform-shrink-switch-nomodel-");
2465+
createOpenCodeDbForTransform("ses-nomodel", [
2466+
{ id: "m-raw-1", role: "user", text: "recent 1" },
2467+
]);
2468+
const db = openDatabase();
2469+
updateSessionMeta(db, "ses-nomodel", {
2470+
lastContextPercentage: 58,
2471+
lastInputTokens: 300_000,
2472+
lastObservedModelKey: OLD_KEY,
2473+
lastUsageContextLimit: 512_000,
2474+
});
2475+
await seedNewModelLimit(272_000);
2476+
2477+
// liveModelBySession = NEW (chat.message set it). The newest user message
2478+
// carries NO info.model, so the resolver must fall back to the live map
2479+
// (NEW), NOT the OLD last-assistant. If it fell to the assistant, the
2480+
// detector would mis-resolve to OLD and the arm would not fire.
2481+
const { transform } = makeTransform(db, "ses-nomodel", NEW_MODEL, {
2482+
percentage: 58,
2483+
inputTokens: 300_000,
2484+
});
2485+
await transform(
2486+
{},
2487+
{
2488+
messages: [
2489+
{
2490+
info: {
2491+
id: "m-assistant-old",
2492+
role: "assistant",
2493+
sessionID: "ses-nomodel",
2494+
providerID: OLD_MODEL.providerID,
2495+
modelID: OLD_MODEL.modelID,
2496+
},
2497+
parts: [{ type: "text", text: "old model reply" }],
2498+
},
2499+
{
2500+
// newest user, no info.model
2501+
info: { id: "m-user-1", role: "user", sessionID: "ses-nomodel" },
2502+
parts: [{ type: "text", text: "continue" }],
2503+
},
2504+
],
2505+
},
2506+
);
2507+
await new Promise((resolve) => setTimeout(resolve, 0));
2508+
2509+
expect(getOverflowState(db, "ses-nomodel").needsEmergencyRecovery).toBe(true);
2510+
});
2511+
});
2512+
22392513
describe("createTransform historian failure handling", () => {
22402514
it("lets an armed empty-head emergency proceed after the bounded no-head escape", async () => {
22412515
useTempDataHome("transform-empty-head-escape-");

0 commit comments

Comments
 (0)