Skip to content

fix(core): compact when a reply stops early at the output limit #58994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rekram1-node wants to merge 1 commit into from
Closed

fix(core): compact when a reply stops early at the output limit #58994

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/core/src/session/runner/llm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,11 @@ const layer = Layer.effect(
},
})
if (outcome.needsContinuation) return true
// Stopping short of the catalog limit means the context window ran out, so continuing alone cannot help.
if (outcome.early && recoverOverflow) {
recoverOverflow = false
yield* compaction.compact({ reason: "overflow", context: loaded })

@opencode-agent opencode-agent Bot Oct 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loaded was taken before this step ran, so it doesn't include the reply that just stopped with length. The summary leaves that reply out, and afterwards it's no longer in the model's context either: the new test ends with context [compaction, synthetic, assistant], and the partial reply only exists in the stored messages. CONTINUE_AFTER_OUTPUT_LIMIT then asks the model to carry on from output it can't see, so whatever the cut-off reply had produced is lost. Could you reload the context (prepareContext(sessionID).pipe(Effect.flatMap(context.load))) before compacting, so the cut-off reply is summarized or kept?

}
yield* bus.publish(SessionEvent.Synthetic, { sessionID, text: CONTINUE_AFTER_OUTPUT_LIMIT })
assistantMessageID = SessionMessage.ID.create()
}),
Expand Down
8 changes: 6 additions & 2 deletions packages/core/src/session/runner/step.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export type Outcome = Data.TaggedEnum<{
}
RecoverFull: {}
Compacted: {}
OutputLimit: { readonly needsContinuation: boolean }
OutputLimit: { readonly needsContinuation: boolean; readonly early: boolean }
}>
export const Outcome = Data.taggedEnum<Outcome>()

Expand Down Expand Up @@ -277,7 +277,11 @@ export const make = Effect.gen(function* () {
if (tools.interrupted && Exit.isFailure(joined)) return yield* Effect.failCause(joined.cause)
if (record.failure) return yield* new StepFailedError({ error: record.failure })
const needsContinuation = input.prepared.request.toolChoice?.type !== "none" && record.needsContinuation
if (record.finish?.finish === "length") return Outcome.OutputLimit({ needsContinuation })
if (record.finish?.finish === "length") {
const output = record.finish.usage?.outputTokens
const early = output !== undefined && input.model.limit.output > 0 && output < input.model.limit.output

@opencode-agent opencode-agent Bot Oct 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This compares output tokens with the catalog limit.output, not with the limit actually sent. The two can differ for reasons that have nothing to do with the context window: a context hook or the model options can lower maxTokens, and the OpenAI plugin removes maxTokens altogether, so the provider's default applies. In those cases a normal stop at the full allowance is treated as early and triggers an expensive overflow compaction. Comparing with input.prepared.request.generation?.maxTokens would avoid that. The reply would count as early when it stops below that value, or when that value was cut below the catalog limit to fit the context window.

return Outcome.OutputLimit({ needsContinuation, early })
}
return Outcome.Completed({ needsContinuation })
}),
)
Expand Down
36 changes: 36 additions & 0 deletions packages/core/test/session-runner.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5732,6 +5732,42 @@ describe("SessionRunnerLLM", () => {
})
}

scenario("compacts before continuing only when a length stop ends below the output limit", function* (s) {
yield* setupOverflowRecovery(s)
const truncated = (outputTokens: number) =>
TestLLM.complete(
{ reason: { normalized: "length" }, usage: { outputTokens } },
LLMEvent.textStart({ id: "partial" }),
LLMEvent.textDelta({ id: "partial", text: "Partial" }),
LLMEvent.textEnd({ id: "partial" }),
)
yield* s.llm.push(
truncated(200),
TestLLM.text("## Objective\n- Early stop", "text-summary"),
TestLLM.text("Finished", "text-final"),
)
yield* s.runPrompt("Continue")

expect(s.requests).toHaveLength(3)
expect(userTexts(s.requests[1]).at(-1)).toContain("## Objective")
expect(userTexts(s.requests[2])[0]).toContain("<summary>\n## Objective\n- Early stop\n</summary>")
expect(userTexts(s.requests[2]).at(-1)).toContain("output token limit")
expect(yield* s.context).toMatchObject([
{ type: "compaction" },
{ type: "synthetic" },
{ type: "assistant", finish: "stop" },
])
expect((yield* s.messages).some((message) => message.type === "assistant" && message.finish === "length")).toBe(
true,
)

s.requests.length = 0
yield* s.llm.push(truncated(1_000), TestLLM.text("Finished", "text-full"))
yield* s.runPrompt("Continue again")
expect(s.requests).toHaveLength(2)
expect(userTexts(s.requests[1]).at(-1)).toContain("output token limit")
})

for (const tools of ["hosted", "unfinished"] as const) {
scenario(`caps output token limit continuations and resets for new input (${tools})`, function* (s) {
const toolEvents =
Expand Down
Loading