fix: continuation loop never stops (stream.switches is never incremented) - #2187
Open
Cadenadb wants to merge 1 commit into
Open
fix: continuation loop never stops (stream.switches is never incremented)#2187Cadenadb wants to merge 1 commit into
Cadenadb wants to merge 1 commit into
Conversation
…ncremented SwitchableStream._switches only increments inside switchSource(), which api.chat.ts never calls. The safety check that's supposed to cap auto-continuation at MAX_RESPONSE_SEGMENTS (stream.switches >= MAX_RESPONSE_SEGMENTS) therefore never trips, so a response that keeps returning finishReason: 'length' triggers an infinite continuation loop -- one real LLM call every ~40s, forever, silently consuming API credits, with the UI appearing frozen since nothing ever completes. Fix: track continuation attempts with a local counter that's actually incremented on every continuation, restoring the intended 2-segment cap. Observed in production against DeepSeek (deepseek-chat / deepseek-v4-flash) on a long-running chat that had grown close to the 128k token context cap: 10 consecutive identical 'Reached max token limit (128000): Continuing message (2 switches left)' log lines over 8 minutes, message count frozen, real balance drained on every retry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
When a response is cut off by the model's token limit (
finishReason === 'length'),api.chat.tsis supposed to auto-continue at mostMAX_RESPONSE_SEGMENTS(2) times, then stop withCannot continue message: Maximum segments reached.That cap never triggers, because it checks
stream.switches, andSwitchableStream._switchesis only incremented insideswitchSource()-- a method thatapi.chat.tsnever calls. Sostream.switchesstays0forever, and the continuation branch recurses indefinitely for any model that keeps returningfinishReason: 'length'.Impact
Observed in production: a long chat that grew close to the 128k context cap got stuck making one real LLM call every ~40s, forever -- 10 consecutive identical
Reached max token limit (128000): Continuing message (2 switches left)log lines over 8 minutes, no progress, no error surfaced to the UI (it just looks frozen), and real API credits silently consumed on every retry (confirmed against a paid provider).This isn't provider-specific -- any model/setup that legitimately hits the output token cap on a large response will loop forever instead of stopping at 2 segments as intended.
Fix
Track continuation attempts with a local counter (
continuationSegments) that is actually incremented on every continuation, instead of relying on the never-updatedstream.switches. Restores the originally intended 2-segment cap.Minimal, single-file change, no behavior change for the normal (non-looping) path.