Skip to content

fix(litellm): record partial output on early stream close - #4016

Open
px12811 wants to merge 1 commit into
Arize-ai:mainfrom
px12811:fix/litellm-partial-stream-output
Open

px12811 wants to merge 1 commit into
Arize-ai:mainfrom
px12811:fix/litellm-partial-stream-output

Conversation

@px12811

@px12811 px12811 commented Oct 10, 2026

Copy link
Copy Markdown

Fixes #4011

Summary

LiteLLM streaming spans only recorded output.value and llm.output_messages after the stream loop completed. If an application stopped iterating early and closed or dropped the stream, the generator was closed at its yield and the span was ended with no output attributes.

This change extracts the streaming output finalization into a shared helper. Normal completion records the full output as before, while generator shutdown also records the output received so far before ending the span.

Validation

  • Added sync and async regression tests that consume one chunk, close the stream, and assert both output.value and llm.output_messages.0.message.content
  • tox run -e py313-ci-litellm on macOS: 131 passed
  • Focused tests/test_streaming_transparency.py: 8 passed on Python 3.10, 3.13, and 3.14
  • Ruff format, Ruff lint, and mypy passed

@px12811
px12811 requested a review from a team as a code owner October 10, 2026 07:53
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@px12811

px12811 commented Oct 10, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Oct 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[bug] litellm: a stream the app stops reading early records no output (llm.output_messages / output.value missing)

1 participant