Skip to content

fix(litellm): set ERROR status on streaming spans that fail mid-stream - #4015

Open
chiliec wants to merge 1 commit into
Arize-ai:mainfrom
chiliec:fix/litellm-streaming-error-status
Open

chiliec wants to merge 1 commit into
Arize-ai:mainfrom
chiliec:fix/litellm-streaming-error-status

Conversation

@chiliec

@chiliec chiliec commented Oct 10, 2026

Copy link
Copy Markdown

Fixes #4013.

What

The sync (_finalize_sync_streaming_span) and async (_finalize_streaming_span) chat-completion streaming finalizers only called span.record_exception(e) when a stream failed mid-way (connection dropped, MidStreamFallbackError, etc.), leaving the span status UNSET. The non-streaming wrappers already set StatusCode.ERROR, so failed streaming calls were invisible to error views and alerts that filter on span status.

This adds span.set_status(Status(StatusCode.ERROR, str(e))) next to the existing record_exception in both finalizers — matching the non-streaming behavior.

Tests

Added two tests to tests/test_instrumentor.py (test_completion_sync_streaming_sets_error_status_on_mid_stream_failure, test_acompletion_streaming_sets_error_status_on_mid_stream_failure) that drive a stream raising mid-iteration and assert the span ends with StatusCode.ERROR plus one exception event.

Validation

$ python -m pytest tests/test_instrumentor.py \
    -k "test_completion_sync_streaming_sets_error_status_on_mid_stream_failure or test_acompletion_streaming_sets_error_status_on_mid_stream_failure"
2 passed

RED→GREEN confirmed: before the source change both tests fail with assert <StatusCode.UNSET> == <StatusCode.ERROR>; after it, both pass.

Broader streaming/exception/status selection (-k "stream or exception or status"): 22 passed. One unrelated test (test_completion_streaming_with_extended_thinking_anthropic) fails on this machine because its VCR cassette tries to reach a live Anthropic endpoint — it fails identically on the unmodified main, so it is not caused by this change.

The sync and async chat-completion streaming finalizers only recorded the
exception on a mid-stream failure, leaving the span status UNSET. The
non-streaming wrappers already set StatusCode.ERROR, so failed streams were
invisible to status-filtered error views. Set ERROR status next to
record_exception in both finalizers.

Signed-off-by: Vladimir Babin <vovababin@gmail.com>
@chiliec
chiliec requested a review from a team as a code owner October 10, 2026 06:51
@github-actions

Copy link
Copy Markdown
Contributor

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[bug] litellm: streaming span ends with status UNSET when the stream fails mid-way

1 participant