test(gateway): correct the Anthropic SSE stub to the shape the API emits - #181
Conversation
anthropicSseBody() puts `usage` inside `delta` on message_delta. The API puts it at the top level, as a sibling of `delta`, and `delta` carries only the stop fields — see RawMessageDeltaEvent in @anthropic-ai/sdk, already a dependency of services/api. message_start is also thin: the real frame carries a full Message envelope, and usage there reports output_tokens alongside the input and cache counts. The shape matters because the stub is what every Anthropic streaming assertion in this suite is measured against. Reading the top-level `usage` is what meters streamed output tokens in production, and no test exercised that path: with the old stub, deleting the top-level source from UsageTee left the streaming tests green. The corrected stub also sends the null cache counters an uncached request reports, so the merge behaviour is exercised rather than assumed. No production behaviour changes; the suite passes unchanged (52/52).
adrian-lorenzo
left a comment
There was a problem hiding this comment.
Thanks for the contribution!
Since #148 has now landed, please update this branch and also correct the MESSAGE_DELTA fixture in services/gateway/test/usage.test.ts, plus the comment in services/gateway/src/usage.ts that still says output usage is under delta.usage. Otherwise the integration stub will match Anthropic’s wire format while the new unit fixtures and production documentation continue to encode the old shape.
… shape Review follow-up on theam#181, after theam#148 landed: the integration stub now matches Anthropic's wire format, but the `MESSAGE_DELTA` unit fixture in `usage.test.ts` still nested `usage` inside `delta`, and the comment in `usage.ts` still documented output usage as living under `delta.usage`. The fixture now emits the real frame — `usage` as a sibling of `delta`, with `delta` carrying only the stop fields — and the comment states that shape. The `parsed.delta` merge source is deliberately kept as tolerance for relays that nest usage inside the delta, and the comment now says so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Done — and since this is the branch's first update: it is updated by merge, not rebase, so the commit you reviewed stays as it was.
Gateway suite 57/57 against |
adrian-lorenzo
left a comment
There was a problem hiding this comment.
Thanks for the solid contribution!
What changes
anthropicSseBody()and the abort-stream stub now emit the framesapi.anthropic.comactually sends:usageat the top level ofmessage_delta, as a sibling ofdelta, withdeltacarrying only the stopfields; a full
Messageenvelope onmessage_start; and thenullcache counters an uncached requestreports.
No production code changes. The suite passes unchanged.
Why
The stub is what every Anthropic streaming assertion in
gateway.test.tsis measured against, and itencodes a frame the provider never sends.
usageis a sibling ofdelta— the shape is typed in adependency this repo already has,
@anthropic-ai/sdk,resources/messages/messages.d.ts:That matters for what the tests can catch.
UsageTeereads usage from the top-levelparsedobject andfrom
parsed.delta; on the real wire the first is what meters every streamed output token and the secondnever fires. With the old stub the coverage is inverted — mutating
services/gateway/src/usage.tsonmain:parsed.deltaread — dead against the real wireparsedread — meters real streamed outputSo the suite pinned dead code and left the live metering path unpinned. Cost from that path lands in
spend_countersand is enforced byhardBudgetBlock, which is why it seemed worth correcting rather thanleaving.
The shape has been in the stub since
a6553e9(2020-07-03), so this is independent of any open work. PR #148copies the same shape into new unit tests; I have left the detail on that thread rather than touching it
here, since its production fix is correct.
Verification
Beyond the suite: the mutation table above, run both ways on
main. Postgres fromdocker-compose.dev.yml; without it the DB-backed cases skip and the count drops to20 passed | 1 skipped.pnpm verifypasses locally — not run. Gateway package suite + biome only. Say the word if you want the full run before review.