Skip to content

Add llmmend/cost: estimateCost and toUsage - #91

Merged
nightlightmare merged 1 commit into
mainfrom
cost-module
Oct 27, 2026
Merged

nightlightmare merged 1 commit into
mainfrom
cost-module

Conversation

@nightlightmare

Copy link
Copy Markdown
Owner

Summary

estimateCost(usage, prices)
  p ← prices, overlaid by the tier with the highest `above` < inputTokens
  input      = max(0, inputTokens − cacheRead − cacheWrite) × p.input
  cacheRead  = cacheReadTokens  × (p.cacheRead  ?? p.input)
  cacheWrite = cacheWriteTokens × (p.cacheWrite ?? p.input)
  output     = outputTokens × p.output                  (all ÷ 1e6)
  missing    = 'input' | 'output' whose count is absent (absent if none)

toUsage(raw)
  promptTokenCount…           → fromGemini
  prompt_tokens…              → fromChat
  cache_*_input_tokens…       → fromAnthropic   (adds cached tokens to input)
  input_tokens / output_tokens→ fromResponses
  anything else               → {}
 src/
+├── cost/              # estimateCost, toUsage
 ├── shared/
+│   └── usage.ts       # fromAnthropic, fromChat, fromResponses, fromGemini
 └── sse/providers/
-    anthropic.ts, openai.ts, openai-responses.ts, gemini.ts: each its own usage conversion
+    the same files, calling src/shared/usage.ts
  • The stream adapters and toUsage now share one conversion per Provider, so a raw reply and a stream of the same call give the same LlmUsage. One behaviour change in the adapters: a null count (Anthropic sends cache_creation_input_tokens: null in some cases) is treated as absent instead of passing through.
  • README section, API and size rows, docs/api.md section: all in three languages. llms.txt regenerated.

Evidence

toUsage({ input_tokens: 1_200, cache_read_input_tokens: 40_000, cache_creation_input_tokens: 0, output_tokens: 800 })
→ { inputTokens: 41_200, outputTokens: 800, cacheReadTokens: 40_000, cacheWriteTokens: 0 }
estimateCost(↑, { input: 3, output: 15, cacheRead: 0.3, cacheWrite: 3.75 })
→ { total: 0.0276, input: 0.0036, cacheRead: 0.012, cacheWrite: 0, output: 0.012 }

Before: the raw Anthropic input_tokens (1,200) priced as all input misses the 40,000 cached tokens; adding them at the input price overstates the cost tenfold.
After: 12 new tests (tiers, missing counts, every raw shape, unknown shapes); all 342 SSE tests pass unchanged on the shared conversion; 875 tests, docs, smoke, publint and attw pass. llmmend/cost is 689 B of its 1 kB limit; the SSE bundles did not grow (collectText 6.78 kB, streamToolCalls 6.92 kB).

Merge Danger

Door: two-way

Blast Radius: stream usage events

Additive module, but the stream adapters' usage code moved to src/shared/usage.ts. The recorded fixtures for all four Providers pass unchanged; the only difference is null counts, now dropped.

Closes #90

🤖 Generated with Claude Code

https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS

Prices a call's token usage with prices the caller supplies per million
tokens, by part, with long-context tiers and a list of missing counts.
toUsage reads raw Provider usage; the stream adapters now share its
conversions (src/shared/usage.ts).

Closes #90

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS
@nightlightmare
nightlightmare merged commit ac96474 into main Oct 27, 2026
9 checks passed
@nightlightmare
nightlightmare deleted the cost-module branch October 27, 2026 21:03
nightlightmare added a commit that referenced this pull request Oct 27, 2026
Releases 0.6.0.

## What ships

From `CHANGELOG.md`, three new modules, all **minor**:

- `llmmend/prompt`: `safePrompt`, `trusted`, `escapeTags`: untrusted
text can't close or forge a prompt's own tags (#89).
- `llmmend/cost`: `estimateCost`, `toUsage`: the cost of a call by part
from its usage and prices per million tokens (#91).
- `llmmend/retry`: `classifyError`: eleven provider-neutral error kinds,
`retryable` and the Provider's `retryAfter` (#94).

One **patch**: stream `usage` events leave out a count the Provider sent
as `null`; the four adapters now share one usage conversion with
`toUsage` (#91). Added to the changelog by hand, since #91's changeset
described only the new module.

The playground's Prompt, Cost and Errors tabs (#92, #94) are already
live on Pages.

## Before release

- **Provider adapters** changed in #91 (the shared usage conversion), so
the providers were re-recorded and compared with the fixtures, then
restored:
- **Anthropic** (via Bedrock), all seven cases: same payload fields, LLM
event types, usage fields, stop reasons; `responseId` set in all.
- **Gemini**: `text`, `thinking`, `max-tokens` the same on every count.
`tool-calls-parallel` got HTTP 503 `UNAVAILABLE` (high demand), which
`classifyError` reads as `overloaded`, retryable.
- **Drift:** the Ollama drift check passes on main at f3af34c, after the
adapter change
([run](https://github.com/nightlightmare/llmmend/actions/runs/40589829344)).
- **Size:** within limits. `prompt` 582 B, `cost` 689 B, `retry` 1.39 of
1.5 kB; the existing bundles did not grow (`collectText` 6.78 kB,
`streamToolCalls` 6.92 kB).
- **Local checks:** tests (911), docs, size, smoke on all seven
subpaths, and `check:publish` pass. CI on main is green at f3af34c.

## After merge

`release.yml` stages 0.6.0 with `npm stage publish` and creates the tag
and GitHub release; approve the staged version on npmjs.com with 2FA.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cost from usage: estimateCost and toUsage

1 participant