Skip to content

Add Prompt and Cost tabs to the playground - #92

Merged
nightlightmare merged 1 commit into
mainfrom
playground-prompt-cost
Oct 27, 2026
Merged

nightlightmare merged 1 commit into
mainfrom
playground-prompt-cost

Conversation

@nightlightmare

Copy link
Copy Markdown
Owner

Summary

 <nav class="tabs">
   JSON · SSE stream · Chunks
+  · Prompt     template with {{input}} → safePrompt, escaped tags marked
+  · Cost       pasted usage or reply → toUsage → estimateCost, by part, missing counts
 Chunks
+  tokenizer: OpenAI (o200k) | Claude | Gemini
-  note: "Claude counts about 1.5× as many"
+  note: calibrated per tokenizer (±15 %), countTokens for exact counts
  • New pages in all three languages (/prompt/, /cost/, /ru/…, /zh-CN/…), with their own titles, descriptions and sitemap entries.
  • Prompt: the template is a textarea (shared in links like other options); examples: a document that closes its wrapper and opens <instructions>, and code with a < b, <system> and List<Document> that stays as written.
  • Cost: prices per 1M tokens as fields (an empty cache price costs those tokens as input) and an optional long-context tier. A whole reply works as well as its usage / usageMetadata. Examples: Anthropic with a cache read, OpenAI Chat Completions, Gemini long context, a cut-off stream with no output count.
  • README playground line and llms.txt mention the new tabs.

Evidence

Checked in the built site (vite preview):

  • Prompt, "Closes its wrapper": Tags escaped: 4; the result has &lt;/document>, &lt;instructions>…&lt;/instructions>, &lt;document> inside the real <document>.
  • Cost, Anthropic example: Total: $0.0276 (input $0.0036, cache read $0.012, output $0.012); LlmUsage inputTokens: 41200.
  • Cost, cut-off example: Missing from the usage, costed as 0: output.
  • Chunks (ru): the tokenizer select is there; 30 chunks of the README.

Before: the price field id="input" collided with the input textarea, which left the cost tab blank; caught in the preview, fixed by prefixing the price ids with price-.
After: all tabs render; typecheck, lint, format, docs checks and 875 tests pass.

Merge Danger

Door: two-way

Blast Radius: playground

Site only: nothing in the package changes. The Chunks share links of earlier versions still open (a missing tokenizer falls back to o200k).

🤖 Generated with Claude Code

https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS

Prompt shows safePrompt on a template with an {{input}} slot, marking
each escaped tag; Cost prices a pasted usage or reply with toUsage and
estimateCost. Chunks gains the tokenizer option, and its note no longer
says Claude counts 1.5x: the presets replaced that advice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS
@nightlightmare
nightlightmare merged commit c823ab0 into main Oct 27, 2026
9 checks passed
@nightlightmare
nightlightmare deleted the playground-prompt-cost branch October 27, 2026 21:09
@nightlightmare nightlightmare mentioned this pull request Oct 27, 2026
nightlightmare added a commit that referenced this pull request Oct 27, 2026
Releases 0.6.0.

## What ships

From `CHANGELOG.md`, three new modules, all **minor**:

- `llmmend/prompt`: `safePrompt`, `trusted`, `escapeTags`: untrusted
text can't close or forge a prompt's own tags (#89).
- `llmmend/cost`: `estimateCost`, `toUsage`: the cost of a call by part
from its usage and prices per million tokens (#91).
- `llmmend/retry`: `classifyError`: eleven provider-neutral error kinds,
`retryable` and the Provider's `retryAfter` (#94).

One **patch**: stream `usage` events leave out a count the Provider sent
as `null`; the four adapters now share one usage conversion with
`toUsage` (#91). Added to the changelog by hand, since #91's changeset
described only the new module.

The playground's Prompt, Cost and Errors tabs (#92, #94) are already
live on Pages.

## Before release

- **Provider adapters** changed in #91 (the shared usage conversion), so
the providers were re-recorded and compared with the fixtures, then
restored:
- **Anthropic** (via Bedrock), all seven cases: same payload fields, LLM
event types, usage fields, stop reasons; `responseId` set in all.
- **Gemini**: `text`, `thinking`, `max-tokens` the same on every count.
`tool-calls-parallel` got HTTP 503 `UNAVAILABLE` (high demand), which
`classifyError` reads as `overloaded`, retryable.
- **Drift:** the Ollama drift check passes on main at f3af34c, after the
adapter change
([run](https://github.com/nightlightmare/llmmend/actions/runs/40589829344)).
- **Size:** within limits. `prompt` 582 B, `cost` 689 B, `retry` 1.39 of
1.5 kB; the existing bundles did not grow (`collectText` 6.78 kB,
`streamToolCalls` 6.92 kB).
- **Local checks:** tests (911), docs, size, smoke on all seven
subpaths, and `check:publish` pass. CI on main is green at f3af34c.

## After merge

`release.yml` stages 0.6.0 with `npm stage publish` and creates the tag
and GitHub release; approve the staged version on npmjs.com with 2FA.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01F61ff7dDPwJn5kuD25LtiS

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant