Skip to content

feat(server): add experimental GET /api/experimental/media/:hash and opt-in experimental-media-uri capability - #59103

Merged
kitlangton merged 3 commits into
devfrom
media-wire
Oct 26, 2026
Merged

kitlangton merged 3 commits into
devfrom
media-wire

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What it does

Lets a client skip inline base64 when reading a session and fetch attachment bytes only when it needs them. It's experimental and opt-in at both ends: the client has to ask for refs, and the session's location needs experimental.media_blob on, which is off by default. Every other client and session gets inline base64 exactly as today.

Replaces #58657, which GitHub marked merged into its stack base while the stack was being reordered. Nothing from it reached dev.

Stack, built on the merged #58656: write (#58741, merged) → wire (this PR) → app (#59104) → backfill (#58742, optional, can land later).

Why

With media stored once in blob (#58741), message reads still turn every ref back into base64, so each load of a long session with screenshots re-sends every image, a third larger than the raw bytes. With refs, a read carries a short media:<hash> per file, and a client fetches each image once and caches it. Before → after bytes per read have not been measured yet.

How it works

  1. A client opts in with X-OpenCode-Capabilities: experimental-media-uri. Only session.messages, session.message and session.inbox.list have ref variants. They return stored files as media:<hash> refs with their size in bytes. No route path is sent or stored.
  2. Whether refs are sent follows the flag in the session's own location config, not the server's working directory. If that config can't be resolved, the read logs a warning and falls back to inline.
  3. GET /api/experimental/media/:hash (generated client: experimental.media.get, OpenAPI tag experimental) serves a blob only while a message or inbox item still references it. A blob that's been released or is waiting to be swept returns 404. Responses carry immutable Cache-Control, an ETag with 304 support, nosniff and a sandbox CSP. Anything other than PNG, JPEG, GIF or WebP downloads as application/octet-stream.
  4. A ref whose blob is gone reads as the [media unavailable: <name> (<mime>)] placeholder instead of a dead link.
  5. A prompt may send a media:<hash> ref back as an attachment. The server re-admits the stored bytes unchanged, without normalizing them again, and rejects a ref nothing references anymore.

Risk

Server, protocol and the generated client. The new route and response shapes are experimental, and clients that don't opt in see no change. These still send media inline on purpose: session.context, the session.prompt response, session export and import, and the runner's model requests. Legacy /api/media/ URLs are not supported, since refs are the only form sent or accepted.

Proof

  • bun test test/media.test.ts (packages/server): 2 pass. They cover the ref and inline shape switch; headers, ETag and 304; the missing-blob placeholder; following the flag from the session's project config when the server's own directory has it off; and a 404 plus a rejected prompt for an unreferenced blob.
  • bun test for the core prompt, owned, projector and store tests: 92 pass.
  • bun run typecheck (core, server, client): clean.

Added by the opencode bot (AI-generated).

Related: #36444, #58436

Public API: adds experimental GET /api/experimental/media/:hash (operationId experimental.media.get), an opt-in X-OpenCode-Capabilities: experimental-media-uri header for message/inbox reads, media: prompt attachments and new Media* schemas (backwards compatible)

…l-media-uri

Clients that send X-OpenCode-Capabilities: experimental-media-uri get user
files and tool results stored in blob as media:<hash> refs with their size
and dimensions instead of inline base64, from session.messages,
session.message and session.inbox.list, when experimental.media_blob is on
in the session's own location (the config its media was written under).
Refs whose blob is gone become the same unavailable placeholder inline
readers get. Every other reader, the runner and export stay inline.

GET /api/experimental/media/:hash serves one blob that a stored message or
inbox item still references, with an immutable ETag, 304 support, nosniff
and a sandbox CSP; anything but a raster image is sent as an
application/octet-stream attachment. session.prompt accepts a referenced
media:<hash> ref as an attachment URI and re-admits its stored bytes as is.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found nothing that blocks a merge, and the change works as described. The new server tests run the real server through ServerFetch and the generated client, both with and without the experimental-media-uri header and with experimental.media_blob on and off. They pass, as does session-owned.test.ts, and typecheck:tsc is clean for core, server and client.

Public API (additive, backwards compatible):

  • New GET /api/experimental/media/:hash (media.get, tag experimental; promise client experimental.media.get, left out of the effect client).
  • New opt-in request header X-OpenCode-Capabilities: experimental-media-uri. It changes session.messages, session.message and session.inbox.list only when the session's location also has experimental.media_blob on.
  • New schemas: Prompt.MediaRef, Prompt.MediaFileAttachment, Prompt.Attachment, Session.Message.MediaUser/MediaInfo, Session.Inbox.MediaUser/MediaUserPayload/MediaInfo.
  • Prompt attachments now also accept a media:<hash> URI.
  • server.experimental now holds both the persistent PTY routes and the media route.
  • Clients that don't send the header see no change.

Worth a look:

  • With refs on, file items in assistant tool output also come back as media:<hash>. describe only adds bytes/width/height to user and inbox files, so tool items have no size, and the spec still types Tool.Content.uri as a plain string. Opted-in clients have to handle that case without it being documented.
  • test/media.test.ts is about 360 lines for 2 tests, each a long chain of checks. Splitting it into a few small cases would make failures easier to locate: ref vs. inline shape, headers/304, the missing-blob placeholder, and the unreferenced-blob 404 with the rejected prompt.
  • SessionBlob.split is a new export that only the test uses here, so it could move to the backfill PR (#58742).

Related: #36444, #58436

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit changes only the test. Each test now takes its temp directory from tmpdirScoped before starting the servers, so the servers close the database before the directory is removed (Windows can't delete an open database). The assertions are the same as before. On the new head, bun test test/media.test.ts passes (2 tests) and server typecheck:tsc is clean. Production code and the public API are unchanged since my last review, and I have nothing new to raise.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new commit is sound. Opted-in reads now only offer a media: ref for a blob that something still references, using the same check the media route uses. Any other stored blob reads as the [media unavailable: …] placeholder instead of a ref that would 404, and the added test checks both the 404 and the placeholder. media.test.ts passes (2 tests), typecheck:tsc is clean in core and server, and re-running the protocol and client generators gives no diff.

Public API: the OpenAPI operationId changes from media.get to experimental.media.get. The route is new in this PR, so nothing released depends on the old id, and the generated client already calls it experimental.media.get. Nothing else changed. I have no new concerns.

@kitlangton
kitlangton merged commit e928d4f into dev Oct 26, 2026
21 checks passed
@kitlangton
kitlangton deleted the media-wire branch October 26, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant