Repository navigation
feat(core): add byte-preserving offline media_blob backfill for legacy rows - #58742
Draft
kitlangton wants to merge 8 commits into
Draft
kitlangton wants to merge 8 commits into
kitlangton wants to merge 8 commits into
Conversation
This was referenced Oct 24, 2026
Merged
kitlangton
force-pushed
the
media-write
branch
from
October 24, 2026 21:27
a2690cb to
e024cce
Compare
kitlangton
force-pushed
the
media-backfill
branch
from
October 24, 2026 21:27
d6c1ed7 to
089383c
Compare
kitlangton
force-pushed
the
media-write
branch
from
October 25, 2026 17:42
e024cce to
27249de
Compare
kitlangton
force-pushed
the
media-backfill
branch
from
October 25, 2026 20:19
089383c to
13a0ed5
Compare
kitlangton
force-pushed
the
media-backfill
branch
2 times, most recently
from
October 25, 2026 23:26
af9d7d0 to
d541fcb
Compare
…l-media-uri Clients that send X-OpenCode-Capabilities: experimental-media-uri get user files and tool results stored in blob as media:<hash> refs with their size and dimensions instead of inline base64, from session.messages, session.message and session.inbox.list, when experimental.media_blob is on in the session's own location (the config its media was written under). Refs whose blob is gone become the same unavailable placeholder inline readers get. Every other reader, the runner and export stay inline. GET /api/experimental/media/:hash serves one blob that a stored message or inbox item still references, with an immutable ETag, 304 support, nosniff and a sandbox CSP; anything but a raster image is sent as an application/octet-stream attachment. session.prompt accepts a referenced media:<hash> ref as an attachment URI and re-admits its stored bytes as is.
kitlangton
force-pushed
the
media-backfill
branch
from
October 25, 2026 23:38
d541fcb to
c1697d1
Compare
kitlangton
force-pushed
the
media-backfill
branch
from
October 26, 2026 00:22
c1697d1 to
773dfb5
Compare
…he media loader, and submit bytes
The server sends media:<hash> refs, never route paths, and user files keep their source, so drop the legacy route parsing and the origin fallback.
Reverting an undelivered prompt, undoing a queued prompt and editing one all end in an inbox cancel that sweeps blobs nothing else references. Each now copies its media: bytes through the media cache into local draft blobs first; revert and undo leave the prompt queued when the bytes cannot load.
…ssion rows script/backfill-blobs.ts moves inline base64 attachments in existing session_message and session_inbox rows into blob and records their blob_ref rows, using SessionBlob.split so rows end up exactly as the live write path stores them. It refuses to run before the blob migration, batches under BEGIN IMMEDIATE with a short busy timeout, skips rows that change mid-run, is safe to re-run, and can VACUUM INTO a compacted copy.
kitlangton
force-pushed
the
media-backfill
branch
from
October 26, 2026 01:05
773dfb5 to
18fdb91
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does
Adds an offline script that moves inline base64 media in existing session rows into the
blobtable, so databases written before the blob write path shrink too. It's experimental and something you run by hand. Nothing runs it automatically, and theexperimental.media_blobflag (off by default) still decides only how new writes are stored.Stack, built on the merged #58656: write (#58741, merged) → wire (#59103) → app (#59104) → backfill (this PR).
Optional offline cleanup: last in the stack, and it can land later. Clients don't need it.
Why
Every image or file a user attached, and every tool screenshot, is stored as base64 inside its message row, a third larger than the raw bytes. The same image is copied again into the inbox row and into each update of a tool result. #58741 stops that for new writes. This handles the rows that already exist. Before → after database size has not been measured yet.
How it works
blobmigration is recorded in the database, so start OpenCode once on it first. It never creates tables.session_messageandsession_inboxrow that has inline media, it stores the decoded bytes once inblob, swaps in amedia:<hash>ref and records ablob_refrow, usingSessionBlob.splitfrom the wire PR. It's byte-preserving: nothing is resized or re-encoded, so reads return the same data as before.BEGIN IMMEDIATEwith a 1s busy timeout, so it fails fast when a server holds the lock. A row that changed after it was read is skipped. Re-running is safe: rows that already hold refs only get their missingblob_refrows.--vacuum-into <out.db>writes a compacted copy. The database path is read correctly whether it comes before or after the flag.Risk
It's an offline tool that rewrites rows in the database you point it at, so back it up first. Running it against a database a server is writing to fails fast instead of waiting.
--vacuum-intoneeds free disk space about the size of the database. Reads already understand ref rows (from #58656), so backfilled rows read the same whether the flag is on or off.Proof
bun test test/session-blob-backfill.test.ts(packages/core): 3 pass. They cover byte-for-byte storage, recorded refs and an idempotent re-run; refusal before the migration with the database untouched; skipping a row that changed mid-batch; and failing fast when another writer holds the lock. They ran with the core prompt, owned, projector and store tests at this head (95 pass).bun run typecheck(core): clean.oxlinton changed files: 0 errors.