Repository navigation
Conversation
…and in as a Blob A Blob over a file holds no bytes: each read of it reads the file again. ReadableStream::to_any_blob lifts the file out of an unread stream for every caller. It now takes what the caller does with the Blob, and its File arm decides: - Read (unchanged): reads the Blob once, or hands the file to a child. - Keep (blob(), clone()): not a descriptor, a FIFO or a socket. Those give their bytes once, so blob() reads the stream and holds the bytes. - InMemory (formData(), macros, fetch upload): no file. These use the bytes where they are, and a Blob over a file has none. A Body reader whose stream cannot start now rejects its promise. Before, the exception was thrown at the call.
|
Updated 8:06 PM PT - Nov 8th, 2026
❌ @robobun, your commit bab7745 has some failures in 🧪 To try this PR locally: bunx bun-pr 48259That installs a local version of the PR into your bun-48259 --bun |
|
Status: draft. It waits for a maintainer's answer to the three questions in the Notes of the description. How it was reproduced (Linux x64): // c.mjs printf abcde | bun c.mjs
const b = await new Response(Bun.stdin.stream()).blob();
console.log(b.size, await b.text(), JSON.stringify(await b.text()));
With // m.mjs the file does not exist
const form = { headers: { "content-type": "application/x-www-form-urlencoded" } };
const r = new Response(Bun.file("zz-none.txt").stream(), form);
console.log(await r.formData().then(f => "fields: " + [...f].length, e => "rejects " + e.code));
On a debug build of main 2596ac5, 20 of the 31 new cases in CI (build 138968, |
Since 1.4.0, formData() resolves with no field over such a stream and text() throws at the call. Both reject with ENOENT here. blob() is not in the list: it gives the Blob of the path.
Problem
await new Response(Bun.stdin.stream()).blob()resolves with a Blob over fd 0. From a pipe,sizeisInfinityandtext()givesabcde, then"".Bun.file(fd).stream()and a FIFO by path do the same. 1.3.14 held the bytes.formData()over aBun.file().stream()body resolves with no fields, even for a missing file. A macro that returns one inlines"".Source::Filearm ofReadableStream::to_any_blob(src/runtime/webcore/ReadableStream.rs:214) gives every caller a Blob over the file of an unread stream.Fix
to_any_blobtakes what the caller does with the Blob (Lift). The File arm asksblob::file_store_lifts.Keep(blob(),clone()) refuses a descriptor, a FIFO and a socket.InMemory(formData(), macros, fetch upload) refuses every file.Read, all other callers, is unchanged.test/js/web/fetch/body.test.ts(31 new cases, 20 fail on main),macro-test.test.ts(1). Suites: Notes.Background
Locked. Until something reads the stream, Bun can lift its payload out. For aBun.file()stream that is a Blob, which reads the file on each read.clone()(webcore: clone() keeps the Blob behind an unread native body stream instead of teeing it #42053) and fetch upload (Keep body stream bookkeeping independent of the body's source #42116) already refuse this lift.blob()only:formData()and macros stay wrong. Refusing a descriptor forbytes()and Bun.serve too changes calls that are right today (Notes).Downsides
blob()overBun.file(path).stream()does 1statperBun.fileon POSIX. Main does 0.process.stdin.pause(),blob()andformData()overBun.stdin.stream()do not settle, liketext()on main (process.stdin: release the reader in _destroy and make destroy() final #48174).blob()over a 64 MiBBun.file(fd).stream(): 184 ms, +197 MiB. Each stream-body read: +66 instructions.Notes
Draft, with three questions for a maintainer. No issue reports this. A fuzz run found it.
read_body()and alift_unread_stream()with no argument (Body.rs:1701at edb9ce8), and by reading it,text()lifts there too. On that branch this PR is the File arm plus onematch action:GetBlobisKeep,GetFormDataisInMemory. Which side carries it?await new Response(Bun.file(path).stream()).blob()stays the lazy, typed Blob of the file, as on main and asclone()keeps it (body-clone.test.ts:1068). So the file can change under it, a missing path resolves and fails at the first read, andnew Blob([b, b])is 0 bytes (Blob constructor ignores File parts if more than one part is given #25851). A temporary file that the program removes after theawaitloses its bytes:blob.text()rejects withENOENT. This is so since 1.4.0, and it was so in 1.2.23. 1.3.0 to 1.3.14 and the Fetch specification give a copy of the bytes with type"". Switching is one word:Lift::KeeptoLift::InMemoryatBodyMixin::blob.bytes(),arrayBuffer(),json(), Bun.serve and Bun.write still lift a descriptor. OverBun.file(fd).stream()afterreadSync(fd, buf, 0, 3),bytes()giveslo txtforhello txtand moves the descriptor to the end. A Bun.serve handler that returns such a Response over one descriptor answers 1 of 3 requests.Lift::Read => !is_fd()fixes both. I built and measured it, and left it out: for a 64 MiB filebytes()goes from 33 to 67 ms and +67 MiB to 87 to 90 ms and +132 MiB, Bun.serve answersHEADchunked, and these readers then also wait afterprocess.stdin.pause(). It belongs with Read a regular-file stdin from its position in process.stdin and Bun.stdin.stream() #46700, Bun.stdin.stream(), process.stdin: read an unsliced file from its current offset #41687 and Bun.serve: read a Bun.file(fd) body at its offset, not at the descriptor's position #45266.History (each release run with
printf abcde | bun c.mjs):size,text(),text()againabcde,abcde"",""(#13704 added the lift to the readers)abcde,abcde(#23313 moved the stream out ofLocked.readable, so the lift found none)abcde,""(#36624 keepsLocked.readableas a weak handle)Rows that this PR changes, main 2596ac5 against this branch (release builds, Linux x64). stdin holds
abcde:< in.binsize,text(),text()againarrayBuffer()twiceslice(1, 3).text(), thentext()new Response(b).text()twicenew Blob([b, b])Bun.write("out.bin", b)b.typeRequestformData()over a file stream, urlencoded[]formData()over a file stream, multipartnew Response(Bun.file(p).stream())""data:;base64,…as 1.3.14r = new Response(Bun.file(fd)); r.body; await r.blob()text()throws at the call,blob()andformData()resolveEBADF,EISDIR)text()overBun.file("/missing").stream()ENOENTformData()overBun.file("/missing").stream()ENOENTThe last two rows are since 1.4.0. Each ran in five forms on 1.3.14, 1.4.4 and this branch:
.catch()in an async function, the second argument of.then(),Promise.all, atryaround the call, and atryaround theawait. Main 2596ac5 ran the.catch()form and matches 1.4.4.text()of a Response or a Request: 1.2.23 and 1.3.14 reject. Since 1.4.0 the call throws, so no handler on its promise runs, in an async function too. Only atrysees the error. This branch rejects.formData(): 1.3.14 throws at the call. 1.2.23 and 1.4.0 to main resolve with no field, so nothing sees an error. This branch rejects.json()rejects on each release.arrayBuffer()andbytes()threw at the call on 1.3.14 and reject since 1.4.0.blob()threw at the call on 1.3.14. 1.2.23, 1.4.0 to main and this branch give the Blob of the path (question 2).Unchanged:
new Response(Bun.stdin).blob()andBun.stdinitself (the Blob the program gave),bytes()/arrayBuffer()/json(), Bun.serve, Bun.write, Bun.spawn stdin, shell redirects, socket writes, static routes.Measurements. Release builds of main 2596ac5 and this branch. Tools:
objdump,nm,size, a ptrace syscall counter,gdbsingle-step.perf,valgrindandstraceare not installed here.Value::to_blob_if_possibleruns the same 18 instructions and 2 conditional branches as on main.staton the JS thread forblob()over a path-file stream: +1newfstatatper 1000 calls on oneBun.file, +1000 per 1000 freshBun.file, +0 when the file was already stat'd.bytes(),json(),arrayBuffer(): +0. On Windows thestatis skipped: by path it cannot tell a named pipe.formData()over a path-file stream: main makes no syscall (and parses nothing). This branch:openat,fstat, 2pread64,closeper call, the same astext().blob()overBun.file(fd).stream(), 9 bytes: main reads nothing at the call, then three reads of the Blob give 9, 0, 0 bytes and the descriptor ends at 9. This branch reads at the call (2fcntl, 1fstat, 2pread64, 1close), the three reads give 9, 9, 9 with no syscall, and the descriptor stays at 0.blob()and onearrayBuffer(): main 44 ms and +65 MiB (and a second read is empty), this branch 184 ms and +197 MiB, 1.3.14 206 ms and +200 MiB.Bun.readableStreamToBlob(stream)is the same read on both builds.PendingValue::set_promisefor onetext()of a JS stream body: 5,543 instructions on main, 5,609 here (+66, no allocation)..text: +1,024 bytes. The stripped binary has the same size.heapStats().objectCountonly moves at a collection.Left as they are, with their owners.
blob(): its stream rejectsEPERMon Linux since Release the poll and surface the error when a Bun.file() stream read fails #41420 (Read a character device synchronously when the kernel refuses to poll it #41621, Bun.file().stream(): read /dev/null, /dev/zero and other devices the event loop refuses to watch #44499)./dev/stdinor/dev/fd/Nis a dup. With a regular file behind it,statsays regular andblob()still lifts. Not run here (Bun.write, fs.writeFile: do not resize the file behind /dev/stdout or /dev/fd/N (macOS, FreeBSD) #44541 has the predicate). The macOS-only test covers the socket case. It passes on the darwin aarch64 lane.clone()keeps its own stricter check before the arm (clone(): keep the stat of an unread Bun.file().stream() body out of the shared Store #46666 owns itsstat). The helpermode_for_clonehere has the name and body of the one in clone(): keep the stat of an unread Bun.file().stream() body out of the shared Store #46666.new Response(Bun.file(p)).formData()(the Blob itself as the body): formData(): read a file- or S3-backed Blob body before parsing it #45161. A macro that returns the Blob itself: macros: read a returned Bun.file() on the transpiling thread and inline its bytes #46688.getReader(), froms.text(),s.json(),s.bytes()ands.blob(), and from the sevenBun.readableStreamTo*helpers. Four of them rejected on 1.2.23 and 1.3.14 and throw since 1.4.0:s.text(),s.json(),Bun.readableStreamToText()andBun.readableStreamToJSON(). This PR wraps the Body readers only. The stream layer is streams: record a failed Bun.file() stream start so promise readers reject and later reads settle #42196, which has a test row for each of the four. With itsBunStreamSource.cppchange (75cb269) applied on this branch, the four reject, and bothbody.test.ts(806 pass) and itsstreams.test.js(754 pass) are green.Suites on the debug build of this branch:
body.test.ts806 pass,body-clone.test.ts100,body-stream.test.ts9086,body-mixin-errors23,body-async-iterator50,body-stream-excess4,macro-test.test.ts35,FormData.test.ts154,streams.test.js727,html-rewriter.test.js202,wasm-streaming33,blob.test.ts161,response23,request19,bun-serve-file176,bun-serve-static47,spawn-stdin-readable-stream52,process-stdin23,bunshell464,readable-stream-blob-consumed,bun-stdin-slice, regression 29267.fetch.stream.test.tshas 18 timeouts, a subset of the 20 on a debug build of main.serve.test.ts: the privileged-port case fails as root, and one 10 s case sits at its limit on both builds.bun-write.test.jsandspawn.test.tshave debug timeouts that differ per run, on main too.cargo checkforx86_64-pc-windows-msvcandaarch64-apple-darwin, andcargo clippy -p bun_runtime, are clean.cargo mordantis not installed here.Self-reviewed: 45 concerns raised. The review asked for this smaller cut in place of a required purpose at 28 call sites, and for the promise wrap, the kernel-pipe and multipart cases and the Windows
statgate: done. Not done: carrying the stdin fix of #48174 (named above), and a flag for whose stream it is (the.bodyrow of the table is pinned as it stands).no test proof · iteration 2 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/web/fetch/body.test.ts