Repository navigation
Conversation
On POSIX, PollOrFd::close_impl queued close(2) on the work pool and called on_close in the same frame. FileSink.end() returned, and Bun.write(path, stream) resolved, with the file still open for writing: a script that was written this way and then run got ETXTBSY. A child did not see the end of its stdin while the pool was busy. close_impl now closes the fd on the calling thread before it calls on_close, for every handle. FileReader does the same for a directory. One close can flush a file: the close that follows a truncate on ext4. Bun.write(path, stream) is the only sink that truncates. It now opens with O_EXCL first. A file that exists is opened with O_TRUNC and closed by a bun_jsc::Job on the work pool. The job's completion settles the promise. A new file is closed in place.
|
Status
|
|
Updated 8:05 PM PT - Oct 21st, 2026
❌ @robobun, your commit 9092e8e has some failures in 🧪 To try this PR locally: bunx bun-pr 46088That installs a local version of the PR into your bun-46088 --bun |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 7 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Beyond the inline findings, I also checked the CloseReplacedFile job's completion path for a double release: then runs report_closed and then drops ClosingSink, which repeats release_pipe and clear_keep_alive_ref — both are idempotent (the pipe cell is swapped for a default, the keep-alive deref is flag-guarded), and the single deref balances the ref_ taken in on_close, so the refcount is balanced on both the run and the dropped-unrun job paths.
Extended reasoning...
The change moves close(2) for every POSIX PollOrFd onto the calling thread in src/io/pipes.rs, threads a close_fd flag through PosixStreamingWriter, and adds a work-pool job in src/runtime/webcore/FileSink.rs for Bun.write(existingPath, stream) over a truncated file, plus test updates. It touches no security-sensitive surface (no auth, injection, or data exposure), but it changes fd-lifetime and thread-affinity semantics shared by FileSink, pipe writers and readers, and four confirmed findings (event-loop stalls on slow mounts, VM teardown waiting on the pool, a weakened double-close test, and Windows left unfixed) are posted inline, so a human decision on the design is still needed.
Findings marked 🟡 are optional suggestions and need no follow-up push.
The first revision closed the fd on the calling thread in PollOrFd::close_impl, for every POSIX handle. That also reverses the pool close for pipes, sockets and readers, and no maintainer has ruled on that. This revision leaves close_impl, FileReader and the nine other test files as on main. It fixes the early report where it was measured: a FileSink on a file. A FileSink now owns the fd of a file it opened (close_fd = false on the writer) and closes it in on_close, before it reports: - Bun.write(dest, stream) reports through a promise. The pool closes the file in a bun_jsc::Job that is filed under the script that called Bun.write(), and the completion settles the promise. - writer.end() and writer.close() report when they return. The file is closed in place. - A sink that is never ended goes to Closer, as on main. A direct stream whose pull() throws now also rejects the promise of Bun.write() after the close. The tests park each pool thread on a FIFO of its own. Two readers on one FIFO hang on macOS: fifo_read clears the end of file after the first reader gets it.
| if let Some(file) = (*this).file.take() { | ||
| // Only `Bun.write(dest, stream)` reports through a promise, which can wait for the pool. | ||
| let global = (*this).js_global().filter(|global| { | ||
| (*this).pipe.get().has_done() && !global.bun_vm().is_shutting_down() | ||
| }); | ||
| if let Some(global) = global { | ||
| (*this).ref_(); | ||
| let opener = global.bun_vm().context_of(file.opener); | ||
| bun_jsc::Job::<CloseStreamFile>::schedule( | ||
| &global.js_thread(opener), | ||
| StreamFile(file.fd), | ||
| ClosingSink(this), | ||
| ); | ||
| return; | ||
| } |
There was a problem hiding this comment.
🟡 (optional) Every await Bun.write(path, stream) now settles only after a work-pool thread runs the close, so a busy pool delays it; on the base branch it settled at once. src/runtime/webcore/FileSink.rs:623-637 schedules Job::<CloseStreamFile> whenever pipe.has_done(), with no check that the file was replaced. A fresh file, a Bun.file(fd) dup or /dev/null needs no flush, yet still queues behind unrelated pool work. Fix: close in place when the sink created the file (or the destination is not a truncated regular file) and take the pool only for the O_TRUNC-over-existing case the description names, while still never reporting before the fd is closed.
Why this was flagged
A script calls Bun.write(newPath, response.body) or Bun.write(Bun.file(fd), stream) while the work pool is busy, e.g. two slow fs.promises.readFile calls on a container whose pool has get_thread_count() == 2 threads (src/bun_core/util.rs:2800-2847). The stream ends, FileSink::on_close runs, and src/runtime/webcore/FileSink.rs:623-637 takes the OwnedFile and schedules Job:: for any sink with pipe.has_done(), returning before report_closed. The promise is settled only in CloseStreamFile::then (FileSink.rs:138-141) after WorkPool::schedule (src/jsc/job.rs:381) hands the job to a free thread, so the await waits out the whole pool backlog. On the base branch the close was also queued on the pool but on_close settled the promise immediately. The PR description says a new file is closed in place and only a replaced file (O_EXCL -> EEXIST) takes the pool, but Options::flags() (FileSink.rs:252-260) has no O_EXCL probe, so the code does not implement that split; the test at test/js/bun/util/filesink.test.ts expects settledWithPoolParked: false even for "new file" rows.
Verification: src/runtime/webcore/FileSink.rs:931-935 sets self.file = Some(OwnedFile{..}) for every fd where !self.pollable.get(), with no O_EXCL/EEXIST distinction. FileSink.rs:622-637: in on_close, Job::<CloseStreamFile>::schedule(...); return;, so report_closed runs only in CloseStreamFile::then (line 139-142). On base, Closer::close(fd, ()) queued the close and immediately invoked on_close_fn, so the promise settled at once.
| #[cfg(unix)] | ||
| struct StreamFile(Fd); | ||
|
|
||
| #[cfg(unix)] | ||
| impl Drop for StreamFile { | ||
| fn drop(&mut self) { | ||
| let fd = core::mem::replace(&mut self.0, Fd::INVALID); | ||
| if fd != Fd::INVALID { | ||
| fd.close(); |
There was a problem hiding this comment.
🟡 nit (optional): maintainers get a second hand-rolled fd-owning type where bun_sys::File already closes on Drop. src/runtime/webcore/FileSink.rs:91-99 defines StreamFile(Fd) with its own Drop plus a mem::replace in run (FileSink.rs:127), duplicating what bun_sys::File (src/sys/file.rs:12-23) provides. Fix: make CloseStreamFile::OffThread a bun_sys::File (or wrap one) so the job and its release-unrun path both close through the one owning type, keeping the explicit close in run so the fd is released on the pool rather than at Drop.
Why this was flagged
src/CLAUDE.md asks runtime code to prefer bun_sys::File, which owns an fd and closes it on Drop, over hand-written ownership. The diff adds StreamFile at src/runtime/webcore/FileSink.rs:91-99 with a manual Drop that does the same thing, and run at FileSink.rs:127 repeats the mem::replace(.., Fd::INVALID).close() idiom. No user-visible failure; this is a convention and duplication nit only. On the base branch neither type exists, so nothing regresses.
Verification: nit. The diff adds struct StreamFile(Fd) at /home/claude/bun/src/runtime/webcore/FileSink.rs:91 with a hand-written impl Drop (lines 94-101), and CloseStreamFile::run at line 127 repeats the idiom. bun_sys::File (/home/claude/bun/src/sys/file.rs:11-23) is already exactly this. Nothing breaks at runtime either way; this is duplication of an existing owning type.
| // | ||
| // Skipped on Windows: the writer there still reports before uv_fs_close completes. | ||
| describe.skipIf(isWindows)("a file is closed when its FileSink reports done", () => { | ||
| async function rowsWithParkedPool(files: Record<string, string>, rows: string) { |
There was a problem hiding this comment.
🟡 nit (optional): Maintainers get a slower suite run than needed: the five new subprocess tests in this describe run one after another. test/CLAUDE.md asks that tests which spawn processes or write files use describe.concurrent/it.concurrent, and the existing describe.concurrent("start() on an open writer") block in the same file follows that. Fix: make the new block describe.concurrent (each test already spawns its own bun -e child with its own tempDir and UV_THREADPOOL_SIZE=2, so nothing is shared), or state in a comment why they must stay serial.
Why this was flagged
test/js/bun/util/filesink.test.ts:691 opens describe.skipIf(isWindows)(...) and the tests at :763, :826, :940, :972 and :1024 each spawn a Bun child via rowsWithParkedPool (:692-:761) or Bun.spawn (:1000). Each child parks its own thread pool and polls with 5000 ms deadlines (:731, :741), so the block can take several seconds serially. test/CLAUDE.md:22 says to make such tests concurrent with test.concurrent or describe.concurrent unless very difficult; the file's own describe.concurrent("start() on an open writer") block does so. No comment gives a reason for staying serial. On the base branch these tests do not exist; after merging the file's wall time grows by the sum of the five children rather than their maximum.
Verification: nit. Triggering condition: every run of test/js/bun/util/filesink.test.ts. The new block at test/js/bun/util/filesink.test.ts:690 is describe.skipIf(isWindows)(...), not describe.concurrent. Nothing is shared between the tests, so running them serially is a pure cost. test/CLAUDE.md:22: "When multiple tests in the same file spawn processes or write files, make them concurrent".
| let file = (!self.pollable.get()).then(|| OwnedFile { | ||
| fd, | ||
| opener: context.id(), | ||
| }); | ||
| self.writer.with_mut(|w| w.close_fd = file.is_none()); |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: pre-existing: sinks on pipes, FIFOs and sockets (including subprocess.stdin) still report "closed" while close(2) is queued on the work pool, so a child waiting for EOF stays blocked until a pool thread is free. The fix only takes ownership of the fd when !self.pollable at FileSink.rs:931; every pollable handle keeps close_fd = true and goes through pipes.rs:107, which hands the fd to Closer and then calls on_close_fn at pipes.rs:115 in the same frame. Fix: make every POSIX FileSink close before it reports, e.g. let the sink own pollable fds too (or make close_impl report after the pool close completes), which the PR description claims for "every POSIX handle" but the code does not do.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
A script spawns a child with stdin: "pipe", writes, and calls subprocess.stdin.end(); the stdin FileSink is on a pollable fd. FileSink::setup at src/runtime/webcore/FileSink.rs:931-935 only creates an OwnedFile when !self.pollable.get(), so for a pipe file is None and w.close_fd stays true. PosixStreamingWriter::close (src/io/PipeWriter.rs:1134) then calls close_impl with close_fd = true; on Linux close_async is true (src/io/pipes.rs:66), so the fd is queued on the pool via Closer::close at pipes.rs:108 and on_close_fn runs immediately at pipes.rs:115-118. FileSink::on_close finds no OwnedFile and reports closed at FileSink.rs:640 with the write end still open in this process, so a child reading stdin does not see EOF until a pool thread runs the close; with the pool busy the child blocks indefinitely. This is the same as the base branch; the PR fixes the regular-file sibling only, while its description says close_impl closes before it reports for every POSIX handle. The new test suite has no FIFO or pipe row, so nothing detects it.
Verification: pre-existing. Any POSIX FileSink on a pollable fd ends while the work pool is busy. At src/runtime/webcore/FileSink.rs:931-935 a pollable fd leaves close_fd = true and file = None; pipes.rs:107-108 hands the fd to Closer::close(fd, ()) and pipes.rs:115-118 invokes on_close_fn in the same frame. The base fails identically. The new filesink.test.ts rows are all regular files.
Problem
await writer.end()andawait Bun.write(path, stream)settle with the file still open. Running it givesETXTBSY(809 to 882 of 1,000). No user reported this.PollOrFd::close_impl(src/io/pipes.rs:107) queuesclose(2)on the work pool, then callson_closeat once (wip use wrapper for managing process #8456, 1.0.31).Fix
FileSinkowns the fd of its file and closes it inon_close, before it reports.close_implis unchanged.Bun.write(dest, stream)reports through a promise: the pool closes the file, then it settles.writer.end()andclose()report by returning: they close in place.test/js/bun/util/filesink.test.ts(17 of 18 new rows fail on main). Self-reviewed: 32 concerns raised, 30 addressed. Not taken: two test rows (a race, a pipe).Background
PollOrFdholds a sink's fd, or aFilePollfor a pipe. Pipes and readers still report early: that needs a ruling (bun lint,bun fmt: write a child's stdin from the event loop #45424, Remove libuv on Windows #42819).bun_jsc::Jobis a pool job whose completion runs on the JS thread.close_implfor every handle (the first revision), and a pool close only from 256 KiB.Downsides
writer.end()closes on the JS thread, as before 1.0.31: under 10 µs (p50) on ext4, 34 ms for an unlinked 256 MiB file, a flush on NFS or FUSE (not measured). Orend()returns a promise.await Bun.write(dest, stream)waits for the pool: 14 µs becomes 216 to 459 µs at 20 bytes (fs.promises.writeFile: 380 µs), 37 to 41 µs with 128 in flight.Notes
History of this PR. The first revision (41dbee2) closed the fd on the calling thread in
PollOrFd::close_impl, for every POSIX handle. A self-review found two things. Every measured failure belongs to one owner, aFileSinkon a file. And the same change for pipes, sockets and readers needs a ruling that nobody has given: #45424 and #42819 close in the owner and leaveclose_impl. So this revision closes the file in its owner.close_impl,FileReaderand nine other test files are as on main again. The pipe and reader half goes to a PR of its own.No user issue. Nobody filed this. An automated fuzz run found it. What is in the repository:
await writer.end()" under "On main too, not fixed here".filesink.test.tsand other tests wait for the pool close (test(spawn): wait for background pipe closes before counting fds #37207, test: fix the race in the filesink pending-write test #45103, test: wait for the fd to close in fileDescriptorLeakChecker() #43794).end()closes: "Close the file descriptor" (packages/bun-types/s3.d.ts:24), anddocs/runtime/file-io.mdxshowswriter.end()withoutawait"to flush the buffer and close the file".await Bun.write(path, response)goes through this sink while the body arrives.Repro. Each round writes a small shell script to a file, waits for bun to say that it is done, makes it executable and runs it.
ETXTBSYper 1,000 rounds, Linux x64, 12 cores, release builds. "main" is 9634520, "this PR" is 9092e8e. The rows of released versions are from the first revision of this PR.writerunawaitedresponseexistinglargecontroltaskset -c 2,3taskset -c 2,3Before #8456
FileSink.cleanup()closed the fd in place and then ran the pending promise.What "closed" means. The descriptor is closed in this process when
end()returns or the promise settles. A child that another thread forks at that moment holds a copy until itsexec. With 2 and 4 Workers that spawn children in a loop, thewriterrepro gotETXTBSYin 2 and 14 of 2,000 rounds on the first revision (the same in-place close as now), and in 5 and 15 of 2,000 withfs.writeFileSync. On main it was 1,744 of 2,000 with 2 Workers.The design.
FileSinkreports "closed" in three ways:end()returns,close()returns, or the promise ofBun.write(dest, stream)settles.FileSink::setuptakes the fd of every sink that is not polled (a file).PosixStreamingWriter.close_fdisfalsefor it, and the fd goes toFileSink.filewith the id of the script that opened it. The writer still callson_closewhen it lets go of its handle. It does not close the fd. The shell'sIOWriterowns its fd in the same way on main.FileSink::on_closecloses the file beforereport_closed. A sink with a pending promise (pipe_streammade one:Bun.write(dest, stream), for a path or an fd) gives the fd to abun_jsc::Jobon the work pool. The job's completion callsreport_closed, which settles the promise. Every other sink (writer()) closes in place.Bun.write()(OwnedFile.opener), and not under the script that runs when the stream ends. WithBun.ModuleGraphthese can differ. A job of a disposed graph comes back without its completion, so its promise stays pending, asdocs/runtime/module-graph.mdxsays for a graph. The fd is closed and the sink is released on that path too (Drop for StreamFile,Drop for ClosingSink).report_closedasserts in debug builds that the sink holds no file.bun_io::Closer, as on main. Nothing reports there.start()on an open writer: the old file is closed throughon_closewhen the new fd is a file, and insetupwhen the new fd is a pipe (main leaves that one open).pull()throws madeBun.write()return a rejected promise with the file still open: the sink was dropped withouton_close. That throw now goes throughhandle_reject_stream, so the promise rejects after the close.controller.end()inside a direct stream thatBun.write()pumps returns its number when the job is queued. The promise ofBun.write()is the report there.The question for a maintainer:
writer.end(). There are three answers.end()returns a number with the file closed, also when the caller does not await it, as the docs show. It is what 1.0.30 did. The cost is aclose(2)on the JS thread: the tables below, a flush on NFS or FUSE (not measured), and the close after a truncate if Bun.file(path).writer(): truncate the destination file #31737 makeswriter()truncate (168 µs at 20 bytes, 38 ms at 64 MiB on ext4).end()returns a promise for a file, and the same job settles it. The type allows it (number | Promise<number>). Code that uses the number breaks, and a caller that does not await has the file open again.Bun.write(dest, stream)only.For answer 2 or 3 the
file.fd.close()line inon_closechanges, and the rest stays.Weighed, and not taken.
close_implfor every POSIX handle (the first revision). It also fixes pipes, but it reverses the pool close for every owner. It goes to its own PR.Bun.write(dest, stream)too. The promise can wait for the pool, so nothing has to run on the JS thread there: not the ext4 flush at the close after the truncate (0.17 ms at 20 bytes, 11 ms at 16 MiB, seconds on a busy disk), and not a flush on NFS or FUSE.openatto know whether the file existed.BUN_FEATURE_FLAG_DISABLE_*switch that brings the pool close ofwriter()back. It would bring the early report back with it.The new tests. Each script runs with
UV_THREADPOOL_SIZE=2and parks each of the two pool threads inreadFile()of a FIFO of its own. A close that waits for the pool cannot run, so each row is the same in every run. (The first revision parked both threads on one FIFO. That hangs on macOS:fifo_readin XNU clears the end-of-file state after the first reader gets it, and the second reader stays blocked. CI build 129000 timed out there.) A row has the number of descriptors that are still open where the API reports done. Release builds of main and of this PR. This PR: 20 of 20 runs the same.writer.end()at return (a number), then run the fileETXTBSYawait writer.end(), then run the fileETXTBSYwriter.close()at return, then run the fileETXTBSYwriter.end()over a file that exists, then run the fileETXTBSYBun.file(fd).writer(): the sink's duplicate afterawait end()Bun.write(newPath, body), four rows:Response(stream), aReadableStreamwith an async pull, a directReadableStream,Bun.file(src).stream()Bun.write(existingPath, body), three rows:Response(stream), aReadableStream,Bun.file(src).stream()Bun.write(Bun.file(fd), Response(stream)): the sink's duplicateBun.write(path, stream)that rejects, two rows: the stream fails, andpull()of a direct stream throwsBun.ModuleGraphwrites and ends the stream, and is disposed with the close in the queueTwo mutations of this PR on a debug build, to check the rows:
js_thread_of_caller_no_frame()): the last two rows fail. The graph's promise is fulfilled and the host's stays pending.handle_reject_streamarm: thepull()row has the file open and rejects with the pool parked.Two more tests cover the job whose completion does not run.
BUN_DEBUG_TEST_WORKER_TEARDOWN_GATEholds the completion until the worker's teardown waits for it, and names the place of its ticket (FileSink.rs). On main no ticket is taken there, so this test fails on a debug build of main (not run here).One test is new in the
start() on an open writergroup:start({ path })moves a writer from a file onto a FIFO. Main leaves the file open for good (a restart onto a polled fd does not close the old handle). This PR closes it.The rest of
filesink.test.tsis as on main. Its waits for the fd leak checker stay: pipes still close on the pool.What the change costs, per call. Release builds of main 9634520 and of this PR (9092e8e, same base).
strace,perf,valgrindandbloatyare not installed in this container, and the kernel has no syscall tracepoints. A smallptracecounter that splits by thread counted the system calls. The numbers are the slope between 100 and 300 calls.futexcounts change with timing underptrace.writer().write().end():closeon the JS thread / on a pool threadfutexon the JS thread / on other threadsawait Bun.write(newPath, stream):closeon the JS thread / on a pool threadopenatwritefrom the pool,epoll_pwait2futexon the JS thread / on other threadsawait Bun.write(existingPath, stream):openatfutexon the JS thread / on other threadsClosertask for each closewriter(): none. Stream write: oneJoband one completion tasksizetextWall clock, idle 12-core VM, interleaved runs. p50 of 300, three runs, unless the row says otherwise.
writer().write(20 bytes),await end()await Bun.write(newPath, stream), 20 bytesawait Bun.write(existingPath, stream), 20 bytesawait fs.promises.writeFile(path, 20 bytes)(one run)On main the promise of a stream write settles before the close. With this PR it settles after it, and the JS thread is free in between. A stream write that is awaited alone now takes what
await fs.promises.writeFile()takes: one round trip to the pool. Writes that are in flight together share the wait.How long
close(2)takes on the calling thread. Linux 6.1, ext4 (delalloc,auto_da_alloc), idle machine, p50 and max. Measured for the first revision.O_TRUNC(writer())O_TRUNC(Bun.write(path, stream): on the pool)Closes that can be slow and now run on the JS thread. Only
writer.end()andwriter.close().writer.end()after an unlink took 15 ms at 64 MiB and 34 ms at 256 MiB (median of 5, max 132 ms) on the first revision, which has the same in-place close. On main it is 0.3 ms: the pool does it.writer()truncate (it fixesFileSinkdoes not truncate existing file #25968). The close after a truncate is the third row of the table above.Not changed.
BaseWindowsPipeWriter::closestartsuv_fs_closeand reports at once. Windows: close a FileSink's file before end() and Bun.write(path, stream) report it closed #46344 (draft) is the sibling. Windows was not run here.close_implis as on main, soproc.stdin.end(),child.stdin'close'and the end ofBun.file(path).stream()still report before the pool closes the fd. Measured in the self-review: with twocrypto.pbkdf2jobs on a pool of two threads, acatchild exits 710 to 999 ms afterchild.stdin.end()on 1.4.2, and 1 to 2 ms on Node 26.3.writer()that is dropped withoutend()goes to the pool, as on main.Bun.servefile responses close throughCloser. They report no close to JS.Related.
close()never stalls the loop", and io: leave the fd with the caller when a POSIX pipe writer fails to start #38354 repeats it. Both describe pipes, which this PR does not touch. The close ofwriter.end()is the one close that this PR moves to the JS thread.bun lint,bun fmt: write a child's stdin from the event loop #45424 (StaticPipeWriter::leave_fd_open) and Remove libuv on Windows #42819 close in the owner and leaveclose_impl. This PR does the same forFileSink. After it, a rewrite ofclose_impl(Remove libuv on Windows #42819) cannot bring the early report back for a file: the writer never closes one.PosixStreamingWriter.close_fdfield with another meaning. One of the two PRs renames it at rebase.FileSinkpanic whenstart({ path })moves a writer from a TTY to a regular file #46283 (merged) changed the same block ofFileSink::setup. This branch has it.Self-review. Thirty-two concerns were raised on an earlier revision. Thirty are addressed:
close_implis not changed, and the pipe half has its own PR.truncateor a size. It is filed under the script that calledBun.write().writer.close(), a native source, a direct stream, two rejecting streams,writer.end()over a file that exists, and threeBun.ModuleGraphrows.bun fmtline).Two are not taken:
Bun.Terminalrow. A terminal is a pty, which is polled, so it belongs to the pipe half.Also run. Debug build of this PR:
test/js/bun/util/filesink.test.ts: 88 pass, 1 skip.test/js/bun/io/bun-write.test.js: 107 pass.test/js/bun/spawn/spawn-stdin-readable-stream.test.tsandspawn-stdin-readable-stream-edge-cases.test.ts: 70 pass.test/js/web/streams/streams.test.js: 711 pass.test/js/bun/module-graph/module-graph-isolation.test.ts: 409 of 414 in a whole-file run. The 4 failures are deadlines of rows that use noFileSink, and each passes in a narrower run.cargo check --workspacepasses forx86_64-pc-windows-msvc,aarch64-apple-darwinandx86_64-unknown-freebsd. macOS was not run here. CI build 129000 ran the first revision on both macOS lanes: its one failure was the test parking that this revision changes.no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/util/filesink.test.ts