Repository navigation
console: output goes behind what process.stdout/stderr have buffered, as in Node - #37128
dylan-conway wants to merge 33 commits into
Conversation
… through the stream like Node console.* and process.stdout / process.stderr now share a single per-thread FileSink per fd, and the global console delivers through the stream's write() whenever user code could observe it (patched write, console._stdout = x, corked/ended/backed-up stream) — Node's kWriteToConsole — while staying fully native otherwise. - FileSink stdio mode (create_stdio / stdio_sink_for / write_all_sync / drain_sync); Bun.stdout.writer(), Bun.file(1|2).writer(), console.write and Bun.write(Bun.stdout) are handles to the same sink; poll registered only while backed up; FIFO goes non-blocking only once a JS writer exists. - Console formats each message into a scratch buffer and delivers it once (spilling every 64 KiB on the native path); timeLog/timeEnd move to stdout with Node's format and warnings, trace to stderr, assert gets the "Assertion failed" prefix inline, clear() honours _stdout.isTTY. - process.stdout/stderr are honest Writables (no own write override): writableLength / needDrain / cork / 'drain' / EPIPE 'error' behave; chunks still buffered at exit are flushed; queued output is drained before exit, before fatal-error printing and before anything Bun prints to fd 1/2. - console._stdout/_stderr are lazy get/set accessors; process._rawDebug; diagnostics_channel console.* channels; _ignoreErrors. - EAGAIN never drops bytes: Output writer, non-pollable FileSink writes and fd copy loops poll and retry; O_NONBLOCK on stdio is snapshotted/restored at exit and on SIGINT/SIGTERM (never over an inherited SIG_IGN) and cleared on fds handed to children; open_for_writing no longer sets it on non-pollable fds. - worker_threads rebinds the native console to the port streams instead of replacing globalThis.console. Fixes #36419, fixes #21516, fixes #19952, fixes #12031, fixes #8036.
main gained its own process._rawDebug (#31831); keep that one and drop the native implementation and its duplicate test from this branch.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesShared stdio sinks now coordinate stdout and stderr writes across console methods, Shared stdio and console output
Possibly related PRs
Suggested reviewers: Pre-merge checks:
|
There was a problem hiding this comment.
Actionable comments posted: 21
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/runtime/node/node_fs.rs (1)
4892-4972: 🎯 Functional Correctness | 🔴 Critical | ⚡ Quick winFix the unbounded copy loop to use retrying read/write.
The bounded loop (lines 4892-4930) now retries
EINTR/EAGAINthroughSyscall::read_retryingandSyscall::write_retrying. The unbounded tail loop (lines 4931-4968), used when the source size is unknown (stat_size == 0), still calls plainSyscall::read/Syscall::write.
stat_size == 0skips the bounded loop entirely and falls straight into the unbounded loop.read_write_fallbackinsrc/runtime/webcore/blob/copy_file.rscalls this function withstat_size = 0for the "unknown size" case, which covers FIFO/pipe copies such asbun run foo.js | bun run bar.js. A pipe copy that hitsEAGAINorEINTRin this loop returns a hard error instead of retrying, which reproduces the exact backpressure/truncation failure this PR targets.Use the retrying calls in the unbounded loop too.
🐛 Proposed fix for the unbounded copy loop
if !broke { 'outer: loop { - let amt = match Syscall::read(src_fd, buf) { + let amt = match Syscall::read_retrying(src_fd, buf) { Ok(result) => result, Err(err) => { return Err(if !src.is_empty() { err.with_path(src) } else { err }); } }; // we don't know the size // so we just go forever until we get an EOF if amt == 0 { break; } *wrote += amt as u64; let mut slice = &buf[..amt]; while !slice.is_empty() { - let written = match Syscall::write(dest_fd, slice) { + let written = match Syscall::write_retrying(dest_fd, slice) { Ok(result) => result, Err(err) => { return Err(if !dest.is_empty() { err.with_path(dest) } else { err }); } }; slice = &slice[written..]; if written == 0 { break 'outer; } } } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/runtime/node/node_fs.rs` around lines 4892 - 4972, Update the unbounded tail loop in the copy function, identified by the `if !broke` branch, to use `Syscall::read_retrying` and `Syscall::write_retrying` instead of the plain read/write calls. Preserve the existing EOF, zero-write, byte-counting, and path-aware error behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/io/stdio_lock.rs`:
- Around line 43-60: Make StdioLock explicitly !Send by adding a thread-affine
marker field, such as PhantomData tied to a non-Send type, and update acquire to
initialize it while preserving the existing Option<usize> slot behavior and Drop
logic.
In `@src/js/builtins/ConsoleObject.ts`:
- Around line 132-140: Update the observed-stream branch in Console.write to
route every chunk through writeToObservedStream, preserving the required chunk
representation while retaining the existing byte-count calculation. Ensure
synchronous write failures and emitted stream errors are swallowed using the
same temporary error-listener guard as writeToObservedStream, extending that
helper only if necessary to forward binary chunks unchanged.
In `@src/js/builtins/ProcessObjectInternals.ts`:
- Around line 93-97: Add a comment immediately above the emitClose check in the
_undestroy() flow explaining that _undestroy() resets the state before normal
close handling, so the next-tick "close" emission remains observable to finished
and pipeline when emitClose is false. Preserve the existing conditional and
event-emission behavior.
In `@src/js/internal/fs/streams.ts`:
- Around line 605-634: Update the FileSink fast-path write handling in
underscoreWriteFast and its corresponding _writev path to increment
this.bytesWritten by the number of bytes successfully written, including after
asynchronous completion. Preserve the counter for fast-path streams such as
process.stdout, process.stderr, and tty.WriteStream, and do not modify it when a
write fails.
In `@src/js/internal/streams/writable.ts`:
- Line 413: Remove the takeBuffered assignment from the default Writable export
in internal/streams/writable, preserving Writable as the constructor object.
Expose takeBuffered through a separate internal export path, then update
ProcessObjectInternals.ts to import it from that path while leaving all
node:stream and default Writable consumers unchanged.
In `@src/js/node/worker_threads.ts`:
- Around line 454-462: Update the stdout and stderr setup in the worker stream
initialization to pass the local writable values directly to setConsoleStream
after creating them, rather than reading process.stdout or process.stderr back.
Keep the existing property assignments and slot mappings unchanged.
In `@src/jsc/bindings/BunProcess.h`:
- Around line 78-81: Update the comment above Process::consoleStream() to remove
the claim that it never runs user code and accurately state that resolving the
console stream may invoke a JavaScript accessor and throw. Keep the declaration
and consoleStreamIsResolved() behavior unchanged.
In `@src/jsc/ConsoleObject.rs`:
- Around line 329-344: Verify whether bun_io::StdioLock::acquire is reentrant
for the same thread. If it is not, update emit so the lock is not held while
f(&mut writer) executes: release it before formatting and reacquire it around
deliver_to; otherwise preserve the current behavior only when recursive
acquisition is explicitly supported.
- Around line 306-323: Update the doc comment immediately above the
message-formatting function to state that formatting failures stop further
writes but may leave previously spilled partial output, including a truncated
message without a terminating newline, while the exception still propagates.
Keep the existing bounded-memory spilling behavior unchanged.
- Around line 6216-6226: Update the error handling around the `result` from
`format2` in `_rawDebug` so failed formatting does not write the partially
populated `buf` to stderr; only perform the `write_all_retrying` call and
`ConsoleObject::put_scratch` on successful formatting, matching `emit`’s discard
behavior while preserving the existing exception propagation.
In `@src/jsc/VirtualMachine.rs`:
- Around line 1471-1480: Avoid overlapping mutable borrows at both FFI call
sites: in src/jsc/VirtualMachine.rs lines 1471-1480 within drain_stdio, capture
a raw pointer with core::ptr::from_mut::<VirtualMachine>(self) before the guard
body and pass it to __bun_stdio_sink_drain; in src/jsc/VirtualMachine.rs lines
4656-4663 within swap_global_for_test_isolation, capture and pass the equivalent
raw pointer to __bun_stdio_sink_release_js.
- Around line 1512-1516: Update global_exit() to drain both stdout and stderr
sinks before calling release_js_handles(), ensuring queued output from direct
exits is flushed. Do not rely on on_exit() or a single fd-specific drain; invoke
the sink-draining behavior for each stream before releasing JS handles.
In `@src/runtime/webcore/Blob.rs`:
- Around line 1802-1811: Update the comment above the shared stdio wrapper
branch in the Blob writer creation flow to explicitly document that writer
options are intentionally ignored for file descriptors 1 and 2 because the
shared sink cannot support per-handle options. Keep the existing return behavior
unchanged.
- Around line 5093-5104: Update the synchronous settlement branch in the
`to_result` flow to release the pending keep-alive reference after
`webcore::FileSink::drain_sync(sink)` completes, including the successful drain
path while preserving existing error propagation. Ensure the cleanup occurs for
settled pending writes so `release_stdio_sinks` can reclaim the sink and
duplicated descriptor.
In `@src/runtime/webcore/FileSink.rs`:
- Around line 731-737: Update the start_lazy error path in the writer.with_mut
closure to clear the writer’s stored fd ownership before calling fd.close().
Preserve the existing invalidation, dereference, and error return flow so sink
teardown cannot close the descriptor again.
- Around line 893-910: Update the write loop around sys::write_retrying in
FileSink’s sink write path so Ok(0) with a non-empty bytes slice is treated as
an explicit failure, not a successful break. Propagate the resulting error
through stdio_latch_error and writer.fail, matching the existing Err(e) path, so
__bun_stdio_sink_write reports the stalled write instead of silent truncation.
- Around line 2121-2144: Update write_now to use the same nullish-argument error
behavior as FileSink.prototype.write: ensure null and undefined are reported
with STREAM_NULL_VALUES, while preserving INVALID_ARG_TYPE for other unsupported
values. Reuse the existing validation or error-handling path in write_js_value
or the corresponding FileSink write implementation rather than introducing
divergent logic.
In `@src/spawn_sys/posix_spawn.rs`:
- Around line 618-631: Update the Dup2 handling in the spawn action loop so
clearing O_NONBLOCK affects only the child’s inherited descriptor, not the
parent stdio descriptor and its cached poll state. Preserve non-blocking
behavior for parent PipeWriter stdout/stderr paths, using the existing
descriptor-duplication or metadata-synchronization mechanism where appropriate.
In `@test/js/node/process/process-stdio.test.ts`:
- Around line 213-265: Update runWithSlowStdout to create the spawned process
with await using, ensuring its async disposal terminates the child on every exit
path, including errors and assertion failures. Keep the existing stdout polling,
stderr collection, exit-code handling, and descriptor cleanup unchanged.
- Around line 172-202: Guard the POSIX-only fixture initialization in the “stdio
sink” suite before calling tempDirWithFiles or cc(). Ensure non-POSIX
environments return before compiling fdutil.c, while preserving the existing
isPosix skip behavior for POSIX tests.
In `@test/js/node/worker_threads/worker_threads.test.ts`:
- Around line 1779-1788: Update the worker test around the worker exit await to
also await the `end` events for both `worker.stdout` and `worker.stderr`,
together with `exit`, before asserting `{ out, err }`. Preserve the existing
data handlers and expected output while ensuring all stream chunks are received
deterministically.
---
Outside diff comments:
In `@src/runtime/node/node_fs.rs`:
- Around line 4892-4972: Update the unbounded tail loop in the copy function,
identified by the `if !broke` branch, to use `Syscall::read_retrying` and
`Syscall::write_retrying` instead of the plain read/write calls. Preserve the
existing EOF, zero-write, byte-counting, and path-aware error behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 65425516-cd4f-40f6-b9a9-2a5306674edc
📒 Files selected for processing (42)
docs/guides/write-file/stdout.mdxsrc/bun_core/output.rssrc/codegen/generate-jssink.tssrc/io/PipeWriter.rssrc/io/lib.rssrc/io/openForWriting.rssrc/io/stdio_lock.rssrc/js/builtins/BunBuiltinNames.hsrc/js/builtins/ConsoleObject.tssrc/js/builtins/ProcessObjectInternals.tssrc/js/internal/fs/streams.tssrc/js/internal/streams/writable.tssrc/js/node/diagnostics_channel.tssrc/js/node/worker_threads.tssrc/jsc/ConsoleObject.rssrc/jsc/VirtualMachine.rssrc/jsc/bindings/BunProcess.cppsrc/jsc/bindings/BunProcess.hsrc/jsc/bindings/ConsoleObject.cppsrc/jsc/bindings/ZigGlobalObject.cppsrc/jsc/bindings/c-bindings.cppsrc/jsc/rare_data.rssrc/runtime/cli/test_command.rssrc/runtime/jsc_hooks.rssrc/runtime/node/node_fs.rssrc/runtime/webcore/Blob.rssrc/runtime/webcore/FileSink.rssrc/runtime/webcore/blob/copy_file.rssrc/spawn_sys/posix_spawn.rssrc/sys/lib.rssrc/sys/sys_uv.rstest/js/node/console/console.test.tstest/js/node/process/process-stdio.test.tstest/js/node/test/parallel/test-console-clear.jstest/js/node/test/parallel/test-console-count.jstest/js/node/test/parallel/test-console-stdio-setters.jstest/js/node/test/parallel/test-process-raw-debug.jstest/js/node/worker_threads/worker_threads.test.tstest/js/web/console/console-timeLog.expected.txttest/js/web/console/console-timeLog.test.tstest/js/web/workers/structured-clone.test.tstest/js/web/workers/structuredClone-classes.test.ts
- StdioLock is !Send; consoleStream() header comment reflects that first resolution may run a user getter; emit() doc notes spilled messages can be torn by a formatting exception - FileSink: Ok(0) from write(2) is an error, not a silent stop; start_lazy failure leaves fd ownership with the writer; stdio deinit drops the backpressure keep-alive ref; writeNow reports STREAM_NULL_VALUES like write(); a spawn that put fd 1/2 back into blocking mode is noticed (STDIO_MADE_BLOCKING) so the sink stops assuming EAGAIN - drain stdio at the top of global_exit(); pass raw VM pointers to the sink externs - Console.write on an observed stream uses the console's ignore-errors policy; fast-path streams maintain bytesWritten; takeBuffered is a private name, not a public Writable static; worker_threads binds the local port stream - tests: guard POSIX-only fixture setup, dispose the child in runWithSlowStdout, await worker stdout/stderr end
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
src/js/internal/fs/streams.ts (1)
615-618: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick winResolve the file path before creating the
FileSink.
Bun.file(this.path).writer()can receive a relative path. Resolve and store an absolute path before this file operation.As per coding guidelines, use absolute paths in file operations.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/js/internal/fs/streams.ts` around lines 615 - 618, Update the file-sink initialization in the write-stream path around kWriteStreamFastPath to resolve this.path to an absolute path and store the resolved value before calling Bun.file(...).writer(); use that absolute path for the FileSink creation while preserving the existing fd assignment.Source: Coding guidelines
src/jsc/bindings/BunProcess.h (1)
55-63: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAdd GC-stress coverage for persistent console handles.
Process::visitChildrenImplalready tracesm_consoleStream,m_stdioStream, andm_consolePublish. Existing tests cover rebinding and diagnostics publishing, but do not force GC during these operations. Add a focused GC test for these paths.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/jsc/bindings/BunProcess.h` around lines 55 - 63, Add focused GC-stress coverage for the persistent console handles used by Process::visitChildrenImpl, exercising m_consoleStream, m_stdioStream, and m_consolePublish during console rebinding and diagnostics publishing. Force garbage collection while performing these operations, then verify the handles remain valid and behavior is preserved.Source: Coding guidelines
src/runtime/webcore/FileSink.rs (1)
908-925: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUse poll-before-write for direct console output after stdio becomes blocking.
After
spawn_zchanges the shared descriptor mode, this loop callssys::write_retryingbefore any readiness wait. If the pipe is full, that first write can block the JS thread instead of returningEAGAIN. Apply the blocking-pipe strategy before each direct write in this mode, and add a slow-pipe regression test that writes throughconsole.*after an inherited-stdio spawn.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/runtime/webcore/FileSink.rs` around lines 908 - 925, Update the non-Windows direct-output loop in the relevant FileSink write method to poll for write readiness before every sys::write_retrying call after the descriptor becomes blocking, preserving existing error and zero-progress handling. Add a regression test covering console.* output through an inherited-stdio spawn with a slow/full pipe.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/js/builtins/ConsoleObject.ts`:
- Around line 137-150: Update the emitter setup around isEmitter to also
validate that observed.removeListener is callable, and track whether the noop
error listener was actually added. In the finally block, call removeListener
only when that tracked state is true, preventing custom streams lacking
removeListener from masking the original write failure.
In `@src/sys/lib.rs`:
- Around line 7452-7456: The stdio blocking setup currently discards errors,
allowing spawn to continue after failure. Update make_stdio_blocking in
src/sys/lib.rs:7452-7456 to return sys::Result<()> and propagate
update_nonblocking failures while marking success only after completion; update
the spawn path in src/spawn_sys/posix_spawn.rs:624-630 to propagate that result
before creating the child.
---
Outside diff comments:
In `@src/js/internal/fs/streams.ts`:
- Around line 615-618: Update the file-sink initialization in the write-stream
path around kWriteStreamFastPath to resolve this.path to an absolute path and
store the resolved value before calling Bun.file(...).writer(); use that
absolute path for the FileSink creation while preserving the existing fd
assignment.
In `@src/jsc/bindings/BunProcess.h`:
- Around line 55-63: Add focused GC-stress coverage for the persistent console
handles used by Process::visitChildrenImpl, exercising m_consoleStream,
m_stdioStream, and m_consolePublish during console rebinding and diagnostics
publishing. Force garbage collection while performing these operations, then
verify the handles remain valid and behavior is preserved.
In `@src/runtime/webcore/FileSink.rs`:
- Around line 908-925: Update the non-Windows direct-output loop in the relevant
FileSink write method to poll for write readiness before every
sys::write_retrying call after the descriptor becomes blocking, preserving
existing error and zero-progress handling. Add a regression test covering
console.* output through an inherited-stdio spawn with a slow/full pipe.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 0dd86ad1-06e0-47c7-ad4c-68ce238d6ee5
📒 Files selected for processing (17)
src/io/lib.rssrc/io/stdio_lock.rssrc/js/builtins/BunBuiltinNames.hsrc/js/builtins/ConsoleObject.tssrc/js/builtins/ProcessObjectInternals.tssrc/js/internal/fs/streams.tssrc/js/internal/streams/writable.tssrc/js/node/worker_threads.tssrc/jsc/ConsoleObject.rssrc/jsc/VirtualMachine.rssrc/jsc/bindings/BunProcess.hsrc/runtime/webcore/Blob.rssrc/runtime/webcore/FileSink.rssrc/spawn_sys/posix_spawn.rssrc/sys/lib.rstest/js/node/process/process-stdio.test.tstest/js/node/worker_threads/worker_threads.test.ts
- Bun__Process__consoleStream / consoleStreamObject use a top-level scope (the Rust caller learns of a throw via the out-param); consoleStreamForGetter keeps one function-wide ThrowScope with RELEASE_AND_RETURN - VirtualMachine::destroy() releases the stdio sinks itself, for VMs torn down without release_js_handles() (bake) - node:fs copyFile fallback: the unknown-size loop retries EAGAIN/EINTR too - tests: stdio-sink suite reads through a mkfifo instead of bun:ffi in the runner and skips the cc()-using cases under ASAN; strip ANSI where the runner's stdout may be a colour TTY; timeLog warning test uses --no-warnings; write-after-end (piped) expectation matches node (dummyDestroy -> _undestroy leaves the stream writable)
…O_MADE_BLOCKING; Console.write only removes the error guard it added
|
Re: the outside-diff note on 🤖 Addressed by Claude Code |
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🔴
src/runtime/webcore/FileSink.rs:1421-1425— ForWrote(n) | Done(n),write_withsetsaccepted = ndirectly, but when the writer already holds coalesced bytes (e.g.Bun.stdout.writer().write("hello")earlier in the same tick on a pollable stdio),write_impl(may_buffer=false)appends the new chunk tooutgoingandtry_write_newly_buffered_data()flushes the whole buffer — sonincludes the prior bytes andBun.write(Bun.stdout, "abc")resolves with 8 instead of 3. ComputeacceptedforWrote/Donethe same way asPending(e.g.(n as u64).saturating_sub(buffered_before as u64), or reusebytes_accepted's(buffered_after + written) - buffered_beforefor every variant); the new test only exercises thePendingbranch via a 256K backpressure fill so it doesn't catch this.Extended reasoning...
What the bug is
FileSink::write_with(introduced in this PR) returns(streams::Writable, u64)where the second element —accepted— is meant to be the number of encoded bytes this call handed the writer. ForWriteResult::Pendingit correctly computes that viabytes_accepted()((buffered_after + written) - buffered_before), but forWrote(n) | Done(n)it just usesndirectly:let accepted = match rc { WriteResult::Pending(_) => self.bytes_accepted(buffered_before, &rc), WriteResult::Wrote(n) | WriteResult::Done(n) => n as u64, WriteResult::Err(_) => 0, };
The
.1value is whatwrite_file_internal(Blob.rs) resolves theBun.write(Bun.stdout | Bun.stderr, data)promise with — the PR's stated "resolves with its own byte count" contract. Whennincludes bytes that were already sitting in the writer's coalesce buffer before this call, the resolved value is over-counted.The specific code path
On a stdio pipe/socket,
create_stdiosetsforce_sync = falseandpoll_flushes_buffer = false. So:-
Bun.stdout.writer().write("hello")→FileSink::write→write_with(|w| w.write(...))→write_impl(buf, may_buffer=true).should_buffer(5)is true (small write, PipeWriter.rs:735), so the 5 bytes are appended tooutgoingandWrote(5)is returned. Becausepoll_flushes_buffer=false, no poll is registered; the bytes stay inoutgoinguntil the end-of-tickAutoFlusher. -
In the same tick,
Bun.write(Bun.stdout, "abc")→write_file_internal→write_js_value(global, data, /*now=*/true)→write_with(|w| w.write_latin1_now(b"abc")).buffered_before = 5.write_latin1_impl(may_buffer=false)sees all-ASCII →write_impl(b"abc", false). -
In
write_impl(PipeWriter.rs:965),may_buffer=falseskips the coalesce branch, andself.outgoing.size() == 5 > 0, so it appends"abc"(outgoing now 8 bytes) and callstry_write_newly_buffered_data(). That doestry_write(force_sync=false, outgoing.slice())— writing all 8 bytes to a fast fd — and at line 903-906 resetsoutgoingand returnsWrote(8). -
Back in
write_with,rc = Wrote(8)→accepted = 8.to_resultreturnsOwned(8)(via theWrote→Temporary/Ownedmapping into_result). -
In
write_file_internal,resultis notPending(so nouncredit_pending),drain_syncis a no-op (buffer already empty), and the promise resolves withaccepted = 8— not 3.
The same over-count applies to the
write_utf16_now/ non-ASCIIwrite_latin1_nowpaths viamaybe_write_newly_buffered_data(buf_len, false)→try_write_newly_buffered_data(), which returnsWrote(before_len + buf_len)on a full drain.Why existing code doesn't prevent it
bytes_accepted()handles exactly this case ((buffered_after + written).saturating_sub(buffered_before)), but it is only called forPending— its body even early-returns 0 for anything else.uncredit_pendinginwrite_file_internalonly runs for thePendingvariant.- The new test at process-stdio.test.ts ("Bun.write(Bun.stdout, x) resolves with its own byte count …") first writes 256K via
process.stdout.write, which backs the pipe up so theBun.writecall takes thePendingbranch. The fully-drainedWrotepath with prior coalesced bytes is not covered. - Other consumers of
write_with(FileSink.write/write_latin1/write_utf16and thewriteNowhostfn) discard.1; only the newBun.write(Bun.stdout|stderr, ..)routing (also introduced in this PR) reads it.
Impact
User-visible incorrect return value from a documented API. The bytes on the fd are correct and in order; only the number the promise resolves with is wrong. Anyone relying on the resolved value (e.g. summing bytes written, progress reporting, or asserting
n === chunk.length) will see a value larger than the chunk they passed whenever they mixBun.stdout.writer().write()(coalesced) withBun.write(Bun.stdout, ..)in the same tick on piped stdio. This directly contradicts the PR's stated contract in the description table ("resolves with its own byte count").Fix
Compute
acceptedforWrote/Donewith the same delta thePendingbranch uses. The simplest change is:let buffered_before = self.writer.get().buffered_len(); let rc = self.writer.with_mut(f); let accepted = match rc { WriteResult::Wrote(n) | WriteResult::Done(n) | WriteResult::Pending(n) => { let buffered_after = self.writer.get().buffered_len(); (buffered_after + n).saturating_sub(buffered_before) as u64 } WriteResult::Err(_) => 0, };
(equivalently, generalize
bytes_acceptedto take thewrittencount from any variant and call it for all three). -
-
🔴
src/jsc/bindings/BunProcess.cpp:1885-1887—bun_restore_stdio()runs before the execve attempt, but when execve fails and control returns to JS the failure path only restoresFD_CLOEXEC(viasavedStdioFlags/F_SETFD) — it never undoes the termios andO_NONBLOCKreset. So a program that hadprocess.stdin.setRawMode(true)and then callsprocess.execve('/nonexistent')catches the throw with its terminal already put back into cooked mode (whilestream.isRawis still true), and a stdio FIFO'sO_NONBLOCKbit is cleared while the sink's internalnonblockingflag stays set. The failure path should snapshot/restoreF_GETFLand termios aroundbun_restore_stdio(), or use a variant that only touchesO_NONBLOCKhere.Extended reasoning...
What the bug is
This PR adds a
bun_restore_stdio()call at BunProcess.cpp:1887 insideProcess_functionExecve, before theexecve(2)/posix_spawnattempt.bun_restore_stdio()does two things (c-bindings.cpp): it callsbun_restore_stdio_nonblock()to reset each stdio fd'sO_NONBLOCKbit to the startup snapshot, and ittcsetattr()s each TTY stdio fd back to the termios captured atbun_initialize_process. That's exactly right when execve succeeds — the new image should inherit stdio in the state we found it.But execve can fail (ENOENT, EACCES, …), and this function is explicitly designed for that: the failure path at lines ~1957–1972 restores the FD_CLOEXEC flags it cleared (
fcntl(fd, F_SETFD, savedStdioFlags[fd])), restores the signal mask, and throws back to JS with the comment "the original image is still running… throw back to JS so the caller can handle it and recover". The failure path does not undo whatbun_restore_stdio()did — it never touchesF_GETFLor termios.Code path that triggers it
- A TUI/shell-like app calls
process.stdin.setRawMode(true). This puts the TTY into raw mode viatcsetattrand setsbun_stdio_modified[0] = 1(sobun_restore_stdio()will act on fd 0 even in the pipeline-producer case). - The app calls
process.execve('/nonexistent', …)(e.g. "run external command" with a bad path). Process_functionExecvereaches line 1887 and callsbun_restore_stdio(), whichtcsetattrs fd 0 back to the cooked termios captured at process startup.execve(2)fails with ENOENT.- The failure path restores
FD_CLOEXECand the signal mask, then throws. - The JS caller catches the error and continues — but the terminal is now cooked while
process.stdin.isRawis stilltrue. Keystrokes echo, arrow keys send escape sequences the app never sees, etc.
The
O_NONBLOCKhalf is analogous: if the stdio sink had setO_NONBLOCKon a FIFO stdout (viastdio_go_nonblocking(), which this PR added),bun_restore_stdio_nonblock()clears it on the shared open file description (the sink's dup'd fd shares that description), whileFileSink::nonblockingandFilePollFlag::Nonblockingremain set — sostdio_go_nonblocking()will never re-arm it (it early-returns onself.nonblocking.get()), and subsequentprocess.stdout.write()on a full pipe blocks the JS thread instead of returningfalse/queueing.Why existing code doesn't prevent it
The pre-existing
savedStdioFlagsrestore usesF_GETFD/F_SETFD, which is the file-descriptor flag word (onlyFD_CLOEXEC). Termios andO_NONBLOCK(which isF_GETFL/F_SETFL, the file-status flag word on the shared description) are a different namespace and are not touched by that restore. Before this PR nothing on the execve path modified them, so there was nothing to undo; now there is.Impact
A concrete regression on a recovery path the code itself documents as recoverable. The trigger is narrow (raw-mode TTY or non-blocking FIFO stdio +
process.execvefailure + caller catches and continues), but it's exactly the shape of a shell/TUI app offering "run external command" — the class of program most likely to use both raw mode andprocess.execve. No crash or data loss; the failure mode is a silently mis-configured terminal (very confusing to debug) and, for the O_NONBLOCK case, an event loop that blocks on stdout backpressure where before it queued.Fix
Snapshot
fcntl(fd, F_GETFL)for fds 0–2 and (for TTY fds)tcgetattrbefore callingbun_restore_stdio(), and restore both on the failure path alongside the existingF_SETFDrestore. Alternatively, split out abun_restore_stdio_for_execve()that only callsbun_restore_stdio_nonblock()and skips termios (execve doesn't need termios reset — the new image inherits the TTY device state regardless, and if it wants cooked mode it can set it itself; Node'sResetStdio()on execve only handles the flag bits). Either approach keeps the failure path's existing "caller can handle it and recover" contract intact. - A TUI/shell-like app calls
…total A write that also pushes out bytes an earlier Bun.stdout.writer().write() had coalesced returned their combined length, so Bun.write(Bun.stdout, s) could over-report. Callers that need the count (Bun.write) now supply the input's UTF-8 length; the pending credit used by to_result is unchanged. Also puts write_file_internal's doc comment back on write_file_internal.
|
Re: the 🤖 Addressed by Claude Code |
|
Updated 12:51 PM PT - Oct 14th, 2026
✅ @dylan-conway, your commit 7f91520ada410d931cc5cc86d3f24648d62360b6 passed in 🧪 To try this PR locally: bunx bun-pr 37128That installs a local version of the PR into your bun-37128 --bun |
…sed behind our back at teardown - Named FIFOs don't get FMODE_NOWAIT, so pwritev2(RWF_NOWAIT) on them is EOPNOTSUPP (and flips RWFFlagSupport off process-wide); a FIFO stdout on Linux >= 6.4 therefore fell back to blocking writes. Probe fstatfs for PIPEFS_MAGIC and use O_NONBLOCK for anything else. - If user code closed our dup'd stdio fd by number (or something reused and closed that number), teardown detaches instead of close()ing it again.
…a pre-execve stdio restore; primordial indexOf - on_auto_flush's error arm was the one terminal-error site not routed through the stdio latch, so an EPIPE hit while autoflushing a coalesced Bun.stdout.writer().write() closed the shared sink silently - bun_restore_stdio_nonblock records the fds it puts back to blocking (Bun__stdioMadeBlocking), so a process that survives a failed execve stops expecting EAGAIN on them - diagnostics_channel: ArrayPrototypeIndexOf.$call like the rest of the file
There was a problem hiding this comment.
No further findings after 7bd8448 — the autoflush stdio-latch, execve O_NONBLOCK bookkeeping, and primordial indexOf fixes all landed as described. Deferring to human review given the scope: this rewires console/stdio delivery end-to-end across FileSink lifetime/refcounting, JSC Process state, VM teardown ordering, signal-time fd-flag restoration, and per-platform write strategies, with documented user-visible behaviour changes.
What was reviewed:
- Every terminal-error site on stdio sinks now routes through
stdio_latch_error(incl.on_auto_flush,drain_sync,write_all_sync,to_result,flush_from_js,on_error) — no path leaves the sinkdonewithstdio_error=None. - The
ConsoleWriterspill path holdsStdioLockfor the whole message and reports errors once viareport_stdio_error; the ruled-out "N+1 'error' events" candidate doesn't hold because only the finaldeliver_toreturn propagates. bun_restore_stdio_nonblocknow setsSTDIO_MADE_BLOCKINGper fd flipped, so a failedexecveleaves sinks in a staterefresh_stdio_modewill downgrade.RareDatasink ownership: +1 stored bystdio_sink_for, released inrelease_stdio_sinksbefore loop/JSC teardown;destroy()andglobal_exit()cover the paths that skipon_exit().
Extended reasoning...
Overview
This PR unifies all stdout/stderr output on a JS thread — console.*, process.stdout/stderr, Bun.stdout.writer(), Bun.write(Bun.stdout), console.write — through one per-VM FileSink per fd, held by RareData. The console formats natively into a scratch buffer and either writes straight to the sink or, when user code could observe it (patched write, corked, custom _stdout), delivers one JS string through a port of Node's kWriteToConsole. It touches 42 files across Rust (FileSink.rs +~460 lines, ConsoleObject.rs largely rewritten, PipeWriter.rs, sys/lib.rs, VirtualMachine.rs, rare_data.rs), C++ (BunProcess.{h,cpp}, ConsoleObject.cpp, ZigGlobalObject.cpp, c-bindings.cpp), builtin JS (ConsoleObject.ts, ProcessObjectInternals.ts, streams.ts, writable.ts, worker_threads.ts, diagnostics_channel.ts), codegen, spawn, and docs.
Security risks
No new attack surface in the security sense — this is stdio plumbing. The relevant risk classes are memory safety (extensive raw-pointer / unsafe work in FileSink.rs around the shared-sink lifetime, JsCell borrows during drain_sync/write_all_sync, and the Strong root for stdio_js), signal-handler safety (bun_restore_stdio runs from a SIGINT/SIGTERM handler and now also from pre-execve), and cross-thread correctness (the new StdioLock reentrant mutex, the STDIO_MADE_BLOCKING atomic read from any thread that spawns). None of these are user-input validation issues.
Level of scrutiny
High. This is hot-path, production-critical code that every Bun program exercises on every console.log. It changes fd flag management (O_NONBLOCK on shared open file descriptions affects parent shells and sibling processes), adds a blocking exit-time drain, rewires Writable semantics for process.stdout, alters signal-handler installation, and introduces documented behaviour changes (ArrayBuffer rejection, console.timeLog fd/format, console.clear escape sequence). The FileSink refcounting across RareData ↔ JS wrapper ↔ keep-alive, and the ordering of release_stdio_sinks vs. loop/JSC teardown, are exactly the kind of lifetime work that benefits from a maintainer's read.
Other factors
Test coverage is extensive and well-designed (real FIFO backpressure via parent-owned mkfifo, EPIPE matrix, fd-flag probes via compiled C, exit-drain via all three paths, worker rebinding), and four Node parallel tests are newly vendored. All five prior review findings across three passes were addressed with targeted commits and verified. The one candidate raised this run (spilled >64 KiB console message to a broken pipe emitting N+1 'error' events) was ruled out: __bun_stdio_sink_write calls report_stdio_error once per write_all_sync invocation, and after the first failure stdio_error is latched so subsequent spill chunks fail fast without re-reporting — but the outer emit() loop calls __bun_stdio_sink_write once per spill and once for the tail via deliver_to, each of which reports; however stdio_latch_error makes every call after the first return the same latched error, and report_stdio_error → reportStdioSinkError → stream.destroy(err) on an already-destroying stream is a no-op inside one tick, so the observable 'error' count is bounded by the destroy/undestroy cycle, matching the tested per-turn behaviour. Given the breadth, cross-platform branches (Windows SyncFile, Linux RWF_NOWAIT ≥6.4, macOS), and the number of interacting subsystems, this should not merge on automated review alone.
…AT, the one bound the native path already has
There was a problem hiding this comment.
No new findings this pass; all earlier inline comments (autoflush stdio latch, execve stdio restore, writeToObservedStream guard, write_with accepted-count, primordial indexOf, doc-comment placement) are addressed in the latest commits. This is a 42-file rework of console/process.stdout wiring, FileSink lifetime, fd-flag management and exit-path draining, with several intentional user-facing behaviour changes — worth a human look.
What was reviewed:
- Stdio-sink error paths (
on_auto_flush,drain_sync,write_all_sync,to_result,flush_from_js,on_error) — all now route throughstdio_latch_errorwithout marking the shared sink done. bun_restore_stdiobefore execve and the failed-execve path — now records the flip viaBun__stdioMadeBlockingso sinks downgrade.StdioLock—!SendviaPhantomData<*mut ()>; per-thread depth balances the process-wide mutex.Process::visitChildrenImplvisits the newm_consoleStream[]/m_stdioStream[]/m_consolePublishbarriers.
Extended reasoning...
Overview
Reworks how console.*, process.stdout/stderr, Bun.stdout.writer(), Bun.write(Bun.stdout) and console.write reach fd 1/2: one per-VM FileSink owns each fd, the console formats into a scratch buffer and delivers either straight to the sink or (when observably different) through the stream's JS write(). Touches FileSink, PosixStreamingWriter, ConsoleObject (Rust and C++), BunProcess, ZigGlobalObject, VirtualMachine lifecycle, rare_data, c-bindings (stdio snapshot/restore, SIGINT/SIGTERM handler install), posix_spawn, the built-in JS for streams/console/process/worker_threads/diagnostics_channel, plus ~500 lines of new tests.
Security risks
None identified. No parsing of untrusted input, no auth/crypto. The signal-handler install now respects an inherited SIG_IGN (a hardening). O_NONBLOCK handling is per-fd on 0–2 only and restored at exit/for children.
Level of scrutiny
High. This is core runtime I/O with cross-cutting lifetime concerns (per-VM sinks held by RareData, released before JSC teardown; JS wrapper roots dropped across bun test global swaps), fd-flag state shared across threads/children, exit-path draining, and several deliberate behaviour changes (process.stdout.write(ArrayBuffer) now throws, console.timeLog moves to stdout, console.clear sequence changes, _stdout/_stderr become accessors). The design decision to drain queued stdio synchronously on process.exit() (blocking on a slow reader, diverging from Node) is documented but should be signed off by a maintainer.
Other factors
Extensive test coverage for ordering under real backpressure, EPIPE per-call semantics, fd-flag preservation, cork/drain, exit-drain via all three exit modes, worker capture, and diagnostics_channel; four Node parallel tests newly vendored. Prior review rounds surfaced six issues, all fixed and verified in the diff. Given the surface area (42 files), the intentional behaviour changes, and the number of superseded PRs it consolidates, a maintainer should review before merge.
…tdout-compat-7d6001
…dArrayBuffer Bun's old own writeFast took these; Node's Writable rejects them. Keep the Bun behaviour for the two stdio streams (accepted in Writable's would-throw branch when the stream is stdio), without an extra prototype or own property so nothing about process.stdout's shape changes. Reverts the two test workarounds that had switched to Uint8Array.
There was a problem hiding this comment.
I've reviewed this PR across several passes and every finding has been addressed; this run found nothing new. Given the scope — a rewrite of how every byte reaches fd 1/2 (per-VM stdio sinks, console→stream observability via StructureID, O_NONBLOCK snapshot/restore across spawn/execve/signals, exit-time drain, new GC roots on Process) — a human sign-off is still warranted.
What was reviewed:
- FileSink stdio lifecycle: every terminal-error path now routes through
stdio_latch_error(incl.on_auto_flush);end()/close()on the shared sink flush without marking done; RareData ref released before loop/JSC teardown. - fd-flag coherence:
STDIO_MADE_BLOCKINGset by both spawn'smake_stdio_blockingandbun_restore_stdio_nonblock(failed-execve case), sorefresh_stdio_modecan't miss a downgrade. ProcessGC: newm_consoleStream/m_stdioStream/m_consolePublishWriteBarriers are all visited;m_consoleWriteFunctionLazyProperty visited.writable.ts_isStdioArrayBuffer branch checked for primordial safety —instanceofon user chunks matches Node's own non-primordial handling here and only widens what's accepted.
Extended reasoning...
Overview
This PR unifies all stdout/stderr writers (console.*, process.stdout/stderr, Bun.stdout.writer(), Bun.write(Bun.stdout), console.write) onto one per-VM FileSink per fd, and makes the global console deliver through process.stdout.write whenever that is observable — mirroring Node's kWriteToConsole. It touches 40 files across the Rust runtime (FileSink.rs, ConsoleObject.rs, VirtualMachine.rs, rare_data.rs, PipeWriter.rs, sys/lib.rs), C++ bindings (BunProcess.{h,cpp}, ConsoleObject.cpp, ZigGlobalObject.cpp, c-bindings.cpp), builtin JS (ConsoleObject.ts, ProcessObjectInternals.ts, streams.ts, writable.ts, worker_threads.ts, diagnostics_channel.ts), codegen, spawn, and adds a new StdioLock reentrant per-fd mutex. Behavioural changes include console.timeLog/timeEnd fd+format, console.clear() semantics, and exit-time draining of queued stdio.
Security risks
None identified. No new user-controlled input reaches parsing or path resolution. The signal-handler change (installing onExitSignal for SIGTERM/SIGINT on FIFO stdio too, but not over inherited SIG_IGN) is defensive. O_NONBLOCK manipulation is restricted to fds 0–2 and restored at exit; the is_on_pipefs fstatfs check gates RWF_NOWAIT correctly.
Level of scrutiny
High. This is core runtime plumbing that every program exercises on every console.log, and it interacts with process-global state (fd flags on shared open file descriptions, signal handlers, termios), VM lifecycle (exit, worker teardown, bun test global rotation), GC (new WriteBarrier fields, a Strong for the shared JS wrapper), and cross-thread ordering (StdioLock, STDIO_MADE_BLOCKING atomic). A regression here silently drops or reorders output, hangs the JS thread on a full pipe, or leaks fds/roots per VM. That is well beyond the bar for automated approval.
Other factors
The PR has been through multiple automated review passes; earlier findings (autoflush error latching, execve-failure fd-flag coherence, doc-comment placement, writeToObservedStream guard symmetry, primordial indexOf, write_with byte accounting) were all addressed with follow-up commits and are visible in the current diff. Test coverage is extensive and targets the specific failure modes fixed (backpressure ordering, EPIPE per-call, exit drain, fd-flag inheritance, spyOn(process.stdout, 'write')). The design decisions — StructureID-based observability detection, blocking-until-JS-writer-exists on FIFOs, draining on process.exit() where Node truncates — are sound but are exactly the kind of runtime-semantics choices a maintainer should ratify.
…t drain, lock scope, colours - Bun.stdout.writer() / Bun.file(1|2).writer() coalesce small writes again when stdout is a file or /dev/null (only a TTY writes through), as before this branch; the console and process.stdout paths never coalesced anyway - Bun.write(Bun.stdout, x) no longer drains synchronously: it switches the sink to its async pipe mode like process.stdout does and, when the bytes had to queue, resolves (with its own count) once the queue drains - exit-time drain: JS can't run any more, so hand every catchable signal back to SIG_DFL first (Node's ResetSignalHandlers) — waiting on a reader that never reads stays killable with ^C / SIGTERM - the Output pre-write hook does nothing while panicking (crash reporter) - StdioLock is taken per write / spill inside write_all_sync, never across formatting or any JS, and the JS delivery path drains only its own fd: no thread holds one stdio lock while waiting on the other - poll dispatch re-checks a stdio sink's fd mode before the writable callback flushes (spawn may have made the description blocking) - Linux: if RWF_NOWAIT turns out to be refused at runtime, fall back to O_NONBLOCK instead of blocking full-chunk writes - colours for a foreign console stream (worker port, console._stdout = x) follow FORCE_COLOR/NO_COLOR, else that stream's isTTY, as Node does - non-stdio FileSink-backed streams (child.stdin, tty.WriteStream(fd)) keep end-of-tick coalescing and keep accepting a bare (Shared)ArrayBuffer - on_write's stricter "wait for the coalesced tail" rule is stdio-only - process.stdout's cork/uncork/end/destroy wrappers are named and non-enumerable; Windows write_all_sync reports a short write
…tdout-compat-7d6001
…tdout-compat-7d6001
…stderr queued before it writes to the same fd
…nd leave a gone reader for the event loop to report
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings (all three are trade-offs the description already discloses), on this cut-down version I also checked: the new m_stdioSinks WriteBarriers are visited in visitChildrenImpl in the same change; __bun_flush_stdio_sink holds a RefPtr guard across flush_buffered (which can reach writer.end() → on_close), and the FlushPendingTask skip path still drops the from_raw ref it balances; the extern "Rust" declaration in ConsoleObject.rs matches the #[unsafe(no_mangle)] safe fn definition parameter-by-parameter and follows the existing cross-tier __bun_* pattern; and extracting wait_for_writable leaves fd_write_all_quiet behavior unchanged (it still only consults is_err()). None of those turned up a problem.
Extended reasoning...
The PR was reduced from roughly 3,000 added lines to about 200 across FileSink.rs, ConsoleObject.rs, BunProcess.cpp/.h, sys/lib.rs and PipeWriter.rs, so every earlier review comment targeted code that is no longer present. The remaining change makes console.* synchronously drain the process.stdout/stderr FileSink queue on unix before writing, which blocks the JS thread on poll(2) and alters pending-promise settlement timing; it touches no auth, crypto or injection surface. The three inline findings are behavioral trade-offs the author acknowledges in the description rather than defects they overlooked, but native refcount/GC-rooting code plus a deliberate blocking-semantics decision warrant a maintainer's judgment, so this is not approved.
…es, before taking the console lock
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline nit, I also checked the slimmed-down flush loop in __bun_flush_stdio_sink (src/runtime/webcore/FileSink.rs) for re-entrancy and termination: a console.* call made from a 'drain'/'error' callback finds the writer's outgoing buffer already emptied (_on_write/_on_error clear it before any JS runs), so the nested flush is a no-op; the loop exits on a closed writer via get_poll() == None, posix::poll already retries EINTR, and the fd_write_all_quiet refactor keeps the old "poll failed → return false" behaviour (an Ok(false) from POLLHUP still proceeds to the write and surfaces EPIPE as before). m_stdioSinks is visited in visitChildrenImpl in the same change.
Extended reasoning...
The PR was reshaped since the last review into a small change: a per-Process WriteBarrier<JSObject> m_stdioSinks[2] (non-Windows) exposed via Bun__Process__getStdioSink, a Rust-ABI __bun_flush_stdio_sink that polls for POLLOUT and runs the writer's on_poll until the stdio FileSink's buffer is empty, a call to it in message_with_type_and_level_ before the console stream lock, a wait_for_writable helper extracted from fd_write_all_quiet, and five new subprocess tests. It touches no auth/crypto/injection surface, but it does touch raw-pointer/refcount FFI code and blocks the JS thread on poll(-1), and the inline nit about the sh-pipeline exit code is already posted, so this note only records what else was examined.
…event loop to report
There was a problem hiding this comment.
I reviewed the latest push and found no bugs; the pipeline tests now carry the writer's own exit status and assert nothing else reached stderr, which closes my last comment. Since the change makes every console.* call block the JS thread until the reader has taken the stream's whole backlog (and reorders write callbacks as the description notes), a maintainer should still sign off on that trade-off.
What was reviewed:
__bun_flush_stdio_sinkagainst the event loop'son_pollpath: samedrain_buffered_data→on_writepair,RefPtrguard held across re-entranton_write, nestedentered()inrun_pendingstays below the no-checkpoint scope so no microtask drain insideconsole.log.Errarm leaves the buffer and the armed poll untouched;wait_for_writablekeepsfd_write_all_quiet's EINTR retry and failure polarity.m_stdioSinksis visited invisitChildrenImpl,from_jsonundefinedreturnsNone, and every new symbol/declaration is gated the same way on both sides (#if !OS(WINDOWS)/#[cfg(unix)]).
Extended reasoning...
The diff adds a POSIX-only flush of the process.stdout/stderr FileSink backlog before each console.* write: a WriteBarrier pair on Process plus an extern "C" getter in BunProcess.cpp/.h, an unsafe Rust loop in FileSink.rs that polls POLLOUT and drains through the existing writer callbacks, a cross-crate extern "Rust" call from ConsoleObject.rs, a small wait_for_writable extraction in src/sys/lib.rs, and six new subprocess tests. It touches no security-sensitive surface. Not approving because the change deliberately stalls the event loop for the length of a slow reader's backlog and changes callback ordering, which is a product decision for a maintainer, the loop is unsafe native code re-entering JS-visible state, and the timeline is incomplete with third-party comments I cannot see.
…tdout-compat-7d6001
…stream's buffer, as in Node
There was a problem hiding this comment.
Beyond the inline findings, I also checked the new append path's lifetime and ordering: behind_sink is declared after _stream_lock, so the guard's append runs under the per-stream lock before it is released; a throwing inspect.custom (format2(...)?) still delivers the partial message through the guard, as the direct write did on main; and the m_stdioSinks WriteBarriers are visited in Process::visitChildrenImpl in the same change, with getObject() only reached after the RETURN_IF_EXCEPTION on getIndex.
Extended reasoning...
The change replaces the earlier flush-before-write approach with formatting console.* output into a buffer and appending it behind the FileSink's queued bytes when process.stdout/stderr's fd is backed up, touching src/jsc/ConsoleObject.rs, src/runtime/webcore/FileSink.rs and the Process object bindings; it touches no security-sensitive surface. The inline findings are behavioral trade-offs of the new design (a console line dropped by process.exit when it sits behind a backlog, unbounded in-RAM buffering of console output during a synchronous loop, and worker output interleaving with a queued main-thread message) that a maintainer should weigh, so this is not approved.
Findings marked 🟡 are optional suggestions and need no follow-up push.
| let writer: &mut dyn bun_io::Write = if __bun_stdio_sink_is_backed_up(global, fd) { | ||
| &mut **behind_sink.insert(scopeguard::guard(Vec::new(), move |message: Vec<u8>| { | ||
| if !__bun_stdio_sink_append(global, fd, &message) { | ||
| let _ = raw_writer.write_all(&message); | ||
| let _ = raw_writer.flush(); | ||
| } | ||
| })) |
There was a problem hiding this comment.
🔴 Programs with Web Workers can now get a worker's console line spliced into the middle of a main-thread console message when stdout is backed up; on the base every console message reached the pipe whole. The message is appended to the sink at src/jsc/ConsoleObject.rs:468 and later written by the poll's drain in as many write(2) calls as the pipe allows, outside the console lock taken at src/jsc/ConsoleObject.rs:419, so a worker's direct console write can land between two parts. Fix: keep a console message atomic across threads once it is queued, e.g. drain the sink's tail under the per-fd console lock or write each queued message in one write(2) that is not split across rounds.
Why this was flagged
Trigger: fd 1 is a pipe or socket to a slow reader; the main thread runs process.stdout.write(Buffer.alloc(1 << 20)) (returns false), then several console.log lines, while a Web Worker (its own ConsoleObject, same fd 1, same STDOUT_MUTEX) also logs. The bytes are written later by on_poll at src/io/PipeWriter.rs:180-188, where drain_buffered_data (src/io/PipeWriter.rs:226-247) writes as much as the kernel takes and returns Pending at an arbitrary byte offset, which can fall inside a queued console message; the rest goes out on a later round after register_poll. No lock is held during those rounds, so the worker's console.log, which writes under the lock straight to fd 1 through its raw writer, lands between the two halves. On the base branch the main thread wrote the whole message through raw_writer and its flush while holding ConsoleStreamLock (src/jsc/ConsoleObject.rs:419), polling on EAGAIN under the lock, so other threads' console output could never split a message.
Verification: On this branch, when __bun_stdio_sink_is_backed_up is true (src/jsc/ConsoleObject.rs:466), the message is formatted into a Vec and, at the scopeguard drop (:467-472), handed to __bun_stdio_sink_append (src/runtime/webcore/FileSink.rs:236-239), which only does w.outgoing.write(message). The lock is released when _stream_lock drops at scope exit; nothing has been written to the fd yet.
| let writer: &mut dyn bun_io::Write = if __bun_stdio_sink_is_backed_up(global, fd) { | ||
| &mut **behind_sink.insert(scopeguard::guard(Vec::new(), move |message: Vec<u8>| { | ||
| if !__bun_stdio_sink_append(global, fd, &message) { | ||
| let _ = raw_writer.write_all(&message); | ||
| let _ = raw_writer.flush(); | ||
| } | ||
| })) |
There was a problem hiding this comment.
🟡 (optional) Users piping to a slow consumer now lose a console line entirely when process.exit() (or an 'exit' listener's log) follows a backed-up process.stdout.write; the base always delivered it. When __bun_stdio_sink_is_backed_up is true at src/jsc/ConsoleObject.rs:466, the formatted message is only appended to the sink's outgoing buffer at :468, and nothing drains that buffer at exit. This follows from the stated purpose (Node parity), and Node drops it too, so the call is the author's. Fix: if console output must survive process.exit() as on the base, drain the stdio sinks' outgoing synchronously before the process exits (also covers 'exit' listeners); otherwise document the loss.
Why this was flagged
Trigger: fd 1 is a pipe or socket to a slow reader (bun app.js | gzip); user code runs process.stdout.write(Buffer.alloc(1 << 20)) (returns false, remainder queued in the FileSink's outgoing) and then console.log("done"); process.exit(0) — or logs from a process.on('exit') listener reached by that process.exit. In message_with_type_and_level_, __bun_stdio_sink_is_backed_up(global, fd) at src/jsc/ConsoleObject.rs:466 is true, so the message is handed to __bun_stdio_sink_append at :468, which only does w.outgoing.write(message) (src/runtime/webcore/FileSink.rs:236-238). No code drains that buffer at exit: there is no drain/flush of m_stdioSinks in VirtualMachine.rs or FileSink.rs. On the base branch the console wrote "done\n" synchronously (blocking on EAGAIN) to fd 1 before process.exit ran, so the line reached the reader. After merging the line is silently lost together with the rest of the backlog; Node behaves the same, so this follows from the PR's purpose, but it is a new data-loss case relative to the base.
Verification: src/jsc/ConsoleObject.rs:466-472 — when __bun_stdio_sink_is_backed_up(global, fd) is true __bun_stdio_sink_append does w.outgoing.write(message). Nothing drains the FileSink's outgoing to completion, so the queued console line dies with the backlog's tail. On the base the console writer wrote the line directly and synchronously, so the line always reached the reader.
| fn __bun_stdio_sink_append(global: &JSGlobalObject, fd: i32, message: &[u8]) -> bool { | ||
| backed_up_stdio_writer(global, fd) | ||
| .is_some_and(|writer| writer.with_mut(|w| w.outgoing.write(message)).is_ok()) |
There was a problem hiding this comment.
🟡 (optional) Programs that log in a synchronous loop after one backed-up process.stdout.write now hold their whole output in memory and print nothing until the loop ends. Once the sink's outgoing is non-empty, every console.* call reaches __bun_stdio_sink_append at src/runtime/webcore/FileSink.rs:238, which only extends outgoing and never tries write(2) or waits for the reader, so the backlog grows by one message per call. Fix: bound the buffered bytes on this path, e.g. when outgoing is above a cap, flush() the writer and poll(POLLOUT) until it drops below it before appending, so order is kept but memory and latency stay bounded as on the base.
Why this was flagged
Trigger: fd 1 is a pipe or socket whose reader has not yet drained it, user code runs process.stdout.write(chunk) that returns false, then a synchronous loop of console.log. message_with_type_and_level_ at src/jsc/ConsoleObject.rs:466 sees __bun_stdio_sink_is_backed_up and formats into a Vec; its drop guard at ConsoleObject.rs:468 calls __bun_stdio_sink_append, which at FileSink.rs:238 does w.outgoing.write(message) and returns. No write(2), flush or poll happens on this path, and nothing else drains outgoing while JS runs synchronously, so every message of the loop accumulates in outgoing until the loop returns to the event loop. On the base branch console.log wrote straight to fd 1 and fd_write_all_quiet at src/sys/lib.rs:9235 waited on poll(POLLOUT) on EAGAIN, so memory stayed flat and output streamed at the reader's pace; a multi-gigabyte dump that streamed on the base now needs that much RAM and shows nothing until the loop ends. Node buffers similarly, but the base did not.
Verification: src/jsc/ConsoleObject.rs:466 picks the Vec path whenever __bun_stdio_sink_is_backed_up is true, and the drop guard at :468 calls __bun_stdio_sink_append; src/runtime/webcore/FileSink.rs:236-238 only does writer.with_mut(|w| w.outgoing.write(message)), no try_write, no flush, no poll wait, so bytes leave only after the synchronous JS returns. The base's console path wrote (blocking via poll on EAGAIN) per call.
|
#47167 takes Two points in #47167 depend on a decision here:
The sink in #47167 has one target, the fd. If this PR must land first, I will rebase #47167 on it. |
A console call wrote its message into a 4096-byte buffer of the VM while it formatted the message. Formatting runs user code, so a console call made from there landed inside the outer line, and a message whose formatting threw left a partial line. The stream mutex was held while that user code ran. Each message now goes through `emit`, which holds it until it is complete: up to 4096 bytes on the stack, then up to 64 KiB in a buffer of the VM. It is then written under the stream mutex, or not at all when formatting threw. A message over 64 KiB takes the mutex once and goes out in blocks, as before. A console.log-type call of one primitive string runs no user code and is written as before, with no copy. A message over 4096 bytes still ends with a write of its last byte alone. A later process.stdout.write then finds room on a nonblocking pipe and is not queued behind the next console message. process.exit() inside a hook of a message over 64 KiB gives the stream mutex back. The before-print hook of `bun test` takes a lock of its own, because a console call no longer runs it under the stream mutex. The names `emit`, `ConsoleWriter` and `SPILL_AT`, the 64 KiB bound and the buffer that the VM keeps come from the first revisions of #37128.
What
console.*writes to fd 1/2 directly.process.stdout/process.stderrbuffer whatever a full pipe or socket refuses and write it later from the event loop. The two do not know about each other, so aconsole.logissued after aprocess.stdout.write()can land in the middle of it.In Node the console writes through the stream, so its output queues behind what the stream already holds. This PR does the same: while a stream's sink is backed up, console output for that stream is appended to the sink's buffer. When nothing is buffered, the console writes as it does today.
Run as
... | (sleep 0.5; cat), runs of a byte collapsed. Linux x64, debug builds, node v26.7.0:<x*65536>MARK1<x*65536>MARK2<x*131072><y*262144><x*262144>MARK1<y*262144>MARK2<x*262144>MARK1<y*262144>MARK2Same result for
process.stderrwithconsole.error, and over a socketpair (Bun.spawn({ stdout: "pipe" })).How
Processkeeps the twoFileSinkwrappers thatprocess.stdout/process.stderrare built on (m_stdioSinks, visited by GC, so no lifetime code). Until a program touches those streams there is no sink.message_with_type_and_level_asks__bun_stdio_sink_is_backed_up(global, fd)when it picks its writer. If so, the message is formatted into aVecand, on the way out of the function,__bun_stdio_sink_appendadds it to the writer'soutgoingbuffer. The poll that is already armed for the bytes in that buffer writes it after them.inspect.custom), and that code can write until the backlog is gone. Then there is nothing to go behind: the append says so and the message is written directly.console.log()with no arguments now prints its newline through the writer the function picked, like every other message, instead of always through the stdout writer.SyncFile(start_syncinBlob::get_writer): every write blocks until it is out, so they are never backed up.Behaviour
Every row is either what Node does or exactly what main does. Pipe with a reader that starts late, stalls or leaves:
write()and console outputwrite(big, cb1); console.log(); write("small", cb2)ret2 false, thennextTick, microtask, immediate, timeout,drain,cb1,drain,cb2drainwrite(big); console.log(); write(big, () => process.exit(0))'drain'and the callbacksignal,drain,cb'error'listenercb1 EPIPE, (error,)cb2 EPIPEcb1 EPIPE,cb2 EPIPE, (error)console.login'beforeExit'/'exit'listeners with a backlogwrite(1 MiB); console.log("LAST"); process.exit()LASTin the middleLASTlostLASTlostconsole.error()/console.warn()with no argumentsconsole.logreturns, soprocess.exit()or a crash right after it drops it, as in Node.'SIGPIPE'listener and the reader gone, the listener runs 4 times on main, once here, never in node: the console no longer writes into the broken pipe itself.write()(injectedepoll_ctlerror; the console message is still printed), a one-off injectedEPIPEon a sink write,finished(process.stdout), a worker'sconsole.logwhile the main thread has a backlog, stdout to a file and to a tty.log,info,debug,dir,table,group,trace,warn,error,assert(false, msg)go behind it.count,time*andassert(false)without a message print through other paths and jump ahead, as on main.traceprints on stdout, as on main.console.log, with and withoutprocess.stdouttouched, to a pipe, a file and/dev/null: identical to main.Bun.stdout.writer(),Bun.file(1|2).writer(),Bun.write(Bun.stdout)andconsole.writehave their own sinks, as on main.Tests
test/js/node/process/process-stdio.test.ts, five tests next to #43868's, using its gated reader (nothing reads the fd until the firstwrite()has returnedfalse, no sleeps). The pipeline tests also assert the writer's own exit status and that it printed nothing else:log/info/debug/log()on a stdout pipe;warn/error/error()on a stderr pipe; stdout on a socketpair. Fail on main with the interleaving above.write(big); console.log(); write(big, () => process.exit(0))delivers every byte in order. Fails on main.inspect.customwrites until the backlog is gone is still printed. Passes on main; it covers the direct write when there is nothing left to go behind.test/js/web/console/console-log.test.ts:console.error()andconsole.warn()with no arguments print their newline on stderr. Fails on main.