Skip to content

node:http2: send the rest of a body when WINDOW_UPDATE arrives inside the transport write - #43562

Open
robobun wants to merge 3 commits into
mainfrom
robobun/bccd6f85/http2-sync-transport-window-update
Open

robobun wants to merge 3 commits into
mainfrom
robobun/bccd6f85/http2-sync-transport-window-update

Conversation

@robobun

@robobun robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Over a synchronous transport (stream.duplexPair()), a node:http2 body larger than the flow-control window can stop after the first window: the second 200 KB response never arrives. Node v26.3.0 completes it.
  • Cause: send_data (src/runtime/api/bun/h2_frame_parser.rs:5193) writes the frames that fit the window as one batch, then queues the rest. Over duplexPair() the peer's WINDOW_UPDATE arrives inside that write, when nothing is queued to flush. send_data then queues the rest on the stale window value, and no later event sends it.

Fix

  • send_data reads the window again after that batch write and continues to send if it reopened. A native socket runs no JS under the write, so nothing changes there.
  • If that write closed the stream (a peer RST_STREAM handled under it), send_data completes the write callback and stops. On main the rest stays queued and the callback never runs.
  • Verified: test/js/node/http2/node-http2-sync-transport.test.ts (4 cases, main fails all 4). Also test/js/node/http2/, the vendored test-http2-* files, and grpc-js.
  • Self-reviewed: 12 concerns raised, 10 addressed. Rejected: a smaller batch reservation (a capacity hint) and the close tail after the last write (separate bug, see Notes).

Background

  • A sender can have one window (65535 bytes by default) of DATA in flight per stream and per connection. WINDOW_UPDATE frames return credit.
  • Over a JS Duplex the session writes through socket.write(), which runs JS. With duplexPair() that write can emit 'data' on the peer side before it returns.
  • Frames that do not fit wait in a per-stream queue. flush() sends it on WINDOW_UPDATE, on 'drain', and from a deferred auto-flush task.
Notes

Repro (hangs on main after request 0 closed after 200000 bytes, node prints both lines):

import http2 from "node:http2";
import { duplexPair } from "node:stream";

const server = http2.createServer();
server.on("stream", stream => {
  stream.on("data", () => {});
  stream.on("end", () => {
    stream.respond({ ":status": 200 });
    stream.end(Buffer.alloc(200_000, "r"));
  });
});
const [clientSide, serverSide] = duplexPair();
server.emit("connection", serverSide);
const client = http2.connect("http://localhost", { createConnection: () => clientSide });

for (let i = 0; i < 2; i++) {
  await new Promise(resolve => {
    const req = client.request({ ":method": "POST", ":path": "/" });
    let bytes = 0;
    req.on("data", chunk => (bytes += chunk.length));
    req.on("close", () => {
      console.log("request", i, "closed after", bytes, "bytes");
      resolve();
    });
    req.end("hi");
  });
}
client.close();
server.close();

Why the first request completes on main: the client's WINDOW_UPDATE write finds clientSide busy with an earlier write, so the Writable buffers it and the server sees it on a later turn, after the rest of the body is queued. The server's first batch is also larger than the 64 KiB high-water mark, so a 'drain' follows and flushes the queue. On the second request both conditions are gone: the WINDOW_UPDATE arrives inside serverSide.write(), and that write reports no backpressure. Variants hang on the first request on main: a response written as ten 20 kB chunks, and a server that pipes the request back. With the Windows default high-water mark (16 KiB) every window-sized write reports backpressure and the 'drain' rescues this exact repro, so the first new test passes an explicit 64 KiB.

BUN_DEBUG_H2FrameParser=1 on the second request, before the fix: four padding: ... size: 16374 lines for the batch, rewriteRead 62109 (the client reads it), engine WU received stream=0 inc=65535 and flushStreamQueue 0 (the server handles the update with an empty queue), then nine server queued ... lines and no further flush.

In the repro the peer session also takes the thread-wide cork slot inside that write (cork() force-uncorks the other parser), which unregisters the auto-flush that the response HEADERS had registered. With a hand-driven Duplex the same hang needs the body to be written after the HEADERS frame left, which is what the hand-driven tests do.

The black_box on stream: send_data holds &mut Stream across the write, and on_window_update / on_stream_reset write the same Stream through the pointer in the streams map. A minimal model of the loop, compiled with the pinned nightly at opt-level=3, keeps the second read only while the &mut Stream escapes somewhere (today through the black_box in queue_frame). Without an escape, LLVM hoists the stream window load out of the loop and the read after the write is folded into the one before it. The launder in send_data makes the second read independent of queue_frame. #40240 turns the Stream fields into Cells, which makes the launder unnecessary.

The second flush_batch_buffer() in the queue branch became a no-op with this change (a batch exists only while no queue condition holds, and the window case is flushed earlier now), so it is a debug_assert! that the batch is empty.

New tests, in their own file (node-http2.test.js spawns many debug-build subprocesses with 5 s timeouts, and these four cases run in seconds):

  • Two 200 KB responses over duplexPair(), the reported case.
  • A client request body over a hand-driven Duplex that answers from inside its _write when the body has used up a window. One case limits the body by the connection window, one by the stream window (that one covers the read through &mut Stream). All 200000 bytes and END_STREAM must arrive.
  • The same transport answers with RST_STREAM and WINDOW_UPDATE. No DATA may follow the reset, the end() callback must run, and session.state.outboundQueueSize must be 0. On bun 1.4.2 the callback never runs and outboundQueueSize stays at 9. On this branch without the can_send_data() check, 134465 bytes of DATA followed the reset.

Fail-before and pass-after were checked the way CI does it: a debug build of main's src/ fails all 4 new cases (timeouts), a debug build of this branch passes them.

Suites run with the debug build: test/js/node/http2/node-http2.test.js (391 pass), the other ten files in test/js/node/http2/, all 261 test/js/node/test/{parallel,sequential}/test-http2-* files, and eight grpc-js files. All of test/js/third_party/grpc-js/ with a release build of this branch: the same 322 pass and 6 fail as stock bun (DNS cases that need the public internet, and the tonic case). A release build passes the repro.

Found on the way, not changed here:


[human-review] gate passed · iteration 1 · 2 files touched

fails on main (without fix)
ASAN without fix: 4 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/http2/node-http2-sync-transport.test.ts"
bun test v1.4.3 (367d939d9)

test/js/node/http2/node-http2-sync-transport.test.ts:
(fail) two responses larger than the window complete over duplexPair [5000.15ms]
  ^ this test timed out after 5000ms.
(fail) a request body that uses up the connection window is answered with WINDOW_UPDATE [5000.06ms]
  ^ this test timed out after 5000ms.
(fail) a request body that uses up the stream window is answered with WINDOW_UPDATE [5000.05ms]
  ^ this test timed out after 5000ms.
(fail) a request body that uses up the connection window is answered with RST_STREAM and WINDOW_UPDATE [5000.05ms]
  ^ this test timed out after 5000ms.

 0 pass
 4 fail
Ran 4 tests across 1 file. [25.59s]
error: script "bd" exited with code 1
__F:4:S:0

release without fix: all passed
bun test v1.4.3-canary.1 (b3fc13254)

test/js/node/http2/node-http2-sync-transport.test.ts:
(pass) two responses larger than the window complete over duplexPair [12.01ms]
(pass) a request body that uses up the connection window is answered with WINDOW_UPDATE [3.42ms]
(pass) a request body that uses up the stream window is answered with WINDOW_UPDATE [0.93ms]
(pass) a request body that uses up the connection window is answered with RST_STREAM and WINDOW_UPDATE [1.94ms]

 4 pass
 0 fail
 4 expect() calls
Ran 4 tests across 1 file. [159.00ms]
__F:0:S:0
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" "test/js/node/http2/node-http2-sync-transport.test.ts"
bun test v1.4.3 (367d939d9)

test/js/node/http2/node-http2-sync-transport.test.ts:
(pass) two responses larger than the window complete over duplexPair [946.67ms]
(pass) a request body that uses up the connection window is answered with WINDOW_UPDATE [438.07ms]
(pass) a request body that uses up the stream window is answered with WINDOW_UPDATE [553.81ms]
(pass) a request body that uses up the connection window is answered with RST_STREAM and WINDOW_UPDATE [104.62ms]

 4 pass
 0 fail
 4 expect() calls
Ran 4 tests across 1 file. [11.31s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 2814ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/8] gen generated_host_exports.rs
generated_host_exports.rs: 121 exports (host=5, lazy=10, generic=106, rust=0); 245 extern-C blocks audited
[1/8] cargo bun_runtime → libbun_runtime.a
�[1m�[33mwarning�[0m�[1m: binary `bun_shim_impl` should have a kebab-case name�[0m
   �[1m�[94m|�[0m
�[1m�[94m 1�[0m �[1m�[94m|�[0m /workspace/bun/build/release/rust-target/.../bun_shim_impl
   �[1m�[94m|�[0m                                              �[1m�[33m^^^^^^^^^^^^^�[0m
   �[1m�[94m|�[0m
   �[1m�[94m= �[0m�[1mnote�[0m: `cargo::non_kebab_case_bins` is set to `warn` by default
�[1m�[96mhelp�[0m: to change the binary name to `bun-shim-impl`, convert `bin.name`
  �[1m�[94m--> �[0msrc/install/windows-shim/Cargo.toml:41:8
   �[1m�[94m|�[0m
�[1m�[94m41�[0m �[91m- �[0mname = �[91m"bun_shim_impl"�[0m
�[1m�[94m41�[0m �[92m+ �[0mname = �[92m"bun-shim-impl"�[0m
   �[1m�[94m|�[0m
�[1m�[33mwarning�[0m: `bun_shim_impl` (manifest) generated 1 warning
�[1m�[92m   Compiling�[0m bun_parsers v0.0.0 (/workspace/bun/src/parsers)
�[1
... (truncated)
diff hotspot
src/runtime/api/bun/h2_frame_parser.rs             |  47 ++++---
 .../node/http2/node-http2-sync-transport.test.ts   | 154 +++++++++++++++++++++
 2 files changed, 184 insertions(+), 17 deletions(-)

gate history · 1 passed · 1 rejected · iteration 1

evidence per changed file
file                                                  reads  edits  tests
src/runtime/api/bun/h2_frame_parser.rs                   17      7     14
test/js/node/http2/node-http2-sync-transport.test.ts      0      1      9

send_data() writes the DATA frames that fit the flow-control window as
one batch, then queues the rest of the payload. Over a synchronous JS
transport (stream.duplexPair) the peer's WINDOW_UPDATE for that batch is
handled before the write returns, when nothing is queued yet. The rest
was then queued on the stale window and no later event sent it.

Read the window again after that write and continue if it reopened.
Stop if a frame handled under that write closed the stream.
@coderabbitai

coderabbitai Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review paused — included plan limit reached

Keep your review moving with free on-demand reviews.

  • Run this review for free

On-demand reviews are free for one more day.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Promotion and pricing details

On-demand reviews are free for one more day. After that, they cost $0.25 per reviewed file.

Review limit details

Or wait 5 minutes for your next included review.

Check out review usage here.

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: df777a26-cbce-4556-86fd-5e0b68b638f0

📥 Commits

Reviewing files that changed from the base of the PR and between 9b7c982 and c47b28c.

📒 Files selected for processing (2)
  • src/runtime/api/bun/h2_frame_parser.rs
  • test/js/node/http2/node-http2-sync-transport.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:28 PM PT - Sep 19th, 2026

✅ @robobun, your commit c47b28c92e3aff9a233e60059320db37bb71597c passed in Build #118629! 🎉


🧪   To try this PR locally:

bunx bun-pr 43562

That installs a local version of the PR into your bun-43562 executable, so you can run:

bun-43562 --bun

@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on main (bun 1.4.3-canary.1+367d939d9, linux x64) with the script in the Notes block of the PR body. It prints request 0 closed after 200000 bytes and then hangs. Node v26.3.0 prints both lines and exits.
  • With this branch the script prints both lines and exits with code 0.
  • The fix and 4 tests (test/js/node/http2/node-http2-sync-transport.test.ts) are in this PR (node:http2: send the rest of a body when WINDOW_UPDATE arrives inside the transport write #43562). A debug build of main's src/ fails all 4 cases. A debug build of this branch passes them.
  • CI (build 118629): no failed job. 179 of 181 jobs passed and 2 had not started at the time of writing. Six tests that this diff does not touch passed on a retry. The diff is ready for a maintainer.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked whether the debug_assert! that replaced flush_batch_buffer() in the queue branch can trip on the has_backpressure() / outbound_queue_size > 0 cases: the batch is only appended inside this loop, and the only point where the loop re-enters JS (window hit zero) drains it first, so those conditions cannot flip between batched frames — the assertion holds. The native-socket path is unaffected by the new re-read since no JS runs under its write.

Extended reasoning...

Findings were reported inline, so this is the brief ruled-out note only. I traced the batch buffer's producers in send_data (src/runtime/api/bun/h2_frame_parser.rs:5293) and confirmed the only append site is the multi-frame else-branch of this same loop, which is reachable only when none of the queue conditions hold; the flush at the max_size == 0 branch is the sole re-entrant transport write within the loop, and it runs before the conditions are re-evaluated, so the removed flush_batch_buffer() was indeed a no-op on the queue branch. The remaining concerns (trailers/no_trailers path, the close tail, reliance on black_box) are covered by the inline findings and should be addressed by a human before merge.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟣 src/runtime/api/bun/h2_frame_parser.rs — A server that uses waitForTrailers over a TLS-upgraded Duplex can dereference a freed Stream and crash. no_trailers at h2_frame_parser.rs:5409 takes a raw &mut *stream without enter_stream_dispatch, then calls send_data whose empty-payload cork write can overflow into generic_write and the JS Duplex without arming dispatch_depth. A nested rewrite_read at depth 0 (3614) frees streams queued by an earlier free_resources, and the close tail at 5355 then reads the freed Stream. Fix: reborrow through enter_stream_dispatch in no_trailers like write_stream does at 5906, so the deferred free cannot run while the borrow is live.

    Extended reasoning...

    write_stream (5906) guards its borrow with enter_stream_dispatch, which arms dispatch_depth for the whole send_data call. no_trailers (5405-5421) does not: it dereferences the map pointer directly and passes it to send_data with an empty payload. The empty path at 5185 writes the header through to_writer(); when the cork is full that flushes via _write (2761). On BunSocket::None the JS call at 2414 arms the guard, but on BunSocket::Tls wrapping an UpgradedDuplex (the case transport_write_runs_js at 2826 names) generic_write writes into the JS Duplex with no guard. The peer processes the frame synchronously and replies; the first inbound record delivers RST_STREAM so on_stream_reset runs free_resources, which pushes the id onto pending_engine_stream_closes. A second synchronous record triggers a second rewrite_read; dispatch_depth is 0, so 3614-3627 removes the map entry and drops the heap Stream. Control returns to send_data, which reads stream.wait_for_trailers and stream.state at 5356-5367 from freed memory. This is pre-existing on the…

    Verification: pre-existing (the base already has this route; the diff modifies send_data — adding the black_box at 5150 and the can_send_data() re-check at 5201 only on the non-empty branch — but leaves the empty-payload sibling and no_trailers unguarded). Trigger: a stream whose free_resources already queued its id (e.g. JS _final at src/js/node/http2.ts:2706 calls native.noTrailers when the…

  • 🟣 src/runtime/api/bun/h2_frame_parser.rs — pre-existing, partial fix: over a JS transport (duplexPair, custom Duplex) a stream the peer resets inside the write of its last DATA frame is re-marked HALF_CLOSED_LOCAL, so Bun later answers the peer's RST_STREAM with its own RST_STREAM and reports a second stream error. The new can_send_data() check guards only the mid-loop flush at h2_frame_parser.rs:5200; the final flush_batch_buffer() at :5344 runs the same re-entrant JS and the close tail at :5361-5365 then overwrites CLOSED. Fix: after every transport write that can re-enter JS, skip the close tail when the stream is no longer sendable, at the 3 sites. Same pattern at 3 sites (h2_frame_parser.rs:5365, :1657, :5866).

    Extended reasoning...

    The PR notes list this as found but not changed; it is the same missing re-check the diff adds at :5201, one write earlier, so the fix is one line at each tail.
    Trigger: a peer whose handler calls stream.close(NGHTTP2_CANCEL) on the last DATA frame (or the RST_STREAM-then-WINDOW_UPDATE transport of the new it.each test, arriving on the write at :5344 instead of :5200).
    send_data batches the last frames of a payload that fits the window, so no iteration hits max_size == 0 and :5200 never runs.
    :5344 flush_batch_buffer -> _write -> onWrite -> the JS Duplex delivers the peer's RST_STREAM synchronously -> rewrite_read -> on_stream_reset at :4181-4187 sets state = CLOSED, :4210 free_resources (detach_context, queue id in pending_engine_stream_closes), dispatches onAborted/onStreamError to JS.
    Back in send_data, enqueued is false and close is true: :5361 compares state to HALF_CLOSED_REMOTE, it is CLOSED, so :5365 sets HALF_CLOSED_LOCAL and settled_state = 5; write_stream returns 5 and http2.ts:2905 runs onEndStreamSettled as if the stream ended cleanly.
    Later the JS close path calls…

    Verification: pre-existing, acknowledged in PR description ("found on the way, not changed here"; the note's claim that the tail overwrites CLOSED and a later RST_STREAM follows is accurate). Trigger: a JS transport (duplexPair / hand-driven Duplex, transport_write_runs_js()) whose peer answers with RST_STREAM synchronously inside the write of a multi-frame payload that fits the window, so no loop…

Comment thread src/runtime/api/bun/h2_frame_parser.rs Resolved
@robobun

robobun commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator Author

Reply to the two findings outside the diff. Both exist on main, and this PR does not change them.

The black_box thread is answered inline.

node-http2.test.js spawns many debug-build subprocesses with 5 s
timeouts. Keep the four new cases in a small file that runs in
seconds.
Comment thread src/runtime/api/bun/h2_frame_parser.rs Resolved
Comment thread src/runtime/api/bun/h2_frame_parser.rs Resolved
Comment thread src/runtime/api/bun/h2_frame_parser.rs Resolved
Comment thread src/runtime/api/bun/h2_frame_parser.rs Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant