Skip to content

Bun.serve: free a gracefully stopped server at VM teardown - #37500

Closed
robobun wants to merge 6 commits into
mainfrom
farm/77e742a3/serve-graceful-stop-teardown-free
Closed

robobun wants to merge 6 commits into
mainfrom
farm/77e742a3/serve-graceful-stop-teardown-free

Conversation

@robobun

@robobun robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator

Repro

// h.js
const server = Bun.serve({ port: 0, fetch: () => new Response("x") });
globalThis.keep = server;
await server.stop(); // graceful
BUN_DESTRUCT_VM_ON_EXIT=1 ASAN_OPTIONS=detect_leaks=1 bun-debug h.js
Indirect leak of 2104 byte(s) in 1 object(s) allocated from:
    ...
    #12 <bun_runtime::server::NewServer<false, true>>::init src/runtime/server/mod.rs:2143
    #13 bun_runtime::api::bun_object::serve src/runtime/api/BunObject.rs:1568
Indirect leak of 3616 byte(s) in 1 object(s) allocated from:   (uWS::HttpContext, from uws_create_app)
    ...
SUMMARY: AddressSanitizer: 13661 byte(s) leaked in 158 allocation(s).

Everything is "indirect" because the app's filter closure points back at the NewServer Box: the whole server graph (Box, TemplatedApp, HttpContext, ~36 router nodes, config strings) is an unreachable cycle. server.stop(true) instead: nothing reported. A node:http server after server.close(): the same, 173 allocations. Whether a request was served makes no difference; what matters is that the server's JS object is still alive when the VM is torn down (without the globalThis.keep line an ordinary GC sometimes collects it first, and that path is clean).

On the ASAN CI lane this never fails a test, because leak:uws_create_app in test/leaksan.supp matches the app and LSan also suppresses everything reachable from a suppressed block. It does cost time: to match a suppression LSan has to run llvm-symbolizer over the binary, about 7s per child with a local debug build and about 15s with the lane's release binary, and every test child that stops a server gracefully (or closes a node:http server) and still references it at exit pays that. test/regression/issue/12042.test.ts took 16s on the asan lane and 0.03s everywhere else for this reason (#37479 works around it in that test); test/js/node/http/node-http-server-socket-end-drain.test.ts was another one. A server that is gracefully stopped inside a Worker leaks the same graph for real every time the worker exits, since worker teardown runs the same code.

Cause

After a graceful stop() the server downgrades its wrapper ref and waits for the wrapper's finalize() to free the Box. #36750 made finalize() free it synchronously when it runs during VM teardown (no event-loop tick will ever run the tasks schedule_deinit() would enqueue, so schedule_deinit() just sets DEINIT_SCHEDULED and returns past is_shutting_down()), but gated that on TERMINATED. Only an abrupt stop sets TERMINATED; the gate was there because, at the time, a graceful stop left keep-alive sockets in the app's socket group and NewApp::destroy would have orphaned them. So for a gracefully stopped server finalized by lastChanceToFinalize, schedule_deinit() sets the flag, finalize() declines to free, and nothing else ever will.

Fix

The fixing line is the removal of the TERMINATED condition in NewServer::finalize() (server_body.rs): the Box is freed whenever DEINIT_SCHEDULED was set during shutdown. That flag can only be set by the deinit_if_we_can() call inside finalize() itself (its only caller requires the Finalized state that finalize() has just set), so it means precisely "drained, and nothing was enqueued to free it".

NewServer::deinit() (mod.rs) additionally closes the app before destroying it when nothing has closed it yet. The close happens through the raw *mut Self, before the Box is reclaimed, because the close callback of any socket that is still linked re-derives &Self from that same pointer (the deinit_running guard keeps such a callback from re-entering deinit_if_we_can, as in stop_listening). NewApp::destroy only deinits the socket groups (us_socket_group_deinit unlinks and, in debug builds, asserts they are empty); every existing caller reached it with the app closed because every path that sets TERMINATED closes it (the abrupt stop synchronously, schedule_deinit in the task queued ahead of the deinit task), and that is still the case, so the new branch is only taken by the teardown path above and by a listen() that fails after binding its TCP listener (the http3 UDP bind failure), which used to destroy the app with the listener still linked.

Why destroying a gracefully stopped app at teardown is safe now: VirtualMachine::teardown runs a stop phase before the JSC heap is destroyed that stops every still-registered server abruptly and closes every connected socket in every linked group (close_all_socket_groups), and a graceful stop() itself now closes idle keep-alives. By the time lastChanceToFinalize reaches a server wrapper its groups are empty, so deinit()'s close() is a walk over empty lists and destroy has its precondition; the websocket scenario below is the case where a connection outlives stop() and is closed by the sweep, and the debug build's assertions in us_socket_group_deinit run under these tests. The Drop chain that runs is the one the abrupt path has been running at teardown since #36750. The non-shutdown path is unchanged.

I also checked the other ways a server could be drained only at teardown: the wrapper ref stays Strong until the server is drained, so a wrapper cannot be finalized by an ordinary GC while connections remain, and a request still in flight at process.exit() is released by the stop phase's abrupt close before finalizers run (tried it; clean). So after this every server that reaches teardown is freed by its finalizer.

Verification

test/js/bun/http/bun-serve-html-405.test.ts, next to the teardown tests from #36750. Four new children, each run against leaksan.supp minus the uws_create_app entry so the cycle is reportable, each creating its server after a timer tick (a Bun.serve() made synchronously from the module body has JSModuleLoader::evaluateNonVirtual, also suppressed, on every allocation's stack) and keeping it on globalThis:

  • Bun.serve stop() after serving a request
  • Bun.serve stop() before any connection
  • node:http server.close()
  • Bun.serve stop() with a websocket still open at process.exit() (drained by the teardown sweep, freed by the finalizer)

All four fail on the unfixed build with the reports above (158 and 173 allocations) and pass with this change; the two existing teardown tests still pass, and the unsuppressed graceful one drops from 7.2s to 0.4s locally since its child no longer needs the symbolizer. The children of 12042.test.ts and node-http-server-socket-end-drain.test.ts no longer use any suppression under the lane's environment; on this PR's asan lane (build 92266) those two files take 0.23s and 0.65s, against 16.0s and 15.0s in builds 91926/91993 on main. websocket-server.test.ts, websocket-server-stop-retained-ws.test.ts, websocket-server-reload-leak.test.ts, server-url-invalid.test.ts and node-http-server-abort-events.test.ts behave the same with and without the change here (the websocket-server.test.ts timeouts I saw under the debug ASAN build reproduce on the unfixed build in the same container; the change cannot run while a test is in progress, it only touches teardown and the listen-failure path).

test/js/bun/http/serve-http3.test.ts gets a validation test for the other path that now reaches the close in deinit(): a child occupies a UDP port, asks for http3: true on that port number (TCP binds, UDP fails, no retry for a fixed port), then binds plain TCP on it. With this change Bun.serve throws Failed to listen on UDP port N for HTTP/3 and the TCP bind succeeds; on the unfixed debug build the child dies with

bun-debug: packages/bun-usockets/src/context.c:67: us_socket_group_deinit: Assertion `group->head_listen_sockets == ((void*)0)' failed.

(a release build would instead keep the port bound by a listen socket whose group has been freed).

Also run by hand under the same LSan setup with the app suppressions removed, each stopped gracefully and still referenced at exit: a TLS server with serverName, an http3 server with and without http1, a node:http server closed while a request was still in flight, a development server with an HTML route, and a websocket server inside a Worker (exits with the worker's loop still alive, as in the process case). All of them are freed; the only things left in those reports are two unrelated leaks this turned up, tracked separately: the per-thread RequestContext pool a Worker never frees, and the HTMLBundle created by importing an .html file.

Not changed: leak:uws_create_app stays in leaksan.supp; removing it is a separate question once the lane shows nothing else hides behind it.

After a graceful stop() (or node:http server.close()) the drained
NewServer is only freed once its JS wrapper is finalized. When that
happens during VM teardown, finalize() has to free it inline because
schedule_deinit() cannot enqueue anything any more, but it only did so
when TERMINATED was set, which only an abrupt stop sets. A gracefully
stopped server still referenced at exit therefore leaked its Box, uws
app and routers (158 allocations; LeakSanitizer hides them behind the
uws_create_app suppression at the cost of a symbolizer run per child on
the ASAN lane, and a Worker leaks them for real).

finalize() now frees every server that schedule_deinit() found drained
during shutdown, and deinit() closes the app before destroying it when
no stop closed it already. The teardown's stop phase has closed every
socket before finalizers run, so that close is a no-op walk there; it
also covers a listen() that fails after binding the TCP listener.
@robobun

robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced with the script in the description (debug ASAN build, BUN_DESTRUCT_VM_ON_EXIT=1 ASAN_OPTIONS=detect_leaks=1: 158 indirect allocations after a graceful stop(), nothing after stop(true), 173 after node:http server.close()). The four new children in test/js/bun/http/bun-serve-html-405.test.ts fail on the unfixed build and pass with this change; the new serve-http3.test.ts case aborts in us_socket_group_deinit on the unfixed debug build and passes with it. Latest push moves the app.close() in deinit() ahead of reclaiming the Box, so no Rust reference to the server exists while close callbacks could re-derive one; both test files pass locally on it, CI running. Build 92266 (9164664) was green on every lane.

@robobun

robobun commented Aug 11, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:55 AM PT - Aug 11th, 2026

✅ @robobun, your commit f7bc94e8db8160fd39ba79e62c8f309f7f3d2863 passed in Build #92326! 🎉


🧪   To try this PR locally:

bunx bun-pr 37500

That installs a local version of the PR into your bun-37500 executable, so you can run:

bun-37500 --bun

@coderabbitai

coderabbitai Bot commented Aug 11, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

The PR updates NewServer shutdown behavior, permits finalization during VM shutdown, and adds LeakSanitizer and HTTP/3 regression tests for server cleanup.

Changes

Server shutdown cleanup

Layer / File(s) Summary
Application deinitialization
src/runtime/server/mod.rs
NewServer::deinit closes remaining non-terminated apps, guards re-entrant callbacks, marks the server terminated, and updates related documentation.
VM-shutdown finalization
src/runtime/server/server_body.rs
NewServer::finalize permits scheduled deinitialization during VM shutdown for gracefully stopped servers.
Leak regression coverage
test/js/bun/http/bun-serve-html-405.test.ts
LeakSanitizer tests cover graceful shutdown, retained servers, node:http closure, and open websocket teardown.
HTTP/3 bind cleanup
test/js/bun/http/serve-http3.test.ts
A regression test verifies that an HTTP/3 bind failure releases the TCP port for rebinding.

Possibly related PRs

Suggested reviewers: jarred-sumner, dylan-conway

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes freeing gracefully stopped servers during VM teardown, which is the main change.
Description check ✅ Passed The description explains the cause, fix, affected paths, and verification results in detail, despite using different headings than the template.

Comment @coderabbitai help to get the list of available commands.

Comment thread src/runtime/server/mod.rs Resolved
Comment thread src/runtime/server/server_body.rs Resolved
Comment thread src/runtime/server/mod.rs Resolved
Comment thread src/runtime/server/server_body.rs Resolved
Comment thread src/runtime/server/mod.rs Resolved
Comment thread src/runtime/server/server_body.rs Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs; the earlier nit about wiring the WebSocket close event in the test fixture was addressed in d4f46aa. Because the fix removes the TERMINATED gate on the synchronous deinit() in a JSC finalizer and adds an app.close() inside deinit() — correctness resting on the teardown ordering between close_all_socket_groups and lastChanceToFinalize — a human look at the server-lifecycle invariants would still be worthwhile.

What was reviewed:

  • Traced finalize() → deinit_if_we_can() → schedule_deinit()'s is_shutting_down() early-return to confirm DEINIT_SCHEDULED here means "drained, nothing enqueued".
  • Confirmed close_all_socket_groups runs in VirtualMachine::teardown before JSC heap destruction, so groups are empty when lastChanceToFinalize runs finalizers.
  • Checked that deinit_running.set(true) in the new deinit() branch guards re-entrance from app.close() callbacks (matches the pattern in stop_listening).
  • Tests: four LSan children run against leaksan.supp minus uws_create_app so the leak is reportable; test.concurrent.each, tempDir, pipes drained concurrently.
Extended reasoning...

Overview

Two Rust changes in the Bun.serve server lifecycle plus a test-file refactor:

  • src/runtime/server/server_body.rs (finalize()): drops the && flags.contains(ServerFlags::TERMINATED) condition on the inline deinit() call, so a gracefully-stopped server whose wrapper is finalized during VM shutdown is freed rather than leaked.
  • src/runtime/server/mod.rs (deinit()): if TERMINATED is not yet set when the app is destroyed, sets it, sets the deinit_running re-entrance guard, and calls app.close() before NewApp::destroy. The deinit_running field doc comment is updated to name deinit alongside stop_listening.
  • test/js/bun/http/bun-serve-html-405.test.ts: extracts an exitUnderLeakSanitizer helper, keeps the two existing teardown tests, and adds a describe with four scenarios (Bun.serve stop after/before a request, node:http close, Bun.serve stop with a live websocket at process.exit) each run against a suppression file with leak:uws_create_app removed.

Security risks

None identified. No user-input parsing, auth, crypto, or network-protocol changes. The change is confined to VM-teardown-time and listen-failure-time cleanup of an already-owned uWS app handle.

Level of scrutiny

High. This is the memory-safety category REVIEW.md flags as most-blocked: it removes a guard on a synchronous free inside a JSC finalizer, and its correctness depends on a non-local ordering invariant — that VirtualMachine::teardown's stop phase (close_all_socket_groups, plus abruptly stopping every registered server) has emptied every socket group before lastChanceToFinalize runs. I verified that ordering in src/jsc/VirtualMachine.rs:1919 and src/jsc/rare_data.rs:860-866, and the PR description enumerates the paths (abrupt-stop, graceful-stop-then-teardown, websocket-outliving-stop, in-flight-request-at-exit). The new app.close() in deinit() can in principle fire close callbacks that re-enter via AnyServer dispatch; the added deinit_running.set(true) blocks that, matching the existing pattern in stop_listening and deinit_if_we_can. But because a mistake here is a UAF or an orphaned socket rather than a visible error, and the invariant spans several files, a maintainer familiar with the server lifecycle should confirm the reasoning.

Other factors

  • The tests are well-constructed for this class of fix: they run under LSan with the masking suppression removed, so they fail on the unfixed build (per the PR: 158/173 allocations) and would catch a regression. They use tempDir, test.concurrent.each, drain stdout/stderr/exited concurrently, and skipIf(!isASAN || isWindows).
  • All prior feedback is addressed and resolved: my WebSocket-open nit (d4f46aa), and the comment-cop bot's length complaints (4d8f2c6, 9164664 — the SAFETY block in finalize() is now the same length as on main, and the deinit() comment is one line).
  • The non-shutdown path is unchanged (is_shutting_down() still gates the inline free), so the blast radius of a mistake is limited to process/worker exit and the http3-bind-failure listen path.

Covers the listen() failure path that now closes the app in deinit():
on the unfixed debug build the child aborts in us_socket_group_deinit
with the listener still linked, and a release build keeps the port
bound.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs; the earlier nit about wiring ws.onclose was addressed in d4f46aa. Because the fix removes a safety gate in the server's GC finalizer and adds a synchronous app.close() inside deinit() — both memory-lifecycle paths whose correctness rests on VM-teardown ordering — a human look would still be worthwhile.

Checked: schedule_deinit() only sets DEINIT_SCHEDULED after deinit_if_we_can() has verified the server is fully drained and js_value is Finalized, so removing the TERMINATED gate does not admit a non-drained server.
Checked: the new close() in deinit() runs after heap::take(this) reclaims the Box; re-entry via socket-close callbacks would alias the unique borrow, but on both callers (VM teardown after close_all_socket_groups, http3 UDP bind failure) the groups hold no connected sockets, and deinit_running is set defensively.
Checked: the http3 test's retry loop only retries on EADDRINUSE from the TCP bind, and the leak tests strip leak:uws_create_app so the cycle is actually reportable.

Extended reasoning...

Overview

The PR fixes a leak where a gracefully stopped Bun.serve (or node:http) server whose JS wrapper survives to lastChanceToFinalize leaks its entire NewServer Box + uws app graph. Two native changes: server_body.rs:2823 drops the && TERMINATED gate on the inline deinit() call in finalize(), and mod.rs:2119-2126 makes deinit() call app.close() before NewApp::destroy when nothing has closed the app yet. Tests: four new ASAN/LSan child-process scenarios in bun-serve-html-405.test.ts (run against leaksan.supp minus uws_create_app so the leak is reportable), plus an http3 UDP-bind-failure test that proves the TCP listener is released.

Security risks

None. This is teardown-time resource cleanup; no user input reaches the changed code, and no auth/crypto/permissions are involved.

Level of scrutiny

High. Per REVIEW.md, native memory safety is the most-blocked category, and this change removes a gate whose stated purpose was to prevent NewApp::destroy from orphaning keep-alive sockets. The argument that the gate is now redundant depends on non-local invariants: (a) VirtualMachine::teardown runs a stop phase + close_all_socket_groups before JSC finalizers, so by lastChanceToFinalize every group is empty; (b) DEINIT_SCHEDULED at that point implies the server was already drained (traced: deinit_if_we_can → js_value == Finalized → schedule_deinit sets the flag and returns early past is_shutting_down()). The reasoning is sound and the PR description is unusually thorough, but a maintainer who owns the teardown ordering (#36750, close_all_socket_groups) should confirm it.

The deinit() change also introduces an app.close() after heap::take(this) has formed the unique Box<Self>. If close() fired a socket-close callback that dispatched back through an AnyServer raw pointer, that would alias the Box's unique borrow. On the two paths that reach this branch (VM teardown with emptied groups; listen-failure with only a listen socket linked) no such callback fires, and deinit_running.set(true) is set to short-circuit deinit_if_we_can if one ever did — but the invariant is worth a maintainer's eye.

Other factors

  • Tests are well-constructed: verified to fail on the unfixed build (158/173 allocations), test.concurrent for the four independent children, subprocess pipes drained via Promise.all, WebSocket wait now rejects on close, tempDir + using for cleanup, and the http3 test retries on TCP-side EADDRINUSE to avoid port-collision flake.
  • All prior review threads (my WebSocket nit, the comment-cop notices) are resolved; the SAFETY comment in finalize() is the same length as on main with the TERMINATED sentence swapped for the teardown-ordering fact.
  • The non-shutdown finalize() path is unchanged (is_shutting_down() still gates the inline free), so runtime behavior outside VM teardown and the http3 listen-failure path is unaffected.

…ox in deinit()

The close callbacks of any socket still linked re-derive &Self from the
same pointer, so do the close before a Box<Self> exists rather than
while holding one.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs (the earlier WebSocket nit was addressed in d4f46aa). Because the fix removes a safety gate in finalize() and adds an unsafe close-before-destroy step in NewServer::deinit() whose correctness depends on non-local teardown ordering (the VM stop phase having emptied every socket group before lastChanceToFinalize), a human look at the lifecycle argument would be worthwhile.

Checked: the new app.close() runs through the raw *mut Self before the Box is reclaimed and sets deinit_running so re-entrant close callbacks short-circuit; the finalize() change is still gated on DEINIT_SCHEDULED && is_shutting_down(), so the non-shutdown path is unchanged. The four LSan children strip leak:uws_create_app from the suppression list so the leak is actually reportable, and the http3 UDP-bind-failure test covers the other new caller of the close path.

Extended reasoning...

Overview

The PR fixes a leak where a gracefully-stopped Bun.serve (or node:http) server whose JS wrapper survives to lastChanceToFinalize was never freed: finalize() only freed the Box inline when TERMINATED was set, and only an abrupt stop sets that flag. The native change is two hunks — dropping the TERMINATED condition in server_body.rs:finalize(), and adding a close-before-destroy step in mod.rs:deinit() for apps that were never closed (the graceful-stop-at-teardown path and the http3 UDP-bind-failure path). Tests add four LSan children run against leaksan.supp minus uws_create_app, plus an http3 test that exercises the listen-failure teardown.

Security risks

None identified. This is teardown-time resource release; no new user-controlled input reaches the changed code, and the change is confined to VM shutdown and a listen-failure error path.

Level of scrutiny

High. This is unsafe native code in Bun.serve's finalize/deinit path — REVIEW.md's most-blocked category. The correctness argument is non-local: it relies on VirtualMachine::teardown's stop phase (close_all_socket_groups) having emptied every socket group before JSC finalizers run, on deinit_running correctly short-circuiting any close-callback re-entry into deinit_if_we_can, and on the DEINIT_SCHEDULED-during-shutdown flag meaning exactly "drained and nothing enqueued". The PR description traces each of these carefully and the ASAN tests (including the websocket-still-open scenario) exercise the debug us_socket_group_deinit assertions, but a maintainer who owns the server lifecycle should confirm the invariant chain.

Other factors

The prior CI run (build 92266 on 9164664) was reported green on every lane; the head commit f7bc94e only reorders the app.close() ahead of the Box reclaim (a safety improvement) and CI is running. All prior review threads (my WebSocket nit and the comment-cop bot's comment-length flags) are resolved. Test coverage is thorough — each new child was verified to fail on the unfixed build — but the change removes a gate that was originally added for a reason, so human sign-off on the replacement reasoning is appropriate.

@github-actions

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@github-actions github-actions Bot closed this Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant