Skip to content

test: run 5 rounds of the reload() while Bun.build() test on a debug build with ASAN - #47241

Open
robobun wants to merge 3 commits into
mainfrom
robobun/52e9cc7b/size-reload-build-race-test-for-debug
Open

robobun wants to merge 3 commits into
mainfrom
robobun/52e9cc7b/size-reload-build-race-test-for-debug

Conversation

@robobun

@robobun robobun commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Behaviour change: none

Problem

  • On a debug build (bun bd, ASAN on), reload() while Bun.build() resolves the same directory (test/js/bun/util/filesystem_router.test.ts:1021) fails: this test timed out after 60000ms. Its fixture needs 58 to 85 s there.
  • The time is in its 161 Bun.build() calls: 0.36 to 0.48 s each on that build, 0.02 s on a release build.

Fix

  • A debug build with ASAN runs 5 rounds. Every other build still runs 40. A round is 4 builds and 50 reload() calls.
  • The 5 rounds find the race. Without the entries_mutex snapshot in the route loader, the fixture ends in AddressSanitizer: heap-use-after-free in 26 of 26 runs.
  • Verified: bun bd test test/js/bun/util/filesystem_router.test.ts, 51 pass, the test takes 11 to 17 s. Main: 50 pass, 1 fail.

Background

  • The test is from resolver: take the per-entry lock when rewriting cached directory entries #33056. router.reload() lists a cached directory while the bundler thread reads that directory again in place.
  • ASAN stops the process at the first read of freed memory. A build without ASAN fails only when that read crashes, so it keeps 40 rounds.
  • The debug build is slow because of memory work: 73 s of system time for 28 s of user time. Each build makes about 187 mimalloc heaps, and mimalloc with MI_DEBUG commits memory with mprotect. The 12 pool threads wait for the address-space lock (mmap_lock).
  • Considered a higher timeout: each run then takes 58 to 85 s. No allocator setting made 40 rounds fit (Notes).
Notes

All numbers are from one 12-core Linux x64 KVM guest. "Fixture" is the body of the test's fixture.ts with performance.now() around the loop. The debug build is bun bd (ASAN on) of main 11c3671.

Where the time goes

run, debug build total user system
fixture, 40 rounds (5 runs) 58.5, 59.7, 67.1, 71.4, 84.6 s 26 to 28 s 58 to 73 s
builds only, no reload() (4 runs) 57.6, 62.3, 75.7, 77.7 s 21 to 23 s 62 to 73 s
reload() only, 2,000 calls 8.0 s 4.5 s 3.9 s
fixture, 4 pool threads (GOMAXPROCS=4) 51.2 s 26 s 32 s
fixture, 2 pool threads (GOMAXPROCS=2) 46.3 s 26 s 23 s

A release build (367d939) runs the same fixture in 3.1 s.

The 12 Bun Pool threads have most of the system time (45 of 58 s in one run). The two samples below are from runs of the builds only. A sample of the kernel stacks of the blocked threads (/proc/<pid>/task/<tid>/stack, 9,573 samples) shows what they wait for:

  • 69%: a page fault that waits for mmap_lock (lock_mm_and_find_vma, do_user_addr_fault)
  • 27%: mprotect (do_mprotect_pkey)
  • 2%: mmap, 2%: madvise

The pool threads are blocked like this for 30% of the run. The guest also reports 287 to 358 s of steal time per run (/proc/stat). I did not measure how much of the wait the host causes.

A sample of the user stacks of the pool threads (13,570 samples that are not asleep) shows who asks for the memory. The ASAN runtime has no frame pointers, so the callers behind it come from a scan of the stack for return addresses:

  • 55% are in the ASAN allocator (__asan::Allocator::AllocateImpl), 14% in mprotect, 11% in the first write to a new mimalloc page.
  • About 30% come from Worker::create (src/bundler/ThreadPool.rs:618), which clones the Transpiler and its BundleOptions for each pool thread of each build.
  • About 32% come from rename_symbols_in_chunk and about 8% from the other work for each chunk. The fixture has 80 entry points, so each build has 80 chunks.

mimalloc statistics for 161 builds (Bun.unsafe.mimallocDump()): 30.1 K heaps, 60.0 K pages, 37.7 K commits, 9.5 K purges, 2.4 GiB purged. The commits are mprotect calls only in a debug build: _mi_prim_decommit (src/prim/unix/prim.c of mimalloc) calls mprotect(PROT_NONE) and asks for a commit later when MI_DEBUG is not 0. scripts/build/deps/mimalloc.ts:158 sets MI_DEBUG=3 for a debug build.

Settings that do not make 40 rounds fit

Builds only, debug build, one run each. The same run without a setting takes 57.6 to 77.7 s.

setting total system
MIMALLOC_PURGE_DELAY=-1 69.0 s 112 s
MIMALLOC_PURGE_DECOMMITS=0 67.6 s 59 s
ASAN_OPTIONS=allocator_release_to_os_interval_ms=-1 63.7 s 56 s
the first and the third 90.1 s 167 s

The 5 rounds still find the race

I removed the entries_mutex snapshot from RouteLoader::load (src/router/lib.rs) and from bust_dir_cache_recursive (src/runtime/api/filesystem_router.rs), so that both loops read the live map again, and built a debug build. These are the loops that #33056 changed. The fixture then ends like this:

rounds runs that end in heap-use-after-free
1 15 of 16
2 6 of 6
3 6 of 6
5 26 of 26

The reports I read show the race of the test: thread T0 reads the map in RouteLoader::load or in bust_dir_cache_recursive, and the Bundler thread freed it in RealFS::read_directory_with_iterator under Resolver::load_as_file.

Is this a regression?

I did not find one. A debug build of 367d939 (18 September) needs 83.1, 95.3 and 95.4 s for the fixture on the same machine. The runs of the two builds were not interleaved, and one build varies from 58 to 85 s, so these numbers do not show a difference between the builds. I did not build a commit between the two.

What I ran

  • bun bd test test/js/bun/util/filesystem_router.test.ts on this branch, 2 runs, the second on the last commit: 51 pass, 0 fail, 57 and 40 s for the file.
  • The test alone, 9 runs on the debug build: 11.4, 13.4, 13.4, 13.9, 14.3, 16.6, 16.7, 17.2, 17.2 s.
  • The test alone on a release build (1.4.3): pass in 2.7 s, with 40 rounds.
  • The file on main 11c3671, debug build: 50 pass, 1 fail (this test timed out after 60000ms), 129 s.

A debug build without ASAN

It keeps 40 rounds, as on main (Windows, or --asan=off). The 5 rounds rest on ASAN, and I have no count that is shown to find the race without it. I did not build this configuration, so its time for the test is not measured.

Other pull requests

#45617 and #46584 contain the same change of the round count (10 rounds on a debug build) next to other changes.


no test proof · iteration 3 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/util/filesystem_router.test.ts

…uild

A round is 4 builds of 80 entry points. A debug build takes about 0.4 s
for one such build, a release build about 0.02 s. The 40 rounds took 58
to 85 s on a debug build, and the test has a timeout of 60 s.

A debug build now runs 5 rounds (11 to 17 s). A release build still runs
40. With the entries_mutex snapshot of the route loader removed, the
5-round fixture ends in an ASAN heap-use-after-free in 26 of 26 runs.
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between d24fa20 and 72567f0.

Files selected for processing: 1
  • test/js/bun/util/filesystem_router.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Walkthrough

The filesystem-router race test selects 5 rounds when debug and ASAN are both enabled, and 40 otherwise. It uses the selected count for the build/reload loop and expected match count.

Changes

Filesystem router race test

Layer / File(s) Summary
Round selection and expected match count
test/js/bun/util/filesystem_router.test.ts
The test imports isDebug, selects 5 rounds when debug and ASAN are both enabled or 40 otherwise, and uses that count for loop iterations and expected matches.

Possibly related PRs

  • oven-sh/bun#46584: Updates the same concurrent FileSystemRouter.reload() and Bun.build() test to select its round count based on debug mode.

Suggested reviewers: alii, jarred-sumner

Priority: Normal

Merge Risk: Minimal · up to 94d8d

Merge Risk: ⚪ Minimal · up to 94d8d

The test change preserves coverage for standard builds and reduces only the intended debug-ASAN workload.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title names the main change: the reload() while Bun.build() test now runs 5 rounds on debug builds with ASAN. The PR objectives and the file summary describe the same change, so the title matches …
Description check Passed The description clearly explains the problem, the fix, the expected behavior, and verification results. It does not use the exact template heading "What does this PR do?", but the required information…
Full details: Title check

Explanation

The title names the main change: the reload() while Bun.build() test now runs 5 rounds on debug builds with ASAN. The PR objectives and the file summary describe the same change, so the title matches the changeset. The title is slightly loose because it omits that release builds keep 40 rounds, but it does not mislead.

Full details: Description check

Explanation

The description clearly explains the problem, the fix, the expected behavior, and verification results. It does not use the exact template heading "What does this PR do?", but the required information is present and complete.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

@robobun

robobun commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. The diff is green.

How I reproduced it: bun bd test test/js/bun/util/filesystem_router.test.ts -t "reload\(\) while Bun.build" on a debug build of main 11c3671 ends with this test timed out after 60000ms (50 pass, 1 fail for the file). On this branch the same command passes in 11 to 17 s.

CI: both runs ended as failed for one reason that is not in the diff. 2 macOS x64 test-bun jobs expired without an agent, in build 134642 and again in build 134850. In build 134850, 188 jobs passed, macOS aarch64 included. The 7 other failed jobs are all test-aot-drawn, a soft-fail lane, and they fail only in test/aot (main build 134661 passed with 9 such jobs failed). No run has a failure in test/js/bun/util/filesystem_router.test.ts, the only file that this PR changes.

Pull request: #47241

The 5 rounds rest on ASAN, which fails on the first read of a freed
listing. A debug build without ASAN has no such check, so it keeps the
40 rounds that every other build runs.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the follow-up commit (72567f0) gates the round cut on isDebug && isASAN, so a debug build without ASAN keeps the base's 40 rounds, which was the one concern raised on the previous version.

What was reviewed:

  • The rounds constant feeds both the fixture loop bound and the expected stdout (rounds * 50, 50 reload()/match() calls per round), so the exact-output assertion stays in sync for 5 and 40 rounds.
  • isDebug and isASAN are the existing harness.ts exports (Bun.version.includes("debug") and detectASAN()); no new helper was hand-rolled.
  • The subprocess still drains stdout/stderr/exited via Promise.all and asserts exit code 0 with signalCode: null, so a UAF crash remains observable; the per-test timeout was not raised.
Extended reasoning...

The change touches only test/js/bun/util/filesystem_router.test.ts, parameterizing the round count of the existing reload()/Bun.build() race test so a debug+ASAN build runs 5 rounds instead of 40 while release and non-ASAN debug builds are unchanged. It touches no security-sensitive surface and no production code. The diff is small and self-contained, the file has no CODEOWNER entry, and the second commit addressed the sole inline finding from the earlier review by narrowing the reduction to the configuration whose ASAN rationale justifies it. No third-party CHANGES_REQUESTED review is outstanding.

@robobun robobun changed the title test: run 5 rounds of the reload() while Bun.build() test on a debug build test: run 5 rounds of the reload() while Bun.build() test on a debug build with ASAN Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant