Skip to content

test: speed up 32492.test.ts (36s -> ~9s on ASAN) and assert on bundle output - #35266

Closed
robobun wants to merge 2 commits into
mainfrom
farm/6a084016/speed-up-32492-test
Closed

robobun wants to merge 2 commits into
mainfrom
farm/6a084016/speed-up-32492-test

Conversation

@robobun

@robobun robobun commented Jul 23, 2026 •

Copy link
Copy Markdown
Collaborator

What

test/regression/issue/32492.test.ts was the slowest file on the debian 13 x64-asan lane at 36s (build #78397). This brings it down roughly 4x and replaces the exit-code-only check with real assertions on the bundled output.

Changes

  • Rounds: 16 -> 4 on debug/ASAN (kept at 16 on release, which is cheap). 96 spawns instead of 384.
  • Output sink: bundle to stdout instead of --outdir/--sourcemap=external, which drops 384 per-build output directories and lets the test assert on the bundle text. The regression lived in thread-pool shutdown, which runs identically regardless of where the bundle is written.
  • Assertions: per-build, check stderr == "", that the bundle contains the head and tail of the 40-module chain (var v40 = 40;, var v1 = 1 + v2;), and the entry-specific console.log(...) line, before checking exitCode. 16 expect() calls -> 484.

Why cutting rounds does not weaken the timing guard

The original regression was a ~10s stall because the first idle-futex wait in Event::wait had a 10 second timeout. #34009 lowered that first-wait timeout to 100ms (for the mi_on_thread_idle sweep). I reverted the || wake_threads == u32::MAX guard from #32494 on current main and re-ran both versions of the test:

with fix fix reverted
p50 build (24-way, 192 samples, debug+ASAN) 928ms 1009ms
max build 1090ms 1220ms
original 16-round test pass pass (3/3)

The ~80-100ms delta is real and consistent with workers hitting the 100ms first-wait timeout, but it sits inside 24-way scheduling noise and well under the 9000ms threshold. The existing 16-round test already cannot distinguish a reverted Event::wake fix on current main, so reducing rounds on debug/ASAN loses no fail-before signal there. The timing assertion remains as a coarse guard against multi-second shutdown stalls, and the new output assertions make it a useful concurrent-build smoke test.

(Note: only the first wait in Event::wait is bounded at 100ms; subsequent waits after the idle sweep are indefinite per ThreadPool.rs. The indefinite-hang case did not trigger across 1152 reverted builds in this workload, but the round reduction is justified by the empirical result above, not by a timeout bound.)

Verification

Local bun bd test test/regression/issue/32492.test.ts (debug+ASAN, 16 cores):

before after
test time 21.6s 4.9s / 5.0s / 5.0s
file time 24.4s 7.0s / 7.2s / 8.1s
expect() calls 16 484

Overlaps with the 32492 hunk of #33704 (which only reduces rounds); this PR is scoped to the one file and also adds the output assertions.


[stamp-90s] gate passed · iteration 0 · 1 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/regression/issue/32492.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "test/regression/issue/32492.test.ts"
bun test v1.4.0 (8a451d4bd)

test/regression/issue/32492.test.ts:
(pass) concurrent bun build does not stall on worker-pool shutdown [5145.81ms]

 1 pass
 0 fail
 484 expect() calls
Ran 1 test across 1 file. [7.59s]
Exit: 0
diff hotspot
test/regression/issue/32492.test.ts | 46 +++++++++++++++++++------------------
 1 file changed, 24 insertions(+), 22 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                 reads  edits  tests
test/regression/issue/32492.test.ts      2      4      0

Reduce rounds from 16 to 4 on debug/ASAN builds (96 spawns instead of 384)
and bundle to stdout instead of --outdir so the test can assert on the
actual bundle contents rather than only on exit code and wall time.

Since #34009 lowered the idle-futex timeout from 10s to 100ms, a skipped
shutdown wake now costs at most ~100ms, which is below the 24-way
scheduling noise floor. The 16-round version no longer distinguishes a
reverted Event::wake fix either, so cutting rounds does not weaken the
timing guard; it remains a coarse check for multi-second stalls plus a
concurrent-build smoke test with real output assertions.
@coderabbitai

coderabbitai Bot commented Jul 23, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 8 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4d8c1a90-3cc2-4130-910f-49653c2285e7

📥 Commits

Reviewing files that changed from the base of the PR and between 892b1da and 8a451d4.

📒 Files selected for processing (1)
  • test/regression/issue/32492.test.ts

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only speedup with strictly stronger assertions.

What was reviewed:

  • Round reduction gated on isASAN || isDebug (both exist in harness); PR description shows the 16-round variant already can't distinguish a reverted fix post-#34009, so nothing is lost.
  • Switch to stdout bundling: pipes are drained concurrently via Promise.all, bundle is small (40 tiny modules), no deadlock risk; root still used for cwd.
  • New per-build assertions check stderr, chain head/tail, and entry-specific line before exitCode — matches repo convention of asserting output before exit code.
Extended reasoning...

Overview

Single-file change to test/regression/issue/32492.test.ts, a regression test for a worker-pool shutdown stall. Three modifications: (1) ROUNDS drops 16→4 on debug/ASAN via isASAN || isDebug, (2) bun build writes to stdout instead of --outdir/--sourcemap=external, dropping the now-unused round parameter and per-build output dirs, (3) the exit-code-only check becomes five per-build assertions on stderr, bundle head/tail markers, the entry-specific console.log line, and exit code.

Security risks

None. Test-only; spawns bunExe() with bunEnv in a tempDir, no network, no external inputs.

Level of scrutiny

Low. No production code touched. The core question is whether coverage is weakened, and the PR description addresses this directly with measurements: after #34009 lowered the idle-futex timeout to 100ms, even the original 16-round test passes with the #32494 fix reverted, so the timing guard was already only a coarse multi-second-stall detector. Reducing rounds on the slow lanes preserves that while cutting ~75% of runtime. The removed --sourcemap=external/--outdir path is orthogonal to the pool-shutdown code path being guarded.

Other factors

  • Harness conventions followed: Promise.all([stdout.text(), stderr.text(), exited]) for concurrent pipe drain, output asserted before exitCode, isASAN/isDebug branching for load reduction.
  • The new toContain assertions on printer output (var v40 = 40;, var v1 = 1 + v2;) tie the test to bundler output shape, but that's normal for a bundler smoke test and makes it strictly stronger than the previous exit-code-only check.
  • The unused root variable is still used (cwd: root); the removed round parameter was only feeding the deleted --outdir path.
  • PR notes overlap with #33704's 32492 hunk; this is scoped to one file and adds assertions on top.

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. make more tests faster #33704 - Also modifies test/regression/issue/32492.test.ts to reduce test rounds on ASAN/debug builds for faster CI

🤖 Generated with Claude Code

Only the first futex wait in Event::wait has the 100ms timeout; later
waits are indefinite (see ThreadPool.rs:1412-1416). The round reduction
is justified by the empirical revert result, not by a timeout bound, so
say that instead.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only speedup with stronger assertions.

What was reviewed:

  • isASAN/isDebug are exported from test/harness.ts and this branching pattern is used in other regression tests.
  • The timing guard (slowestMs < STALL_MS) and per-build exit-code check are both retained; new stdout/stderr assertions are added on top, so the test is strictly stronger per build.
  • Pipes are still drained concurrently via Promise.all, and output is asserted before exitCode per repo convention.
  • Checked that dropping --outdir/--sourcemap=external doesn't remove the property under test — the PR justifies this (shutdown path is sink-independent) and the round reduction is backed by an empirical revert experiment showing the 16-round version already can't distinguish the reverted fix on current main.
Extended reasoning...

Overview

Single-file change to test/regression/issue/32492.test.ts, a regression test for a ~10s worker-pool shutdown stall. The PR reduces ROUNDS from 16 to 4 on debug/ASAN builds (release stays at 16), switches from --outdir/--sourcemap=external to stdout bundling, and replaces the throw-on-nonzero-exit check with per-build expect() assertions on stderr, three bundle-content substrings, and exitCode. The unused round parameter to buildOnce is dropped since per-round outdirs are gone.

Security risks

None. Test-only change; no production code, no untrusted input handling, no network.

Level of scrutiny

Low-to-moderate. It's a test speedup, but REVIEW.md flags "never silently weaken an existing test" as a hard block, so the key question is whether the round reduction and dropped --sourcemap=external lose signal. The PR description addresses this head-on with an empirical revert of the #32494 fix showing the original 16-round test already passes on current main with the fix reverted (because #34009 lowered the first-wait timeout to 100ms). Given that, cutting to 4 rounds on debug/ASAN loses no fail-before signal, and the new bundle-content assertions add value as a concurrent-build smoke test. The 9000ms coarse guard against multi-second stalls is unchanged.

Other factors

  • isASAN and isDebug verified as exports from test/harness.ts:27,54; the same pattern appears in other test/regression/issue/ files.
  • Subprocess pipes are still drained concurrently (Promise.all([stdout.text(), stderr.text(), exited])), avoiding the 64KB-pipe deadlock — relevant now that stdout carries the full bundle.
  • Assertions are ordered output-before-exit-code per CLAUDE.md/REVIEW.md guidance.
  • No CODEOWNERS entry covers this path.
  • The PR notes overlap with #33704 (which only reduces rounds); this PR is a superset of that hunk and additionally strengthens assertions, so there's no conflict in intent.
  • Author reports local bun bd test passes (484 expect calls, ~5s test time), which also confirms the var vN = N; bundle-output shape and empty stderr on the stdout path.

@robobun

robobun commented Jul 23, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:09 AM PT - Jul 23rd, 2026

❌ @robobun, your commit 8a451d4 has 1 failures in Build #78631 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 35266

That installs a local version of the PR into your bun-35266 executable, so you can run:

bun-35266 --bun

@robobun

robobun commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

CI build #78631: 193 passed, 1 failed. 32492.test.ts (the only file this PR touches) passed on every lane including linux-x64-asan.

The one hard failure is test/js/bun/http/proxy-stress-errors.test.ts on ubuntu 25.04 aarch64, a panic: Failed to start File Watcher: EAGAIN crash that is unrelated to this test-only change. The remaining 6 are marked flaky and passed on retry.

Ready to merge from my side.

@github-actions

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@github-actions github-actions Bot closed this Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant