Skip to content

fix(wb): run parallel unit test workers without per-file isolation - #1421

Merged
exKAZUu merged 4 commits into
mainfrom
fix/wb-test-timeouts-under-load
Oct 1, 2026
Merged

exKAZUu merged 4 commits into
mainfrom
fix/wb-test-timeouts-under-load

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Why

Since wb switched unit tests to bun test --parallel (#1347), CI has failed intermittently on tests that normally take well under a second. They run into their full timeout with killed 1 dangling process: deployRailway.test.ts (30 s), project.test.ts's mise env case (5 s), wbfy's packageJson.test.ts (60 s), buildWb's beforeAll (120 s), and others. PR #1420 hit it in 4 of about 10 runs. After #1420, 3 of 7 workflow_dispatch runs of the Test workflow on main failed the same way.

These are hangs, not slowness under CPU load. In the project.test.ts failures, mise env had already exited with status 0 and its full output was parsed, yet the spawnSync returned only when Bun killed the "dangling" child at the test timeout. This matches a Bun 1.4.2 bug (oven-sh/bun#34069). Its fix, oven-sh/bun#40078, is unmerged, and no newer stable Bun exists. With --isolate, which --parallel implies, the stdout/stderr writers of each discarded per-file global can be garbage-collected inside a later spawnSync. That corrupts spawnSync's private event loop, so it and later spawnSync calls in the same worker spin without noticing the child's exit, and the child stays a zombie. The new releasePlugin.test.ts does not cause the hang. Its many spawnSync calls only make the hang more likely to show up.

--no-isolate keeps the worker processes but runs each worker's files in one global, like a plain bun test run, so no per-file global is discarded. The other options were worse:

  • Moving every spawnSync reachable from tests (including production code in wb and wbfy) to async spawns is a large change that stays exposed to the same bug.
  • Dropping --parallel slows every repository's CI.
  • Raising timeouts or adding retries would only hide the hang.

Requirements

  • required: find the root cause of the intermittent timeouts in unrelated tests (mostly deployRailway.test.ts) that PR feat(wb): add the draft-first release plugin and complete pending releases in wb release #1420's CI runs hit, with evidence (CI logs, timings on main, local reproduction under load)
  • required: fix it at the root, with no blind retries and no blanket timeout inflation without justification
  • required: wb test keeps running unit test files in parallel worker processes (the contract in docs/expected-repository-rules.md)
  • chosen: pass --no-isolate instead of rewriting spawnSync callers, because it removes the trigger with a one-flag change and keeps parallel speed (a forced Bun.gc(true) preload per isolated file was measured and does not help: 10/10 wedged)
  • chosen: test files restore the process.env they change, and buildWb builds dist/ only once per parallel run, because files in a worker now share process.env and a leaked change made a later buildWb rebuild dist/ under another worker's CLI run

Customer Summary

wb test still runs unit test files in parallel worker processes, but the files that one worker runs now share a global, a module registry, and process.env. Tests that call spawnSync (directly or through wb/wbfy code) no longer hang until their timeout at random. A test file must now restore the global state, module spies, and environment variables it changes.

Technical Summary

  1. packages/wb/src/scripts/execution/baseScripts.ts: testUnit passes --parallel --no-isolate. E2E commands are unchanged (they never used --parallel).
  2. docs/expected-repository-rules.md: the unit-test rule now says that files in a worker share a global and module registry. A file must therefore not depend on state another file leaves behind, and must restore what it changes (spies, globals, environment variables). The suites here already restore their spies, and this matches the contract before feat(wb): run bun unit tests in parallel worker processes #1347, when bun test ran every file in one process.
  3. Files in a worker also share process.env. In this PR's first CI runs, privateRegistry.test.ts deleted the VERDACCIO_TOKEN that CI defines (empty), which changed the environment buildIfNeeded hashes. A later buildWb then rebuilt dist/ while another worker ran the CLI (Cannot find module .../wb/dist/index.js). Fixes:
    • packages/wb/test/unit/privateRegistry.test.ts restores the original VERDACCIO_TOKEN.
    • packages/wb/test/helpers/shared.ts: createTempDir restores process.env as the file found it, undoing initializeProjectDirectory's removal of npm/Yarn variables.
    • packages/wb/test/helpers/build.ts: after the first successful build of a parallel run, buildWb skips the build (a marker next to the run's lock), so a future leak can no longer delete dist/ mid-run.

Check hardest: whether any repository's unit tests rely on a fresh global or environment per file. Such a test would now fail depending on file order, which the rule change above forbids.

Testing

  • CI history: the failed attempts of feat(wb): add the draft-first release plugin and complete pending releases in wb release #1420's runs and of every failed or re-run Test run since 2026-09-21 show this hang signature (killed 1 dangling process on a normally fast test), apart from network-dependent cases. 3 of 7 workflow_dispatch runs on main failed with it (deployRailway.test.ts 30 s timeouts, and a 120 s buildWb hang).
  • This branch: 17 completed Test runs (push and workflow_dispatch) with no hang. The only 2 failures were the dist/ rebuild race described above, and the 9 runs since its fix all passed.
  • Linux reproduction (Docker, Bun 1.4.2 on Ubuntu 24.04, 4 CPUs): 12 test files, each calling execFileSync 150 times, run with bun test --parallel=4 and a 60 s cap. With --isolate, 7/10 and 9/10 runs wedged (the worker in state R, its sh child a zombie). With --no-isolate, 0/10 wedged. In that aarch64 container, the real wb suite did not wedge in either mode across 8 alternating runs, which is why the CI runs above are the evidence for the actual suite.
  • bun run verify passes. The changed test files and the CLI-using files (privateRegistry, project, setup, typecheck, buildIfNeeded, binDotenv, localServerUrl) pass with bun wb test.
  • Not exercised: WillBooster repositories other than this one under --no-isolate.

Notes

Remove --no-isolate once a stable Bun release contains the fix for oven-sh/bun#34069.

In Bun 1.4.2, the stdout/stderr writers of a global that bun test --parallel
discards after each file can be garbage-collected inside a later spawnSync,
after which spawnSync in that worker misses its child's exit and hangs until
the test times out. Running each worker's files in one global removes that
trigger while keeping the worker processes.

Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
@exKAZUu exKAZUu self-assigned this Sep 30, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses intermittent CI test failures caused by hangs in Bun's parallel test runner. By enabling the --no-isolate flag, the changes prevent the corruption of the event loop during spawnSync calls, ensuring stability while maintaining the performance benefits of parallel test execution. The documentation has been updated to reflect the requirement for tests to clean up global state due to the shared environment within worker processes.

Highlights

  • Workaround for Bun test hangs: Added the --no-isolate flag to unit test execution to resolve intermittent test hangs caused by a known bug in Bun 1.4.2 related to garbage collection of stdout/stderr writers during parallel execution.
  • Updated testing documentation: Modified the expected repository rules to clarify that test files within a worker process now share a global and module registry, requiring tests to properly restore state after execution.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.


Tests running fast in parallel rows, / But zombie processes cause us woes. / With no-isolate set, the hang is gone, / Now our CI can carry on.

Footnotes

  1. Review the Generative AI Prohibited Use Policy, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the unit test execution command in wb to use 'bun test --parallel --no-isolate' instead of just '--parallel'. This change is introduced to work around a bug in Bun 1.4.2 (issue #34069) where the garbage collection of stdout/stderr writers in isolated workers can cause spawnSync to hang. Correspondingly, the documentation in docs/expected-repository-rules.md and helper comments in packages/wb/test/helpers/shared.ts have been updated to reflect that test files running in the same worker process will now share a single global state and module registry, requiring them to restore any modified globals or imported modules. There are no review comments to address.

exKAZUu and others added 3 commits October 1, 2026 08:08
… leave

Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
… files

Workers now run their files in one global, so a file's process.env changes
reach the files after it. privateRegistry.test.ts deleted a VERDACCIO_TOKEN
that CI defines, which changed the environment buildIfNeeded hashes, so a
later buildWb rebuilt dist/ while another worker ran the CLI. Restore the
environment in the tests that change it, and build dist/ only once per run.

Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
…onment

Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant