Skip to content

bun test: add [[test.projects]] to scope a preload to files matched by a glob - #45379

Draft
robobun wants to merge 4 commits into
mainfrom
robobun/0c2837b5/test-projects
Draft

robobun wants to merge 4 commits into
mainfrom
robobun/0c2837b5/test-projects

Conversation

@robobun

@robobun robobun commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #44642

Status: waits for #44833. That draft adds [test] environment, --environment and the @jest-environment / @vitest-environment docblocks, switches the DOM per file without --isolate, and makes the builtins immune to DOM globals. It covers the reporter's DOM versus server case and changes the same runner files as this PR. After it lands, this PR is rebased onto its Environment plumbing as a thin [[test.projects]] layer for preloads that are not a DOM (a database, a server mock) scoped to a glob. The rework plan is in Notes.

Problem

  • bun test has one process-wide preload list. A happy-dom registration in [test] preload (the setup docs/test/dom.mdx recommends) replaces fetch, Request, Response, URL and the timers for every file, so server tests in the same repo break.
  • The only way around it is two bunfig files and two runs, or --isolate for the whole suite with a preload that branches on Bun.main. The reporter measured the second at 6 to 9x slower.

Fix

  • Adds [[test.projects]] to bunfig: name, testMatch (globs, relative to the project root like pathIgnorePatterns), preload. A file belongs to the first project that matches it. Files that match no project run with [test] preload only.
  • Files are grouped by project after --randomize and --shard. Each group runs in its own global: the runner swaps the global at a group boundary with the same machinery --isolate uses between files, then loads [test] preload and the project's preload into the fresh global.
  • load_preloads (src/runtime/jsc_hooks.rs) now consumes vm.preload once per global. The serial loop (src/runtime/cli/test_command.rs) and the parallel worker (src/runtime/cli/test/parallel/runner.rs) refill it after every swap. Both derive the project key from the path with one function (src/runtime/cli/test/Projects.rs). A worker swaps lazily when the next file's project differs.
  • Verified: test/cli/test/test-projects.test.ts (6 tests, all fail on the released bun). Also test/cli/test/isolation.test.ts, parallel.test.ts, path-ignore-patterns.test.ts, concurrent-test-glob.test.ts, test-randomize.test.ts, test-shard.test.ts, rerun-each.test.ts, bun-test.test.ts.

Background

  • --isolate gives each test file a fresh JS global on the same VM: swap_global_for_test_isolation stops leaked handles, retires the old global and creates a new one. It asserts vm.test_isolation_enabled, which this PR sets when the projects table is non-empty.
  • FirstLast tells bun:test whether a file is the first or last in its global, so preload-level beforeAll and afterAll run once per global. Groups set it from the group bounds. A --parallel worker keeps first: true, last: true as before.
  • The issue asks for a [[test.environments]] glob table with a per-entry isolate flag, or a per-file docblock. The review weighed both. A per-entry isolate flag fights the run-wide isolation state and re-runs the shared preload after every isolated file, the slowdown the reporter wants to avoid. A docblock costs every bun test user one file read per test file. The projects table is read once per process, matches Jest and Vitest projects, and groups files so the shared preload runs once per group.

Downsides

  • Users with a projects table: the base preload and its beforeAll/afterAll run once per project instead of once per run, and each group boundary pays one global swap (the --isolate per-file cost, paid once per project). Not measured: hyperfine is not available in the build container.
  • Users with [test] preload and no table: run_all_tests clones the preload list once per run (one Vec plus one Box<[u8]> per preload), the clone that VM init did before. Per file: zero allocations and zero syscalls, key_for returns at the is_empty check. Under --isolate, one Vec plus one Box per preload per file replaces a cached re-import of the same list. Binary size not measured: bloaty is not available in the container.
  • --parallel workers read [[test.projects]] from their own bunfig. A coordinator run with --config=<other file> does not forward it to workers (pre-existing for every [test] key).
  • Self-reviewed: 7 concerns raised, 2 addressed in this revision (comment length, project grouping in the parallel coordinator). The other 5 are the rework plan in Notes.
Notes

Rework plan once #44833 lands:

  • Rebase onto its Environment plumbing. A project entry becomes { name?, testMatch, environment?, preload? }. name becomes optional: nothing consumes it today.

  • Stop overloading vm.test_isolation_enabled for the group swap: add a distinct swaps-globals flag, and restore the if !test_isolation_enabled { preload.clear() } guard in load_preloads.

  • Fold the swap, the hook-scope reset and the vm.preload refill into one project-keyed helper used by the serial loop, the worker loop and the --rerun-each boundary from bun test: give each --rerun-each iteration a fresh global under --isolate #43184.

  • Keep Projects.rs (key_for, group, preloads_for) and the stable partition after --randomize and --shard.

  • Point docs/test/dom.mdx at [test] environment and the docblocks for the DOM case. Reserve projects for other preloads.

  • jsc::initialize(short_lived_globals) stays tied to --isolate. It is a JIT tier-up threshold. Project globals are long-lived.

  • reporter.projects.group() is a stable sort by key, so the order inside a group is the one --randomize or --shard produced. The parallel coordinator sorts lexicographically first and then groups, so each worker's contiguous chunk stays inside one project where possible.

  • A worker that steals a file from another project swaps its global before that file and again when it returns (tested: a --parallel worker swaps its global when the project changes).

  • --rerun-each repeats a file inside TestCommand::run without a swap. The preload list is already consumed, so repeats do not re-import it. Under --isolate the old code re-imported the (cached) list on each repeat with no visible effect.

  • Probed on the released bun (1.4.3): bunfig preload = ["./dom-preload.ts"] where the preload sets globalThis.window, then server.test.ts asserting typeof window === "undefined" fails. No existing key scopes the preload.


[human-review] gate passed · iteration 0 · 12 files touched

fails on main (without fix)
ASAN without fix: 6 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/cli/test/test-projects.test.ts
bun test v1.4.4 (367d939d9)

test/cli/test/test-projects.test.ts:
92 |   test("the project preload runs only for the files testMatch selects", async () => {
93 |     using dir = tempDir("test-projects", files);
94 |     const { stdout, stderr, exitCode } = await run(String(dir), []);
95 | 
96 |     // Files are grouped by project: the base group first, then "dom".
97 |     expect(runs(stdout)).toEqual(["a.server undefined", "c.server undefined", "b.dom object", "d.dom object"]);
                              ^
error: expect(received).toEqual(expected)

  [
    "a.server undefined",
+   "b.dom undefined",
    "c.server undefined",
-   "b.dom object",
-   "d.dom object",
+   "d.dom undefined",
  ]

- Expected  - 2
+ Received  + 2

      at <anonymous> (/workspace/bun/test/cli/test/test-projects.test.ts:97:26)
(fail) [[test.projects]] > the project preload runs only for the files testMatch selects [937.23ms]
188 | import { test, expect } from "bun:test";
189 | test("y", () => { expect((globalThis as any)
... (truncated)

release without fix: 6 FAILED
bun test v1.4.3-canary.1 (367d939d9)

test/cli/test/test-projects.test.ts:
188 | import { test, expect } from "bun:test";
189 | test("y", () => { expect((globalThis as any).project).toBe("two"); });
190 | `,
191 |     });
192 |     const { stderr, exitCode } = await run(String(dir), []);
193 |     expect(stderr).toContain("2 pass");
                         ^
error: expect(received).toContain(expected)

Expected to contain: "2 pass"
Received: "\nx.both.test.ts:\n1 | \n2 | import { test, expect } from \"bun:test\";\n3 | test(\"x\", () => { expect((globalThis as any).project).toBe(\"one\"); });\n                                                          ^\nerror: expect(received).toBe(expected)\n\nExpected: \"one\"\nReceived: undefined\n\n      at <anonymous> (/tmp/test-projects-order_vCuQOh/x.both.test.ts:3:55)\n(fail) x [0.38ms]\n\ny.test.ts:\n1 | \n2 | import { test, expect } from \"bun:test\";\n3 | test(\"y\", () => { expect((globalThis as any).project).toBe(\"two\"); });\n                                                          ^\nerror: expect(received).toBe(expected)\n\nExpected: \"two\"\nReceived: undefined\n\n      at <anonymous> (/tmp/test-projects-order_vCu
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/pr_gate.xml" test/cli/test/test-projects.test.ts
bun test v1.4.4 (367d939d9)

test/cli/test/test-projects.test.ts:
(pass) [[test.projects]] > with --isolate, every file gets its base and project preloads [1088.40ms]
(pass) [[test.projects]] > with --randomize, files stay grouped by project [1105.69ms]
(pass) [[test.projects]] > the project preload runs only for the files testMatch selects [1265.10ms]
(pass) [[test.projects]] > the first matching project wins and testMatch accepts a string [1093.70ms]
(pass) [[test.projects]] > a --parallel worker swaps its global when the project changes [1388.74ms]
(pass) [[test.projects]] > a project without a name or testMatch is a config error [375.99ms]

 6 pass
 0 fail
 27 expect() calls
Ran 6 tests across 1 file. [3.30s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     814a32f1b3
  features     lto, baseline

23 deps, 146 codegen, 1176 objects in 1799ms

ninja: Entering directory `/workspace/bun/build/release'
[1/4] fetch lolhtml
[lolhtml] up to date
[2/4] fetch rust-argon2
[rust-argon2] up to date
[2/4] cargo plan → /workspace/bun/build/release/rust-target/plan.json
264 units: 192 lib, 16 proc-macro (host), 19 custom-build (host), 15 run custom-build, 17 lib (host), 4 run custom-build (host), 1 rlib
[3/4] reconfigure
[1/1529] mkdir stamps
[2/1529] mkdir codegen
[3/1529] gen bake.{client,server,error}.js
-> bake.client.js, bake.server.js, bake.error.js
[4/1529] install /workspace/bun
bun install v1.4.3-canary.1 (367d939d9)

Checked 22 installs across 44 packages (no changes) [5.30s]
[5/1529] install /workspace/bun/packages/bun-error
bun install v1.4.3-canary.1 (367d939d9)

Checked 1 install across 2 packages (no changes) [262.00ms]
[6/1529] rustc unicode_ident 
[7/1529] rustc unicode_xid 
[8/1529] gen string-map defines_table
[9/1529] rustc heck 
[1
... (truncated)
diff hotspot
docs/guides/test/migrate-from-jest.mdx  |   1 +
 docs/runtime/bunfig.mdx                 |  14 ++
 docs/test/configuration.mdx             |  20 +++
 docs/test/dom.mdx                       |  16 +++
 src/bunfig/bunfig.rs                    | 119 ++++++++++++++++-
 src/options_types/context.rs            |  10 ++
 src/runtime/cli/mod.rs                  |   4 +
 src/runtime/cli/test/Projects.rs        |  81 ++++++++++++
 src/runtime/cli/test/parallel/runner.rs |  33 +++--
 src/runtime/cli/test_command.rs         | 109 +++++++++-------
 src/runtime/jsc_hooks.rs                |  13 +-
 test/cli/test/test-projects.test.ts     | 225 ++++++++++++++++++++++++++++++++
 12 files changed, 576 insertions(+), 69 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                     reads  edits  tests
docs/guides/test/migrate-from-jest.mdx       0      0      6
docs/runtime/bunfig.mdx                      0      0      7
docs/test/configuration.mdx                  0      0      6
docs/test/dom.mdx                            0      0      7
src/bunfig/bunfig.rs                         0      0      6
src/options_types/context.rs                 0      0      6
src/runtime/cli/mod.rs                       0      0      5
src/runtime/cli/test/Projects.rs             0      1      7
src/runtime/cli/test/parallel/runner.rs      0      0      6
src/runtime/cli/test_command.rs              2      0      7
src/runtime/jsc_hooks.rs                     0      0      6
test/cli/test/test-projects.test.ts          0      1      4

root cause · written by the author bot

The underlying problem is that bun test has a single process-wide preload list, so a DOM registrator such as happy-dom replaces Bun's real fetch, URL, Request, timers and similar globals for every file in the run, forcing repositories with both server and DOM tests into two separate invocations with duplicated config. The proposed change adds [[test.projects]] entries keyed by testMatch globs, partitions files by their first matching project, and runs each group's preloads in a freshly swapped global at group boundaries so the DOM registration cannot leak into server files. Th…

…y a glob

Files that match a project's testMatch run together in their own global
with [test] preload first and the project's preload after it. The serial
loop and the parallel worker both derive the project from the file path
and swap the global only when the project changes. load_preloads now
consumes vm.preload once per global and the runner refills it after
every swap.
@robobun

robobun commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:02 PM PT - Oct 15th, 2026

✅ @robobun, your commit 814a32f1b3e85c8407ee9030afa6cc31d704bbf2 passed in Build #126091! 🎉


🧪   To try this PR locally:

bunx bun-pr 45379

That installs a local version of the PR into your bun-45379 executable, so you can run:

bun-45379 --bun

Comment thread src/bunfig/bunfig.rs Outdated
Comment on lines +349 to +350
/// `[[test.projects]]`: each table needs a unique `name` and a
/// `testMatch`. `preload` is optional.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/options_types/context.rs Outdated
Comment on lines +472 to +475
/// `[[test.projects]]` in bunfig.toml, in file order. A test file belongs
/// to the first project whose `testMatch` matches it, else to the base
/// run. Each project runs in its own global with `[test] preload` plus
/// its own `preload`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/options_types/context.rs Outdated
Comment on lines +482 to +483
/// Globs matched against the test file path relative to the project
/// root, like `pathIgnorePatterns`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +1 to +8
//! `[[test.projects]]`: which project a test file belongs to, and which
//! preloads its global needs.
//!
//! Files are grouped by project and each group runs in its own global.
//! The serial loop and the parallel worker both derive the key from the
//! path with [`TestProjects::key_for`] and swap the global when the key
//! changes. `[test] preload` runs first in every global, then the
//! project's own `preload`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +13 to +14
/// Index into the `[[test.projects]]` table plus one. `BASE_KEY` is a file
/// that matches no project: it runs with `[test] preload` only.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +30 to +31
/// `preloads` and `projects` are the process-lifetime CLI context's
/// fields, so the views never dangle.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +52 to +53
/// The first project whose `testMatch` matches `abs_path` relative to
/// the project root, else `BASE_KEY`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +72 to +73
/// The preload list a fresh global needs before it runs a file with
/// this key: `[test] preload`, then the project's own.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/Projects.rs Outdated
Comment on lines +85 to +87
/// Stable partition of `files` by key, so each project's files are
/// contiguous and the global is swapped once per project. The order
/// inside a group is kept (`--randomize` and `--shard` already ran).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/parallel/runner.rs Outdated
Comment on lines +94 to +95
// Keep each project's files contiguous so a worker swaps its global
// (and re-runs the preloads) as rarely as possible.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/parallel/runner.rs Outdated
Comment on lines +564 to +565
// `fresh`: the global has not loaded its preloads yet. `cur_key`:
// the project whose preloads the current global loaded.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test/parallel/runner.rs Outdated
Comment on lines +592 to +593
// Without --isolate a worker keeps its global across files and
// swaps it only when the next file belongs to another project.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test_command.rs Outdated
Comment on lines +1953 to +1954
// `vm.preload` is filled per global by the file loops (`run_all_tests`
// and the parallel worker) from `reporter.projects`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test_command.rs Outdated
Comment on lines +1981 to +1982
// `[[test.projects]]` swaps the global between project groups with
// the same machinery `--isolate` uses between files.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test_command.rs Outdated
Comment on lines +2779 to +2781
// The global is fresh before the first file and after every
// swap: its preloads have not run yet, so the first file in
// it loads them and runs the preload-level `beforeAll`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/runtime/cli/test_command.rs Outdated
Comment on lines +2837 to +2839
/// Retires the global the last file ran in and creates a fresh one.
/// The caller sets `vm.preload` for the next file: `load_preloads`
/// consumes the list once per global.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

Comment thread src/options_types/context.rs Outdated
Comment on lines +472 to +473
/// `[[test.projects]]`, in file order. A file belongs to the first
/// project whose `testMatch` matches it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need a paragraph-long comment to justify why the workaround is OK, the code is wrong — fix the code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bun test: per-file test environment (preload and isolate selected by glob), so DOM and server tests can share one run

1 participant