Skip to content

bun test: run no beforeAll/afterAll inside an aroundAll that doesn't run - #48455

Draft
robobun wants to merge 2 commits into
mainfrom
robobun/eed2d49d/around-all-gates-its-hooks
Draft

robobun wants to merge 2 commits into
mainfrom
robobun/eed2d49d/around-all-gates-its-hooks

Conversation

@robobun

@robobun robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Draft: a maintainer picks the direction. Where no test runs, a bun:test beforeAll runs and the aroundAll around it does not. This PR stops the beforeAll (B). Alternatives: run the aroundAll too (A), or document main (C).

Problem

  • In a block of skipped tests, bun:test leaves out the aroundAll and runs the block's beforeAll/afterAll without its set-up. The repro fails: TypeError: undefined is not an object (evaluating 'db.query'), exit 1.
  • Cause: Order::generate_all_order (src/runtime/test_runner/Order.rs:43) decides per hook. It drops around hooks and keeps a bun:test beforeAll.

Fix

  • Order::enter_describe, in the pass that builds the run queue, decides per scope: one that runs no test and has an aroundAll queues none of its suite hooks. Nested scopes inherit that.
  • Correct because runSuite() runs those hooks (docs/test/lifecycle.mdx:132). Files with no aroundAll are unchanged.
  • Verified: 7 new tests in test/js/bun/test/test-test.test.ts, 6 fail on main. Self-reviewed: 18 concerns raised, 18 addressed.

Background

  • aroundAll wraps a file or describe block. No stable release has it (bun test: Vitest compatibility #44833).
  • bun:test runs beforeAll in a block of skipped tests (docs/test/writing-tests.mdx:508). Vitest and Jest do not.
  • Considered A: it runs set-up where no test runs. Considered a check in one block: nested blocks stay wrong.

Downsides

  • A cleanup afterAll in such a block stops running. A file-level afterAll that needs a gated beforeAll fails (exit 0 to 1).
  • Not fixed: a vitest or test.extend() beforeAll is left out beside a bun:test hook that runs.
  • Cost: +3 to +7 instructions per describe scope per file load. Release .text: 256 bytes smaller.
Notes

The question

In a describe block or file that runs no test (every test skipped or todo, or no test), main decides for each suite hook alone. It leaves out three kinds of hook, and it runs a plain bun:test beforeAll/afterAll beside them or in a block nested below them.

Hook that main leaves out Repro bun 1.4.2 main this PR
aroundAll, from any module 1 no aroundAll export exit 1 exit 0, no hook runs
beforeAll/afterAll from "vitest" 2 exit 0, every hook runs exit 1 exit 1, not changed
test.beforeAll/test.afterAll of a test.extend() test 3 no test.extend exit 1 exit 1, not changed
  • B, this PR, for aroundAll only. No beforeAll/afterAll runs inside an aroundAll that does not run. docs/test/lifecycle.mdx:145 stays as written and the runs cell of docs/test/writing-tests.mdx:508 gets "unless its aroundAll doesn't".
  • A. The aroundAll runs when a bun:test hook inside it runs. writing-tests.mdx:508 stays as written and lifecycle.mdx:145 gets the exception. The new tests stay, with the expected logs changed.
  • C. Keep main and add one docs sentence. On main, each of these changes to repro 1 gives the output of this PR: move the db.query() call into the aroundAll, register it as a second aroundAll, or import beforeAll from "vitest".
  • Rows 2 and 3. If the answer is B for all three rows, the gate gets one more condition (a scope with any left-out hook queues none of its suite hooks) and the per-hook filter in generate_all_order goes away. I will add that to this PR. For row 2, A gives the output of 1.4.2 back and B runs no hook.

Why now

  • An automated fuzz run found this. There is no issue and no user report.
  • aroundAll is in no stable release: git tag --contains 9e76acc6fb prints nothing. It is on main since 2026-10-31, so canary builds have it.
  • Without an answer, main's behaviour ships in the first stable release that has aroundAll. After that release, A and B both change released behaviour.
  • The rule that B bends was kept on purpose for a block with no aroundAll: bun:test: hooks of a skipped describe are not tests #44832 says "Not changed: a describe whose tests are all test.skip still runs its beforeAll / afterAll (bun:test: skip beforeAll/afterAll for describe blocks with no non-skipped tests #35497)". The test at vitest-flavor.test.ts:763 pins it and still passes.

Repros

Repro 1. main: (fail) D > (unnamed), (pass) T, exit 1. This PR: (skip) D > S, (pass) T, exit 0.

import { test, describe, beforeAll, aroundAll } from "bun:test";
let db;
describe("D", () => {
  aroundAll(async runSuite => { db = { query: () => 1 }; await runSuite(); db = undefined; });
  beforeAll(() => { db.query(); });
  test.skip("S", () => {});
});
test("T", () => {});

Repro 2. Exit 1 on main and on this PR, exit 0 on bun 1.4.2.

import { test, describe, beforeAll } from "bun:test";
import { beforeAll as vitestBeforeAll } from "vitest";
let db;
describe("D", () => {
  vitestBeforeAll(() => { db = { query: () => 1 }; });
  beforeAll(() => { db.query(); });
  test.skip("S", () => {});
});
test("T", () => {});

Repro 3. Exit 1 on main and on this PR.

import { test as base, describe, beforeAll } from "bun:test";
const test = base.extend({});
let db;
describe("D", () => {
  test.beforeAll(() => { db = { query: () => 1 }; });
  beforeAll(() => { db.query(); });
  test.skip("S", () => {});
});
base("T", () => {});

The same as repro 1 on main, and fixed here: hooks in a block nested below the aroundAll, a file-level aroundAll over a file of skipped tests, an afterAll alone, a "vitest" aroundAll over bun:test hooks, and an AsyncLocalStorage store that reads undefined in beforeAll.

What B changes besides the repro

  • A cleanup afterAll inside a left-out aroundAll no longer runs, as under -t, .only and describe.skip. Node made the same change for node:test (test_runner: do not invoke after hook when test is empty nodejs/node#51389, "do not invoke after hook when test is empty") and reverted it (Revert "test_runner: do not invoke after hook when test is empty" nodejs/node#51998).

  • A bun:test hook of an enclosing scope that has no aroundAll still runs. This file exits 0 on main and 1 here. With describe.skip("block", ...) and no aroundAll it exits 1 on bun 1.4.2 too.

    let server;
    afterAll(() => { server.stop(); });
    describe("block", () => {
      aroundAll(async runSuite => { await runSuite(); });
      beforeAll(() => { server = { stop() {} }; });
      test.skip("S", () => {});
    });
  • The gate also closes for a block with only test.todo and for a block with no test.

  • test.concurrent tests before and after a gated block share one concurrent group (5 queue groups become 1), as they do around a block with no hooks.

References, run here

  • Vitest 5.1.0 on Node 26.3.0: in a block whose tests are all skipped, no aroundAll, beforeAll or afterAll runs, nested blocks included. Repro 1 with the import changed to "vitest" passes.
  • Jest 30.5.2: no beforeAll/afterAll runs in a block or file whose tests are all skipped.
  • node:test on Node 26.3.0: before()/after() run in a suite whose tests are all skipped.

Measurements

Release builds of 2596ac5 and of this diff on it, from one build directory. The diff is the same after the rebase.

  • Ordering pass: +3.2 (empty describe), +7.0 (describe with beforeAll, afterAll and a skipped test) and +7.0 (the same with a test that runs) instructions per describe scope, once per file load. Main-thread user instructions, BUN_JSC_useJIT=0, (I(10,000 scopes) - I(5,000 scopes)) / 5,000, minimum of 9 interleaved runs. The spread between runs is about 6 per scope. valgrind is not installed in the container and could not be installed, so the counts come from the hardware counter (perf_event_open).
  • Release binary: .text 108,257,833 to 108,257,577 bytes. Order::enter_describe 2,085 to 2,090 bytes, BunTest::run 12,402 to 12,146 bytes, no other code symbol changed size (llvm-size -A, linker map). No struct field, allocation or queue entry is added.
  • Queue groups on debug builds (WANTS_LOUD=1), main to this PR: repro 1 3 to 2, the nested form 4 to 2, the file-level form 6 to 2, the "vitest" aroundAll over bun:test hooks 4 to 2. A file that imports every name from "vitest": 10 to 10.
  • Files with no aroundAll: the whole WANTS_LOUD=1 log (tree, queue and run, with addresses and times removed) is the same on both builds for the bun:test fixture of the test at vitest-flavor.test.ts:763 (180 lines), for a node:test file with before()/after() in skipped, todo and empty suites (566 lines, 11 of 11 hook lines), and for a "vitest" file (428 lines).

Other open PRs

Where the tests are

In test-test.test.ts, beside hooks of a describe block that does not run. That block and the test after it hold the bun:test rule for a block that runs no test. The other aroundAll tests are in vitest-flavor.test.ts. That file starts about 200 bun test children, and on a debug build some of its tests reach the timeout of 5 seconds when the whole file runs, so it cannot show that a change passes.

Suites run

On a debug build of this PR: vitest-flavor.test.ts, test-test.test.ts, describe.test.ts, jest-hooks.test.ts, bun_test.test.ts, failure-skip.test.ts, nested-describes.test.ts, only-inside-only.test.ts, test-only.test.ts, test-on-test-finished.test.ts, concurrent.test.ts, vi-utils.test.ts, require-bun-test.test.ts, test-filter-lifecycle-snapshot.test.ts, node-test.test.ts, and test/integration/bun-types/bun-types.test.ts for the JSDoc line. No expectation of an existing test changes. The 7 new tests also pass when their child runs get --aot (the preload of test/aot/runner-suites). In vitest-flavor.test.ts and vitest-apis.test.ts, some tests reach that timeout when the whole file runs on the debug build in this container, on main too. Each passes when it runs alone.


no test proof · iteration 5 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/test/test-test.test.ts

@robobun

robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:00 PM PT - Nov 10th, 2026

❌ @robobun, your commit 23b684b has 1 failures in Build #139806 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 48455

That installs a local version of the PR into your bun-48455 executable, so you can run:

bun-48455 --bun

@robobun

robobun commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: draft. It waits for a maintainer to pick the direction (A, B or C in the description).

How I reproduced it: I ran the file under "Repro 1" in the description with bun test ./a.test.ts on a debug build of main. It fails 3 of 3 runs with TypeError: undefined is not an object (evaluating 'db.query'), exit 1. On this branch it prints (skip) D > S, (pass) T and exits 0. Of the 7 new tests in test/js/bun/test/test-test.test.ts, 6 fail on main and all pass here.

CI: no failure touches this diff. Build 139806 (the current head) finished with 187 of 188 jobs passed. The one failed job is a debian 13 x64 test shard, red on test/aot/heap-limit.test.ts: a compiled program ends with a segmentation fault when its address space is used up. This PR does not change that code, and that shard ran neither the new tests nor their --aot re-run.

PR: #48455

A scope that runs no test leaves out its aroundAll hooks. The "bun:test"
beforeAll and afterAll hooks of that scope, and of the scopes nested in
it, still ran. They ran without the set-up and teardown of the
aroundAll, and outside the async context of runSuite().

Order::enter_describe now decides once for a scope. A scope that runs no
test and has an aroundAll queues none of its aroundAll, beforeAll and
afterAll hooks, and the scopes nested in it inherit that. A file with no
aroundAll queues what it queued before.
They now sit beside the tests for the hooks of a describe block that
does not run, which hold the "bun:test" rule for a block that runs no
test. vitest-flavor.test.ts is as it is on main again.
@robobun
robobun force-pushed the robobun/eed2d49d/around-all-gates-its-hooks branch from 4e873ca to 23b684b Compare November 10, 2026 15:31

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants