Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/test/lifecycle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,8 @@ To avoid typing `--preload` every time you run tests, add it to your `bunfig.tom
preload = ["./setup.ts"]
```

To fail the run from a global hook, call `reportError(err)` in the hook. `bun test` counts an error and exits with `1`, and the hooks that follow still run. With `--parallel` workers, `bun test` ignores a `process.exitCode` that your setup script sets. See [Exit Codes](/test/runtime-behavior#exit-codes).

## Practical Examples

### Database Setup
Expand Down
2 changes: 2 additions & 0 deletions docs/test/parallel.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ The coordinator merges coverage, JUnit XML and snapshot writes, so `--parallel -

If a worker crashes (a test calls `process.exit`), the coordinator reports the file that worker was running as failed, and a replacement worker picks up the remaining files. A crash from a fatal signal aborts the whole run, so later passing files can't mask it. With a single effective worker (`--parallel=1`, or a suite with one test file), `bun test` runs the files in the main process, so `process.exit` ends the run with that exit code.

With workers, `bun test` ignores a `process.exitCode` that a test file or a `--preload` script sets. See [Exit Codes](/test/runtime-behavior#exit-codes).

### When `--parallel` helps, and when it doesn't

`--parallel` pays off when the suite is dominated by test execution — I/O waits, real computation, subprocesses, many files. It costs something too: every file re-evaluates its imports in a fresh global (see [`--isolate`](#--isolate)), and each worker is a separate process with its own JIT warm-up. For a suite of very fast files that all import the same large module graph, plain `bun test` (one process, one shared module registry) can be faster. Try both; Bun prints the numbers at the end of every run.
Expand Down
34 changes: 34 additions & 0 deletions docs/test/runtime-behavior.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,40 @@ import { test, it, describe, expect, beforeAll, beforeEach, afterAll, afterEach,
- `0`: All tests passed, no unhandled errors
- `1`: Test failures or unhandled errors occurred

The test results decide the exit code. `bun test` ignores a `process.exitCode` that test code sets, so code under test can store an exit code and a test can read the code back:

```ts title="cli.test.ts" icon="/icons/typescript.svg"
import { test, expect } from "bun:test";

// Code under test stores an exit code, as a CLI does for a usage error.
function main(argv: string[]) {
if (argv.includes("--unknown")) process.exitCode = 1;
}

test("an unknown flag sets the exit code", () => {
main(["--unknown"]);
expect(process.exitCode).toBe(1);
});
```

This run exits with `0`. Because `process.exitCode` is still `1` when the run ends, `bun test` prints a note after the summary:

```txt
note: process.exitCode is 1 at the end of the run. bun test ignores it: no test failed, so the exit code is 0. To fail the run, call reportError(err) or throw an error.
```

To silence the note, set `process.exitCode` back to `0`, for example in an `afterEach` hook. `bun test` prints the note only when one process runs all the files and then exits: a run with [`--parallel`](/test/parallel) workers prints no note, and neither does `--watch`.

To fail a run on purpose:

- Call `reportError(err)`. In a hook, `bun test` counts an error, no test fails, and the hooks that follow still run. In a test, that test fails.
- Throw an error. In a test, that test fails. In a hook, `bun test` reports a failure. A throwing `afterAll` hook also stops the `afterAll` hooks that follow it in the same scope.

`bun test` does not ignore a `process.exitCode` in two cases:

- A [`--preload` script](/test/lifecycle#global-setup-and-teardown) sets the code while the script loads, or in a `beforeAll` or `afterAll` hook that the script registers. A passing run exits with that code, unless `--parallel` workers run the files or a test file sets `process.exitCode` to `0` afterwards. A preload's `beforeEach` and `afterEach` hooks count as test code.
- A file of the run, or a preload, calls a `node:test` API: it registers a test, a suite or a hook, or uses `mock`. An import alone does not count. `process.exitCode` is then the exit code of a passing run, as in Node.js, unless `--parallel` workers run the files.

### Signal Handling

The test runner handles common signals:
Expand Down
13 changes: 11 additions & 2 deletions src/jsc/bindings/BunProcess.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,7 @@ JSC_DECLARE_HOST_FUNCTION(Process_functionCwd);

extern "C" uint8_t Bun__getExitCode(void*);
extern "C" void Bun__setExitCode(void*, uint8_t);
extern "C" void Bun__Jest__didStoreExitCode(void*, uint8_t);
extern "C" void Bun__closeChildIPC(JSGlobalObject*);

extern "C" bool Bun__GlobalObject__connectedIPC(JSGlobalObject*);
Expand Down Expand Up @@ -2513,6 +2514,15 @@ JSC_DEFINE_CUSTOM_GETTER(processExitCode, (JSC::JSGlobalObject * lexicalGlobalOb
return JSValue::encode(jsNumber(Bun__getExitCode(process->globalObject()->bunVM())));
}

// Every store of `process.exitCode` goes through here: `bun test` records which code stored it.
static void storeExitCode(Process* process, uint8_t exitCode)
{
void* bunVM = process->globalObject()->bunVM();
Bun__setExitCode(bunVM, exitCode);
if (isBunTest) [[unlikely]]
Bun__Jest__didStoreExitCode(bunVM, exitCode);
}

bool setProcessExitCodeInner(JSC::JSGlobalObject* lexicalGlobalObject, Process* process, JSValue code)
{
auto throwScope = DECLARE_THROW_SCOPE(process->vm());
Expand All @@ -2534,8 +2544,7 @@ bool setProcessExitCodeInner(JSC::JSGlobalObject* lexicalGlobalObject, Process*
RETURN_IF_EXCEPTION(throwScope, false);

process->m_isExitCodeObservable = true;
void* ptr = process->globalObject()->bunVM();
Bun__setExitCode(ptr, static_cast<uint8_t>(exitCodeInt % 256));
storeExitCode(process, static_cast<uint8_t>(exitCodeInt % 256));
}
return true;
}
Expand Down
28 changes: 28 additions & 0 deletions src/runtime/cli/test_command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -936,6 +936,12 @@ pub(crate) fn skip_exit_listeners(reporter: &CommandLineReporter) -> bool {
!(reporter.jest.node_test_used || should_drain_event_loop())
}

/// Whether `process.exitCode` is the exit code of a passing run, as in a node process: any file of the run
/// called a `node:test` API, or `BUN_TEST_DRAIN_EVENT_LOOP=1`.
fn run_has_node_exit_semantics(reporter: &CommandLineReporter) -> bool {
reporter.jest.node_test_used || should_drain_event_loop()
}

/// `ExitHandler::requested` at the end of a run, which does not wait for the event loop to run dry.
pub(crate) fn exit_is_requested() -> bool {
!should_drain_event_loop()
Expand Down Expand Up @@ -1869,6 +1875,7 @@ impl TestCommand {
unhandled_errors_between_tests: 0,
summary: Summary::default(),
node_test_used: false,
preload_exit_code: core::cell::Cell::new(0),
},
repeat_count: 1,
last_printed_dot: core::cell::Cell::new(false),
Expand Down Expand Up @@ -2675,6 +2682,27 @@ impl TestCommand {
|| reporter.jest.unhandled_errors_between_tests > 0
{
vm.exit_handler.exit_code = 1;
} else {
// The `process.exitCode` that is left does not decide a passing run, as in vitest (vitest-dev/vitest#3023)
// and in jest with workers (jestjs/jest#14501). The one that the setup or teardown of the run stored does.
let left = vm.exit_handler.exit_code;
let kept = reporter.jest.preload_exit_code.get();
if left != 0 && left != kept && !run_has_node_exit_semantics(&reporter) {
if kept == 0 {
bun_core::note!(
"process.exitCode is {} at the end of the run. bun test ignores it: no test failed, so the exit code is 0. To fail the run, call reportError(err) or throw an error.",
left,
);
} else {
bun_core::note!(
"process.exitCode is {} at the end of the run. bun test ignores it: no test failed, so the exit code is {}, which a preload script set.",
left,
kept,
);
}
Output::flush();
vm.exit_handler.exit_code = kept;
}
}
vm.exit_handler.skip_exit_listeners = skip_exit_listeners(&reporter);
vm.exit_handler.requested = exit_is_requested();
Expand Down
16 changes: 16 additions & 0 deletions src/runtime/test_runner/bun_test.rs
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,22 @@ impl BunTest {
}
}

/// Whether the running entry is a `beforeAll`/`afterAll` hook that a `--preload` registered.
/// `Order::generate_all_order` gives each such hook a group of its own: one sequence, no test.
pub(crate) fn is_running_preload_all_hook(&self) -> bool {
let Some(active_group) = self.execution.active_group_ref() else {
return false;
};
let [sequence] = active_group.sequences(&self.execution) else {
return false;
};
let (None, Some(active_entry)) = (sequence.test_entry, sequence.active_entry) else {
return false;
};
// SAFETY: arena-owned entry, alive for the lifetime of BunTest which owns Execution
unsafe { active_entry.as_ref() }.added_in_phase == AddedInPhase::Preload
}

pub(crate) fn ref_(this_strong: &BunTestPtr, phase: RefDataValue) -> RefPtr<RefData> {
let _g = group_begin!();
bun_core::scoped_log!(bun_test_group, "ref: {}", phase);
Expand Down
30 changes: 30 additions & 0 deletions src/runtime/test_runner/jest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,10 @@ pub(crate) struct TestRunner<'a> {
/// Set once any `node:test` registration API is called; gates `process.on('exit')` dispatch at the end of the run.
pub(crate) node_test_used: bool,

/// The `process.exitCode` a passing run exits with: the last one that a `--preload` stored while it loaded
/// or in a `beforeAll`/`afterAll` hook it registered. Any other store of 0 clears it.
pub(crate) preload_exit_code: core::cell::Cell<u8>,

pub(crate) bun_test_root: bun_test::BunTestRoot,
}

Expand Down Expand Up @@ -515,6 +519,32 @@ pub(crate) fn file_generation() -> u32 {
})
}

/// `BunProcess.cpp` calls this under `bun test` after every store of `process.exitCode`.
#[unsafe(no_mangle)]
pub(crate) extern "C" fn Bun__Jest__didStoreExitCode(vm: &VirtualMachine, code: u8) {
// Not `worker_ref()`: a macro of an imported module runs in a VM of its own on a transpiler or bundler thread,
// with no Worker. That VM, a Worker's and the debugger's each have their own exit code, and RUNNER is the main thread's.
if !vm.is_main_thread {
return;
}
// `runner_ptr()`, not `runner()`: the caller is script that a `&mut TestRunner` borrower is running.
let Some(runner) = Jest::runner_ptr() else {
return;
};
// SAFETY: the check above leaves the one thread that reads and writes RUNNER.
let runner = unsafe { runner.as_ref() };
// The setup and teardown of the run, as docs/test/lifecycle.mdx names them.
let is_global_setup_or_teardown = vm.is_in_preload
|| runner
.bun_test_root
.active_file
.as_deref()
.is_some_and(|active_file| active_file.is_running_preload_all_hook());
if code == 0 || is_global_setup_or_teardown {
runner.preload_exit_code.set(code);
}
}

/// Reached only from `node:test` (`t.skip()` / `t.todo()` at runtime): overrides
/// the running sequence's result so bun:test reports skip/todo instead of pass.
/// `done`'s bound `DoneCallback.r#ref.phase` names the intended sequence so a
Expand Down
Loading
Loading