test_runner: support test order randomization by pmarchini · Pull Request #61747 · nodejs/node

pmarchini · 2026-02-08T22:09:32Z

This PR adds deterministic randomization to the Node.js test runner at both levels:

test file execution order
the order of tests inside each file

--test-randomize enables randomized execution, and --test-random-seed=<seed> replays the same order deterministically (and implicitly enables randomization).
When no seed is provided, one is generated and printed in the summary so runs are easy to reproduce.

nodejs-github-bot · 2026-02-08T22:09:37Z

Review requested:

@nodejs/config
@nodejs/test_runner

codecov · 2026-02-08T23:37:49Z

Codecov Report

❌ Patch coverage is 91.90476% with 17 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.68%. Comparing base (4d0cb65) to head (85b2a9c).
⚠️ Report is 224 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/test_runner/utils.js	75.36%	17 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #61747      +/-   ##
==========================================
- Coverage   91.63%   89.68%   -1.95%     
==========================================
  Files         337      676     +339     
  Lines      140376   206783   +66407     
  Branches    21768    39589   +17821     
==========================================
+ Hits       128627   185449   +56822     
- Misses      11527    13469    +1942     
- Partials      222     7865    +7643

Files with missing lines	Coverage Δ
lib/internal/test_runner/runner.js	`93.64% <100.00%> (+0.76%)`	⬆️
lib/internal/test_runner/test.js	`96.91% <100.00%> (+0.16%)`	⬆️
src/node_options.cc	`76.44% <100.00%> (ø)`
src/node_options.h	`97.96% <100.00%> (ø)`
lib/internal/test_runner/utils.js	`63.43% <75.36%> (+1.41%)`	⬆️

... and 467 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

nodejs-github-bot · 2026-02-22T15:11:49Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71402/

JakobJingleheimer

🙌

I see I'm late to the party. None of my comments are pressing :)

JakobJingleheimer · 2026-02-23T08:07:14Z

+```
+
+Using suite-style APIs such as `describe()`/`it()` or `suite()`/`test()`
+still allows randomization, because sibling tests are queued together.


nit: I thiiink it should be "enqueued"? "queued" together means they wait together; enqueued together means they're added to a queue one-by-one setting up a sequence.

JakobJingleheimer · 2026-02-23T08:11:45Z

@@ -191,6 +198,12 @@ function getRunArgs(path, { forceExit,
  if (rerunFailuresFilePath) {
    ArrayPrototypePush(runArgs, `--test-rerun-failures=${rerunFailuresFilePath}`);
  }
+  if (randomize === true) {


Related to the skip and todo discussion: should this check truthiness rather than strict?

JakobJingleheimer · 2026-02-23T17:03:42Z

I couldn't find the actualCI failure, but Richard did: https://ci.nodejs.org/job/node-test-binary-windows-js-suites/38847/RUN_SUBSET=3,nodes=win11-arm64-COMPILED_BY-vs2022_clang-arm64/testReport/junit/(root)/parallel/test_domain_abort_on_uncaught/

The failure looks legit?

Ethan-Arrowood

great work!

nodejs-github-bot · 2026-03-19T01:25:31Z

CI: https://ci.nodejs.org/job/node-test-pull-request/71883/

nodejs-github-bot · 2026-03-26T20:39:56Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72046/

nodejs-github-bot · 2026-03-27T08:15:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72065/

nodejs-github-bot · 2026-03-27T20:34:50Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72100/

nodejs-github-bot · 2026-03-28T10:55:56Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72136/

nodejs-github-bot · 2026-03-30T16:11:35Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72309/

nodejs-github-bot · 2026-04-02T03:30:04Z

CI: https://ci.nodejs.org/job/node-test-pull-request/72402/

nodejs-github-bot · 2026-04-02T16:55:13Z

Landed in 5ff1eab

PR-URL: #61747 Reviewed-By: Jacob Smith <jacob@frende.me> Reviewed-By: Marco Ippolito <marcoippolito54@gmail.com> Reviewed-By: Ethan Arrowood <ethan@arrowood.dev> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Aviv Keller <me@aviv.sh> Reviewed-By: Chemi Atlow <chemi@atlow.co.il> Reviewed-By: Moshe Atlow <moshe@atlow.co.il>

Notable changes: buffer: * (SEMVER-MINOR) add `end` parameter (Robert Nagy) #62390 crypto: * (SEMVER-MINOR) accept key data in `crypto.diffieHellman()` and cleanup DH jobs (Filip Skokan) #62527 * (SEMVER-MINOR) implement `randomUUIDv7()` (nabeel378) #62553 debugger: * (SEMVER-MINOR) add edit-free runtime expression probes to `node inspect` (Joyee Cheung) #62713 fs: * (SEMVER-MINOR) add `signal` option to `fs.stat()` (Mert Can Altin) #57775 * (SEMVER-MINOR) expose frsize field in statfs (Jinho Jang) #62277 http: * (SEMVER-MINOR) harden `ClientRequest` options merge (Matteo Collina) #63082 * (SEMVER-MINOR) add `req.signal` to `IncomingMessage` (Akshat) #62541 lib,src,test,doc: * (SEMVER-MINOR) add `node:ffi` module (Colin Ihrig) #62072 process: * (SEMVER-MINOR) throw on `execve(2)` failure instead of aborting (Bryan English) #62878 src: * (SEMVER-MINOR) allow empty `--experimental-config-file` (Marco Ippolito) #61610 stream: * (SEMVER-MINOR) propagate destruction in `duplexPair` (Ahmed Elhor) #61098 test_runner: * (SEMVER-MINOR) align mock timeout api (sangwook) #62820 * (SEMVER-MINOR) add mock-timers support for `AbortSignal.timeout` (DeveloperViraj) #60751 * (SEMVER-MINOR) support test order randomization (Pietro Marchini) #61747 util: * (SEMVER-MINOR) colorize text with hex colors (Guilherme Araújo) #61556 PR-URL: #63137

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Feb 8, 2026

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch 2 times, most recently from 5556788 to e6f9a59 Compare February 8, 2026 22:20

pmarchini added test_runner Issues and PRs related to the test runner subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Feb 8, 2026

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch from e6f9a59 to b3b2388 Compare February 9, 2026 08:09

pmarchini marked this pull request as draft February 9, 2026 08:42

pmarchini force-pushed the test_runner/deterministic-test-randomizer branch from b3b2388 to 08db017 Compare February 9, 2026 09:19

pmarchini marked this pull request as ready for review February 9, 2026 09:19

pmarchini requested review from JakobJingleheimer, MoLow and atlowChemi February 9, 2026 11:19