Skip to content

test(aot): compile less of Bun's library in workers-from-texts-that-the-build-knows.test.ts and state what each command prints - #47536

Closed
robobun wants to merge 1 commit into
mainfrom
robobun/ecbd0602/faster-workers-from-texts-test
Closed

robobun wants to merge 1 commit into
mainfrom
robobun/ecbd0602/faster-workers-from-texts-test

Conversation

@robobun

@robobun robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Behaviour change: none

Problem

  • test/aot/workers-from-texts-that-the-build-knows.test.ts takes 64 s on darwin x64 (build 135961). Builds take 98% of its CPU time.
  • Six programs print console.log("message", event.data). For a value of unknown type the build compiles Bun's value formatter: 436 functions, not 66.
  • Three programs differ in one expression and have a build each.

Fix

  • The programs hand console.log one text and print the same lines.
  • The three programs are one. Its first argument selects the Worker. 15 builds and 11 new executables, was 17 and 13.
  • Runs state stdout, stderr, exit code and signal. Builds state their warnings by code and place. A stop at the interpreter names its Worker.
  • Verified: bun bd test test/aot/workers-from-texts-that-the-build-knows.test.ts takes 117 to 164 s, was 142 to 271 s. CI, beside the same files: 12% to 39% less time on 10 lanes. darwin x64 ran the file alone (14.5 s). macOS is not measured locally.

Background

  • A test builds an executable with bun build --compile --aot and starts it. The build compiles the library code that the program reaches.
  • BUN_JSC_crashIfInterpreterIsUsed=1 ends a process where it interprets code.
  • Considered one build for the three left-out texts: one text changes what the build does for another. Considered --aot-separate-programs: canary builds only.

Downsides

  • Six builds no longer compile the value formatter. Two programs here and test/aot/parallel/workers do.
  • A build error in the shared program fails three tests.
  • The node:worker_threads build is unchanged: a third of the CPU time now. 12 local vCPUs gain nothing (7.6 s, 7.4 s).
Notes

Where the time goes. Release build (canary 34cbc70fa), Linux x64. Each build alone, one at a time, best of 3. CPU is user plus system time. The functions and the engine runs are from --aot-report.

Tests Builds before CPU before Builds after CPU after Functions Engine runs
a program that starts no Worker 1 0.2 s 1 0.2 s 1 1
a program that imports node:worker_threads 1 8.7 s 1 8.7 s 2411 1
3 ways that the program makes at run time 3 4.2 s 1 1.4 s 442, now 446 2
a relative path, a package 1 0.7 s 1 0.7 s 75 2
a package of the workspace 1 1.1 s 1 1.1 s 77 2
a listed entry point 2 1.5 s 2 1.5 s 75 2
names, packages, files above 1 1.1 s 1 1.2 s 84 8
a file that is left out 1 1.3 s 1 1.3 s 74 3
3 texts that are left out 3 8.5 s 3 3.8 s 436, now 66 4, 4, 3
TypeScript that names the globals 1 4.4 s 1 2.8 s 703, now 316 7
2 type errors 2 5.4 s 2 2.2 s 436, now 65 (with the error removed) no report
total 17 37.0 s 15 24.9 s
  • In a run of the whole file on 12 vCPUs the commands take 54.1 s of CPU time, and the builds 53.2 s of that. The two runs of plain Bun and the 18 runs of executables take the rest.
  • The row of the package of the workspace is from that run. The other rows are builds of a copy of the folder that the test writes.

What console.log costs a build. A program of two lines that starts one Worker and prints in its onmessage:

The call Functions compiled CPU of the build
console.log("message " + event.data) 64 0.8 s
console.log(`message ${event.data}`) 64 0.8 s
console.log("message", "text") 64 1.0 s
console.log(1 + 1) 64 0.8 s
console.log(event.data) 435 1.7 s
console.log("message", event.data) 435 1.7 s

132 of the 371 functions more are in internal:util/inspect. The rest is what it reaches: iterator helpers, Array.prototype, the sort, node:url. Buffer.from(text).toString() brings the same code, so the shared program still compiles 446 functions. #47129 took node:worker_threads out of the programs of this file that are not about it, for the same reason.

All local times. Linux x64, 12 vCPUs. The host takes a changing part of the CPU time: /proc/stat counts 19 s of steal while 12 builds run at once for 2.6 s. So the rows with taskset are the stable ones. Main's file and this PR's ran in turns.

Bun CPUs Before After
debug with ASAN (bun bd test <file>) 12 142.3, 161.6, 200.1, 255.3, 271.2 s 116.8, 118.0, 159.8, 164.4 s
debug, CPU time of all processes 12 1022 to 1238 s, median 1092 s 700 to 772 s, median 738 s
release 4 (taskset -c 0-3) 9.9 to 11.5 s, median 10.9 s (15 runs) 6.5 to 9.5 s, median 7.4 s (25 runs)
release 2 (taskset -c 0-1) 16.4 to 17.2 s, median 16.6 s (4 runs) 10.9 to 11.5 s, median 11.0 s (4 runs)
release 12 5.3 to 12.9 s, median 7.6 s (15 runs) 5.0 to 9.6 s, median 7.4 s (15 runs)
release, CPU time of all processes 4 median 35.5 s median 24.1 s
  • The three debug pairs that ran one after the other: 161.6 s then 118.0 s, 200.1 s then 159.8 s, 142.3 s then 116.8 s. The runs of 271.2 s and 255.3 s were the first two of the session. The second of them started the debug binary directly, with a preload that records each command.
  • Release is bun-profile of bun run build:release with the variables that scripts/runner.node.ts sets: BUN_GARBAGE_COLLECTOR_LEVEL=1, BUN_JSC_randomIntegrityAuditRate=1.0, BUN_FEATURE_FLAG_NO_ORPHANS=1.
  • The Linux lanes of CI have 4 vCPUs and run two files that build executables side by side. The row with 2 CPUs stands for that: 16.6 s here, 16 s on debian 13 x64 in build 135961.
  • On 12 vCPUs the file ends when the node:worker_threads build ends. That build takes 3.1 to 4.4 s alone and longer beside the other builds.
  • Every run above has 16 pass. Main's file has 51 expect() calls and this PR's has 59.

macOS. This machine is Linux x64, so no macOS time is measured locally. The table of CI below has both macOS lanes. In build 135961 the file ran on a darwin x64 agent beside three other files that build executables. #47463 and #47468 measured that the first start of a new executable waits 1 to 3 s on such agents. This file starts 11 new executables now, was 13.

CI, like for like. The runner puts a test file that a PR changes in front and runs it alone (modifiedSerialTests in scripts/runner.node.ts). In build 136054 it could not do that on 10 of the 11 lanes: there the GitHub API answered 403 (rate limit) to the request for the files that this PR changes. So on those lanes this file ran in its usual group, beside other files that build executables. Build 135961 is a build of another PR, with main's file. In both builds the file is in the same shard, in the same group of files. Seconds for this file, one run each:

Lane Group Build 135961, main's file Build 136054, this PR
debian 13 x64 5 files, 2 at a time 16.32 11.77
debian 13 aarch64 5 files, 2 at a time 12.80 8.85
ubuntu 25.04 x64 5 files, 2 at a time 16.31 13.05
ubuntu 25.04 aarch64 5 files, 2 at a time 10.32 8.66
alpine 3.23 x64 6 files, 2 at a time 14.95 10.60
alpine 3.23 aarch64 6 files, 2 at a time 10.76 6.59
debian 13 x64-asan 7 files, 3 at a time 37.13 28.63
windows 2019 x64 5 files, 2 at a time 20.93 18.47
windows 11 aarch64 12 files, 2 at a time 13.47 11.47
darwin aarch64 14 files, 3 at a time 4.83 3.37
darwin x64 14 files, 4 at a time in build 135961 64.42 14.45, alone and in front

The shared program. Its second line starts a Worker from a text that the build knows, as before. Then ways[process.argv[2]]() starts one of four Workers:

  • Three hand over what Buffer.from(text).toString() makes at run time: a data: URL, the text of a Blob, the type of a Blob. Each run prints message 1 or message 2. With the interpreter forbidden each stops with crashIfInterpreterIsUsed: Worker <URL of that Worker> and prints nothing.
  • The fourth is the text of the second line again. The build of the shared program runs it once with the interpreter forbidden, and it must print message 1. So the executable holds that text, and a run that stops has stopped at the Worker of its own way.
  • The build runs the engine twice and compiles 446 functions. Each of the three programs alone ran it twice and compiled 442.

Why the three texts that are left out keep a build each. A build with one such text runs the engine 4 times for a syntax error and 3 times for an import that is not found. A build with all three texts in one program runs it 3 times. So one text changes what the build does for another, and a shared build would not test what each build alone does.

Assertions, before and after.

  • A run of plain Bun (2 places). Before: one test looked for two lines, the other for the lines that end in : error. After: every line, an empty stderr, exit code 0, no signal.
  • A run of an executable that must end by itself (15 runs). Before: stdout and the exit code, and for three of them message 1 or message 2 for any of the three. After: stdout, an empty stderr, exit code 0 and no signal, and the one message of each way.
  • A run that must stop at the interpreter (3 places). Before: stderr contains crashIfInterpreterIsUsed. After: the line names the Worker (data:text/javascript,postMessage(1) or a blob: URL), stdout is empty, and the exit code is not 0.
  • A build that leaves a file or a text out (4 places). Before: stderr contains warn: <code> and warn: TS100072. After: the first warning has the code and the line and column in the file or text. The TS100072 warning names the line and column of the new Worker that starts from it. Every other warning is about that text.
  • A build that must say nothing (9 places). Before: no line that begins with error. After: also no warning with a code.
  • A build that a type error stops (2 places). Before: stderr contains error: TS2322 and no TS100072, exit code 1. After: the one message is TS2322 at line 1, column 5, in worker.ts for the file. Exit code 1, no signal, and no executable is written. For the text only :1:5 is compared: the build names the text /$bun/plugin/entrytext/data%3atext/javascript%2c... there.
  • The listed entry point. Before: Bun.file(path).size of both executables is equal. That is 0 for a path that does not exist, and on Windows the files are program.exe and listed.exe. After: statSync of the file with .exe on Windows, and both builds report the same number of engine runs.
  • No inline snapshot: the tests of this folder state what they expect in toEqual, and most of these are rows of test.each.

The assertions fail when they must. Each line is one change to this PR's file, run with the release build:

  • inner.js:1:2 for inner.js:1:1 in the expected warning: the test of the file that is left out fails.
  • 1:13 for 1:14 as the place of the syntax error: its test fails.
  • new Blob(["postMessage(2)"]) without made(): the build knows the text, the run does not stop, and the test fails at the missing crashIfInterpreterIsUsed line.
  • The fourth way with a text that is made at run time: the check in the shared build fails, and the three tests with it.
  • worker.ts without its type error: the test fails on exit code 0, an empty list of messages and an executable that was written.

Not done.

  • The node:worker_threads program. A program of one line, import { isMainThread } from "node:worker_threads", compiles 2338 functions and takes 8 s of CPU time to build. The test asserts that the engine runs once for it, so the build cannot be smaller.
  • made() with TextEncoder and TextDecoder: a program of one way then compiles 66 functions, not 442. It was not taken: Buffer.from(text).toString() is how the file makes a text that the build cannot know.
  • More commands at once inside a test. On 2 and 4 CPUs the file is bound by CPU time, and on 12 by one build.

Seen in bun, not changed here.

  • A build that leaves a Worker text out runs the engine twice for what it keeps. The program with the sound text and the syntax error takes 4 runs (EngineRuns in --aot-report). The same program without the bad Worker takes 2. The three builds of that kind are 3.8 s of the 24.9 s.
  • An error in a text names the text by an internal path (/$bun/plugin/entrytext/data%3atext/javascript%2c...:1:5). A warning about a text names it by its URL (data:text/javascript,postMessage((:1:14).

Also run.

  • 30 runs in a row of this PR's file pass with the release build (20 on 4 CPUs, 10 on 12).
  • The debug build with the variables of the ASAN lane (BUN_JSC_validateExceptionChecks=1, BUN_DESTRUCT_VM_ON_EXIT=1, ASAN_OPTIONS, LSAN_OPTIONS): 16 pass in 128.8 s.
  • In a bucket: bun test --parallel=3 --max-concurrency=2 with a-worker-in-a-loop-is-terminated-under-any-inherited-signal-mask.test.ts and a-program-that-starts-itself-as-a-worker-ends-every-time.test.ts passes.
  • With one slot (AOT_PROGRAMS_JOBS=1) and with -t for one of the three tests of the shared program: pass, and the folder of the shared program is removed.
  • test/aot/test-files-self-skip.test.ts, test/aot/every-test-runs-in-ci.test.ts and test/internal/source-lints pass.
  • bun fmt --check, oxlint and tsc report nothing for the file. tsc reports the same four errors in test/aot/supported.ts as on main.
  • AOT: bun test --aot on large suites, module loading fixes, fuzzer fixes #47316 changes two other places of this file (one start expression and three more lines of the test of names). Whichever merges second has a conflict to resolve by hand.

…he-build-knows.test.ts and state what each command prints

The programs of this file hand console.log one text. A value of unknown
type made six builds compile the code that formats any value: 436
functions of Bun's library, not 66.

The three ways to start a Worker from what the program makes at run time
are in one program. The first argument of a run selects the way. That
program is built once. A fourth way starts a Worker from the text that
the build knows, to show that the executable holds that text.

Each run of plain Bun and of an executable states its stdout, an empty
stderr, the exit code and the signal. A build states its warnings and
errors by code and place. A run that must stop at the interpreter names
the Worker that it stops at.
@robobun

robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:37 AM PT - Oct 31st, 2026

❌ @robobun, your commit 7115b59 has 2 failures in Build #136054 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 47536

That installs a local version of the PR into your bun-47536 executable, so you can run:

bun-47536 --bun

@robobun

robobun commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: closed without a merge. #47942 rewrites the front end of the ahead-of-time compiler and its tests, so this change no longer applies.

The measurements of this pull request are in its description. They are from main at 34cbc70.

@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 34cbc70 and 7115b59.

Files selected for processing: 1
  • test/aot/workers-from-texts-that-the-build-knows.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 1 remain after this review.

Walkthrough

The PR updates AOT Worker tests. It adds shared build and diagnostic checks, and strengthens assertions for runtime-created Workers, path resolution, executable output, and build failures.

Changes

AOT Worker test coverage

Layer / File(s) Summary
Build helper and executable checks
test/aot/workers-from-texts-that-the-build-knows.test.ts
The shared build helper accepts entry-point arrays and executable names. Tests check build diagnostics, executable output, and compiled program counts.
Runtime-created Worker texts
test/aot/workers-from-texts-that-the-build-knows.test.ts
Tests check runtime-created Worker messages and interpreter-forbidden failures. A Worker text known to the build remains executable when the interpreter is forbidden.
Worker path resolution
test/aot/workers-from-texts-that-the-build-knows.test.ts
Tests compare exact output for package, workspace, relative, normalized, scoped-package, and missing Worker paths.
Worker diagnostics and build failures
test/aot/workers-from-texts-that-the-build-knows.test.ts
Tests assert diagnostic codes and locations for failed imports and invalid data URLs. Type errors must fail the build without writing an executable.

Possibly related issues

  • oven-sh/bun#47237: The tests cover compiled Worker path resolution, but the summary does not establish that they address the issue’s subfolder case.
  • oven-sh/bun#40903: The tests exercise embedded Worker loading in AOT builds, but the issue concerns --bytecode, and coverage of its failing path is not established.

Suggested reviewers: jarred-sumner

Merge Risk: Minimal · up to 7115b

The changes consolidate AOT Worker test builds and strengthen output and diagnostic assertions. No merge-blocking issue is identified; merge after normal checks pass.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly identifies the main changes: reducing AOT compilation work and improving command-output assertions in the specified test.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template, which has two sections: "What does this PR do?" and "How did you verify your code works?". The Problem, Fix, and Background subsections explain the pur…
Full details: Description check

Explanation

The description follows the repository template, which has two sections: "What does this PR do?" and "How did you verify your code works?". The Problem, Fix, and Background subsections explain the purpose of the change, and the Fix section and the Notes block include measured build counts, CPU times, and test run results, so the verification section is filled in. The description is mostly complete and states the intended behavior change (none) and the test results. Gaps: the description is very long and dense, and it does not state the macOS timing (the author says it was not measured locally, and the CI table is the only macOS evidence). The description also refers to unrelated PR numbers (#47129, #47316, #47463, #47468) without a clear statement of how they relate to this change, and it lacks a short plain summary at the top. These gaps are non-critical, so the check passes.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Since the exact stdout listings and warning ordering were only verified on Linux per the description, and dropping the value formatter from six fixtures is a deliberate coverage trade-off, a human look from an aot test maintainer would still be worthwhile.

What was reviewed:

  • The shared builtWithEveryWay() memo: its slot() is released before each of the three tests takes its own, so no slot deadlock; a failed build rejects all three tests (as the description states) and afterAll disposes the folder.
  • said()/placeOf(): the warn:/error: TSnnnn ... \n at <place> shape matches what sibling aot tests already parse; . does not match \r so CRLF lines still match; data: URLs are kept whole, file paths reduced with basename.
  • Every replaced assertion is strictly tighter than base (exact {stdout, stderr, exitCode, signal}, exact warning lists, .exe-aware size comparison that was vacuous on Windows before, existsSync(file) false on a failed build).
  • The crash line crashIfInterpreterIsUsed: Worker <specifier> is what src/jsc/bindings/BunAOTRuntime.cpp prints, so the per-way regexes can match.
Extended reasoning...

The change rewrites one test file, test/aot/workers-from-texts-that-the-build-knows.test.ts (222 insertions, 117 deletions), with no production code touched and no security-sensitive surface. It merges three per-text builds into one memoized shared executable selected by argv, changes fixture console.log calls to single-string form so the AOT build compiles fewer library functions, and replaces partial matches with exact object comparisons via new helpers (ended, said, placeOf, builtSaying). The bug hunt ran dry with no findings and five candidates ruled out, and no CODEOWNER covers test/aot. Defer rather than approve because the restructuring is not mechanical, the exact-output and warning-order assertions were verified only on Linux by the author, and the reduced fixture coverage of the value formatter is a judgment call for a maintainer.

@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

On the two points of the review:

Platforms other than Linux. Build 136054 runs this file on every lane. It passes on the seven lanes that have finished it: debian 13 x64 and aarch64, ubuntu 25.04 x64 and aarch64, alpine 3.23 x64 and aarch64, and debian 13 x64-asan. The Windows and macOS shards have not reached the file yet. Their results go into the description when they finish.

The value formatter. Two programs of this file still make the build compile it. The program that imports node:worker_threads reaches it through that import, and it is only built. The shared program reaches it through Buffer.from(text).toString(), and each of its seven runs starts a Worker. The programs under test/aot/parallel/workers hand console.log values of unknown type in a main program that starts Workers (how-a-worker-of-text-fails.js is one), and CI runs that folder on every lane.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

Closing: #47942 rewrites the front end of the ahead-of-time compiler and its tests for the engine's new compiler (oven-sh/WebKit#997, merged), so this no longer applies and would conflict with it. If the problem still shows once that is in, please open a new pull request against main.

@robobun

robobun commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator Author

Understood. This pull request stays closed.

Two findings here do not depend on the diff, for whoever looks at this file after #47942. I measured both on main at 34cbc70, so the numbers can differ with the new compiler.

  • A program that hands console.log a value of unknown type made its build compile the value formatter: 436 functions, not 66. Six builds of this file did that.
  • The three programs that start a Worker from what the program makes at run time differ in one expression. One build can serve them, with the first argument of a run to select the Worker.

The description above has the measurements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants