Repository navigation
Conversation
…store The pin is the preview build of oven-sh/WebKit#941: main's pin 8c6c963e3c plus one commit. An import() under AsyncLocalStorage.run() lost the store when the call returned. What the load ran later read no store: Bun.plugin() onResolve and onLoad hooks of static dependencies, module bodies, CommonJS bodies, mock.module() factories and the then() of a namespace. With this pin a request of the module loader keeps the async context of its caller, and its jobs run under it where it is set. Tests: - test/js/bun/plugin/plugins.test.ts: the hooks, the bodies, Module.runMain(), import.defer(), a load that fails, a top-level await below another import(), enterWith() in a hook and in a CommonJS file, Bun.ModuleGraph, require() of an ES module, mock.module(), and the cells that a load allocates. - test/js/node/async_hooks/async-context/async-context-dynamic-import.js: runs on Bun and on Node.js. - test/bundler/bundler_compile_prelinked.test.ts: an import() of a chunk in a compiled executable. - test/aot/parallel/modules: seven programs that print the same as plain Bun and as compiled executables.
|
Updated 12:51 PM PT - Nov 10th, 2026
❌ @robobun, your commit a598799 has 1 failures in
🧪 To try this PR locally: bunx bun-pr 47371That installs a local version of the PR into your bun-47371 --bun |
|
Status. Reproduced with the script in the Notes of this pull request. On main, The branch has main at 89452d2. The pin is the head of oven-sh/WebKit#941, On a debug and ASAN build of a598799: the 44 new tests of |
…o async context With oven-sh/WebKit#941 a load keeps the async context of its caller for its jobs. The context of a graph is in the same slot as the AsyncLocalStorage stores, and what runs in it ends with the graph. The jobs of a load run the host's hooks and the modules of the realm's own loader, and neither is a graph's. JSModuleGraph sets the loader's filter for what a load keeps: none where the caller is in the context of a graph, so those jobs run as before. Tests: a synchronous load, a top-level await and a Bun.ModuleGraph under AsyncLocalStorage stores, in plugins.test.ts and as programs of test/aot.
…b59d8fc79 main pins a7b59e61de now. The preview is that commit plus the commits of oven-sh/WebKit#941.
main pins 4d45755fed now. 1a52269eca is that commit plus the commits of oven-sh/WebKit#941.
… its call in a compiled executable A program for test/aot/parallel that runs with the transpiler on its own threads. Without the commit of oven-sh/WebKit#941 that keeps the async context in TextThatWaits, five of its ten lines differ from plain Bun.
…f its call in a compiled executable A program for test/aot/parallel. With oven-sh/WebKit#974 the second import() waits behind the first load and is made again in a job of that load. Without the commit of oven-sh/WebKit#941 that keeps the async context in ImportBehindLoad, three of its seven lines print the store of the first import().
main has the rewritten ahead-of-time compiler (#47942) and pins WebKit 5a61071f79. 328b6c9bb4 is that commit with the commits of oven-sh/WebKit#941 merged in.
…llected, and a compiled test run The driver runs every reference at default flags now, where plain Bun reads the texts of modules on its own threads. - Seven programs have a reference that is a race between those threads: they say what the executable prints. - Two programs call import() while a require() loads an ES module. The rewritten run time runs that import() after the require(), as Node.js does, so they say what the executable prints too. - New: a require() of a graph that awaits, whose CommonJS file calls import() under a store. The module behind the await runs under no store. - New: six import() calls under a store each, whose CommonJS files queue a job. A collection takes the stores when the calls are through. - New: an import() in a compiled test run (bun test --aot) reads the store of its call. - One program no longer sets BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER, which the driver refuses.
test/aot/ci-lists.test.ts asks that every test file of test/aot is in test/expected-durations.json, and that a long one is in excludeFiles of test/parallel-allowlist.json.
… and evaluate() continues its caller The pin is oven-sh/WebKit#941 on the pin of main, with its last commit: the filter of what a load keeps is one pointer on the VM, and a host function that evaluates a module for its caller says so. - GlobalObject::resetAsyncContextAtEventLoop() is the statement that drainMicrotasks() has where a checkpoint begins. The host now also calls it where it begins a load with no script on the stack: a preload, the entry point, a test file, the built-in entry point, a Module.runMain that a preload replaced, a module of the bundler's server. What the script that ran last left in the slot with enterWith() is not the async context of that load. With the pin, the load would keep it for all of its jobs. - ModuleGraph.cpp sets VM::asyncContextOfLoadFilter. - vm.SourceTextModule#evaluate() continues its caller: the modules that wait there for a top-level await read what the caller reads. Tests: a preload that calls enterWith() (the entry point, a CommonJS entry point, a replaced Module.runMain, the next preload and its hooks, the entry point of a Worker, test files under three modes of bun test), an entry point that waits for a module that a preload began to import, node:vm, mock.module() for a linked module, a load outside any store, and three AOT programs.
The fixture gave the Worker the pathname of a file URL as its preload. On Windows that is /C:/..., which does not resolve, so the test of the Worker's preload failed there.
main has #48120, which pins WebKit at ee59053d32. WEBKIT_VERSION is now 37998c5702: oven-sh/WebKit#941 with that commit merged in. Conflicts: the pin, the end of plugins.test.ts (main's new tests, then the block of this branch), and the two lists of CI's runner, which take main's and get the test of this branch again (test/aot/ci-lists.test.ts).
main has #48436, which pins WebKit at 873ab3a426. WEBKIT_VERSION is now bacfec4b27: oven-sh/WebKit#941 with that commit merged in. Conflicts: the pin, and one list of CI's runner, which takes main's and gets the test of this branch again (test/aot/ci-lists.test.ts).
Fixes #32693
Draft until oven-sh/WebKit#941 merges:
WEBKIT_VERSIONis its head. Supersedes #37933.Problem
import()underAsyncLocalStorage.run()loses the store.Bun.plugin()hooks of static dependencies printonLoad(b):undefined. So do module bodies. Bun 1.3.13 and Node.js read the store.runInternalMicrotask,JSMicrotask.cpp).Fix
resetAsyncContextAtEventLoop(): the host resets the slot where it begins a load (entry point, preload, test file).plugins.test.ts(44 new, 24 fail on main), 20test/aotprograms. Self-reviewed: 161 concerns raised, 149 addressed (12 not taken: Notes).Background
AsyncLocalStoragestores. A job captures it when scheduled.enterWith()in a module body leaves its store in the slot until the next checkpoint (AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190).Module.runMaingoes past it) and a reset where a checkpoint ends (it changes what listeners read: Preserve AsyncLocalStorage context in unhandledRejection handlers #31721).Downsides
Bun.ModuleGraph(bun test --isolate=modules) animport()still loses the store.import()in abeforeExitlistener keeps a store that the entry point entered: the listener reads it since Bun 1.4.2.libJavaScriptCore.a: +6,211 bytes.Notes
Repro.
module.registerHooks)onResolve(a)onLoad(a)onResolve(b)onLoad(b)The program of #32693 (a module body under
store.run("CONTEXT", () => import(...))) printsCONTEXTon Bun 1.2.23 and 1.3.13,undefinedon 1.3.14 to 1.4.4, andCONTEXTwith this pin and on Node.js. The module-loader rewrite (#29393) is in that range. The range was not bisected.The pin.
bacfec4b27is the head of oven-sh/WebKit#941: the pin of main (873ab3a426) with the change. Its build is the releaseautobuild-bacfec4b276fb3277267c8c1f76d8a0863c36a88. The merged commit replaces the pin before this leaves draft. oven-sh/WebKit#941 has the design, the table of jobs and the measurements. The size is from the release builds of873ab3a426and ofbacfec4b27(bun-webkit-linux-amd64-aot,size -t): text 62,582,628 to 62,588,839 bytes.What Bun does besides the pin.
ModuleGraph.cppsetsVM::asyncContextOfLoadFilter: a load that begins in the context of a graph keeps nothing.GlobalObject::resetAsyncContextAtEventLoop()(ZigGlobalObject.cpp) is the statement that AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190 put where a checkpoint begins: with no script on the stack, the slot gets the event loop's value. The host now also calls it where it begins a load by itself:JSModuleLoader__import(a preload),JSC__JSModuleLoader__loadAndEvaluateModuleandJSC__JSModuleLoader__resolveAndLoadAndEvaluateModule(the entry point, a test file, the entry point of a Worker, a--hotreload),Bun__loadBuiltinEntryPoint,NodeModuleModule__callOverriddenRunMain(aModule.runMainthat a preload replaced) andBakeLoadModuleByKey. While script is on the stack the call does nothing, so a load that script begins through one of them keeps the caller's context.drainMicrotasks()compiles to the same 108 instructions as on main, and a load that the host begins executes 14 more and a call (counted in the assembly ofZigGlobalObject.cppat the flags of a release build, without LTO).vm.SourceTextModule#evaluate()says that it evaluates for its caller (JSC::ModuleEvaluationAsyncContextScope,CalledByScript). The modules that wait there for a top-level await continue the call.Tests.
test/js/bun/plugin/plugins.test.ts, block "AsyncLocalStorage in what an import() loads", 44 tests. On main 24 fail (measured one merge back, at 98cda0d). The other twenty are controls for what must stay as it is: the cells that a load allocates with and without a store, a load outside any store,require()of an ES module (six tests, one of them inside a graph), what code of aBun.ModuleGraphloads (three tests) and what the host loads behind a preload (nine tests). On a build of main'ssrcwith the engine change 18 fail (measured one merge back: main at 98cda0d with 37998c5702): thirteen of the fourteen tests of a preload,node:vm(three tests) and two of a graph. Seventeen fail on this pull request as it was at a60b510, before the host's rule moved to where a load begins. Four fail on main for the host's part alone, which needs no engine change: the hooks for the next preload and a replacedModule.runMain, each in both ways in which Bun reads a preload. The tests cover:onResolveandonLoadhooks and module bodies of static importsimport()from a timer and from a promise reactionModule.runMain()andimport.defer()build.module()factorythen()of a namespaceimport()enterWith()in the hook of a dependency, in the hook of the imported module itself, and in a CommonJS fileenterWith(): the entry point (--preload,--import,--require, a CommonJS entry point, a preload that replacesModule.runMain, a preload whose body runs behind a top-level await), the next preload and the hooks for it, the entry point of a Worker, and two test files underbun test,bun test --isolateandbun test --parallel=2vm.SourceTextModule#evaluate()underrun(), underexit()and plain: in a module of a loadingimport(), in the drain of arequire(), in the entry point, behind a second top-level await, and insidegraph.run()mock.module()for a linked module with a top-level await below itrequire()of an ES module: alone, with animport()that the module calls, with animport()in flight (two shapes), and with top-level awaits that the call begins or settlesBun.ModuleGraph: the host's hooks for a load that the host starts, a module of the host that a graph asks for and that outlives the graph, a hook of the host for a load that a graph starts,require()inside a graphmock.module()factorytest/js/node/async_hooks/async-context/async-context-dynamic-import.js:AsyncLocalStorage-tracking.test.tsruns each fixture of that folder on Bun and on Node.js. It fails on main.test/bundler/bundler_compile_prelinked.test.ts,DynamicImportAsyncLocalStorage: animport()of a chunk in abun build --compileexecutable, with and without--bytecodeand--splitting. The two variants with--splittingfail on main.test/aot/parallel/modules: twenty programs, forbun build --compile --aot. They pass in the driver (test/aot/run-parallel.ts) in each of its ways. At an earlier pin an executable of main and one of the change differed in 71 of the 143 lines that sixteen of them print.import()while arequire()loads an ES module. A compiled executable runs thatimport()after the call, under the store of theimport(), as Node.js does (docs/compiler/compatibility.mdx). They say what the executable prints too.require-of-graph-that-awaits-under-store-file-calls-import: the module behind a top-level await of a required graph runs under no store, and not under the store of animport()that a CommonJS file of the graph called.imports-under-stores-files-queue-jobs-stores-are-collected: a collection takes the stores of six calls that are through, three fulfilled and three rejected. Before d6f315cb63 of [JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 the executable kept them.import-from-text-of-a-required-cjs-under-store-reads-its-exports:import()from text of a CommonJS file of the executable calls a getter and the traps of a Proxy under the store of the call, as plain Bun does.import-without-store-is-made-again-while-slot-holds-store: a call with no store that is made again when its text has come reads no store that its caller entered behind the call.import()under a store (test/aot/parallel-test-runs). On the branch of AOT: the next base #48120 it passed in six of six runs where the process had one processor, so that the compiler had one thread. With the default threads the debug build of the compiler stopped in its link stage in seven of eight runs, before the program ran ([JSC] A job of the module loader runs under the async context of the import() that asked for it WebKit#941 names the frames). In a debug build it would fail for a reason that is not its subject.test/aot/an-import-call-in-a-compiled-test-run-reads-its-store.test.ts:bun test,bun test --isolateandbun test --aoton one project. A compiled test run reads the text of what animport()names in a later turn, and the modules still read the store of the call. They fail on main.bun test --isolate=modulesis a todo there: a test file is a module of a graph.Other suites. On this head, in a debug and ASAN build with the CI build of the pin, with five tests at once and a limit of 60 s:
plugins.test.ts(323 tests),AsyncLocalStorage.test.ts(63),AsyncLocalStorage-tracking.test.ts(77),vm.test.ts(372),node-module-module.test.js(80),preload-test.test.js(8),module-graph.test.ts(297),module-graph-matrix.test.ts(893),bundler_compile_prelinked.test.ts(91),test/aot/ci-lists.test.tsand the test of a compiled test run pass.module-graph-isolation.test.tspasses 411 of 413: the two cells that fail belong to a group of 88 that waits a fixed time for background work, and that group fails in four of six runs on a build of main alone on this machine. The twenty programs pass in the driver in its four ways. 118 stored test runs of main for animport()under a mock, or with a text that comes in a later turn, print in a compiled run what plain Bun prints (oven-sh/WebKit#941 says how they were run). One merge back (main at 98cda0d, pin 37998c5702) 2,829 tests of the five files oftest/js/node/async_hooks, the nine oftest/js/node/vm, seven oftest/js/bun/module-graphand the files above passed.module-graph-workers.test.tspassed 475 and 476 of its 479 tests in two runs there, and 478 on a build of main: the cells that fail differ from run to run and wait a fixed time too. On a build of main'ssrcwith the engine change no other test of main failed there for that reason: the 279 other tests ofplugins.test.ts,vm.test.ts,AsyncLocalStorage.test.ts,AsyncLocalStorage-tracking.test.ts,node-module-module.test.js,preload-test.test.jsand seven files oftest/js/bun/module-graphwere run. At the default of twenty tests at once many tests of these files run out of their 5 s limit in a debug build on this machine, with and without this change.What changes besides the fix.
Bun.ModuleGraph: where the host starts a load under a store (graph.import()), the host's hooks for the static dependencies read that store. A load that begins in the context of a graph keeps no context, so what code of a graph loads runs as before. The context of a graph is in the same slot as the stores, and what runs in it ends when the graph is disposed. A hook of the host and a module of the host's loader must not end with a graph: with a top-level await, such a module would never go on.JSModuleGraphsets that rule as the VM's filter. So inside a graph animport()under a store still loses the store, and a test file underbun test --isolate=modulesis a module of a graph.import()can start a dependency that has a top-level await. A module of another load that waits for that dependency runs in the store of the firstimport(). The other load can be the host's: an entry point or a test file that waits for a module that a preload began to import reads the store of thatimport(), and so does what it starts (a server's handlers, later timers). Node.js does the same. Bun 1.3.14 to 1.4.4 read none there.import()with a store, a module enters another store withenterWith(), and the next module of the graph has a top-level await. The modules that wait for it run under the entered store. Node.js does the same.enterWith()ends with the hook: the hooks behind it and the module bodies read the store of theimport(). Node.js and Bun 1.4.4 hand the entered store on to what the load runs next. Bun 1.3.13 handed it to the hooks behind it and not to the bodies. The hooks for the module thatimport()names run inside the call, so what they enter is what the load keeps.vm.SourceTextModule#evaluate()under a store: the modules that wait for a top-level await read the store of the call, also underrun(), behind a second top-level await, and in the entry point. Underexit()they read none. Node.js does the same. On main they read none in each case.Module.runMainthat a preload replaced, read no store that the preload before them entered. Bun read it where it reads a preload on the thread of the program: on main by default (bun test: Vitest compatibility #44833,is_maininsrc/runtime/jsc_hooks.rs), and in Bun 1.4.2 to 1.4.4 withBUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1and in a compiled executable whose preloads are embedded.What does not change.
unhandledRejectionlisteners see no store in Bun, for a failedimport()as for any rejection (AsyncLocalStorage context is not preserved whenunhandledRejectionis emitted #40223, AsyncLocalStorage store is not available inside process.on('unhandledRejection') listeners (Node restores it) #44650). Open work: Preserve AsyncLocalStorage context in unhandledRejection handlers #31721, [JSC] An unhandled rejection is reported in the async context of what scheduled the job that rejects WebKit#729.enterWith()outside anyrun()reaches the jobs of the same turn of the event loop, and not a job behind a timer (AsyncLocalStorage: drop enterWith() frames at the event-loop boundary, stop allocating per await/then #41190).enterWith()gives its store neither to the entry point, nor to the next preload, nor to a test file, as in a file run of every release of Bun (1.0.36 to 1.4.4).node --requiregives the store of a CommonJS preload to the entry point, andnode --importgives none. Bun waits for every preload asnode --importdoes. Nobody has asked for the answer ofnode --require. It is a maintainer's choice, and it would be made inload_preloads(src/runtime/jsc_hooks.rs).enterWith(): abeforeExitlistener, anunhandledRejectionlistener for a rejection of that turn, the first signal or message listener. Since Bun 1.4.2 it reads that store, and it does here. Animport()in such a callback keeps the store, as anyimport()keeps what its caller reads: that part is new. Preserve AsyncLocalStorage context in unhandledRejection handlers #31721 resets the slot where a checkpoint ends, together with the capture that gives anunhandledRejectionlistener what Node.js gives it. An earlier shape of this pull request had that reset alone. A review of it measured 34 readings of such listeners that change, 12 of them away from Node.js, so it is not here.mock.module(): where it evaluates a linked module early, the job of the loader that runs the calling module says what the modules behind a top-level await get, as before. An earlier shape continued the caller there, and handed a pending mock factory's frame to modules of another load.require()call runs, it runs in the context of its caller. That coversrequire()of an ES module and of any other file that goes through the module loader.Bun.build()plugin hooks: bundler: run Bun.build plugin hooks in the AsyncLocalStorage stores of the Bun.build() call #41562.Differences from Node.js that remain (Node.js v26.3.0, and v26.12.1, which this repository pins, for the rows of a preload and of
node:vm). The tests of the block were also run on Bun 1.3.13 and, ported tomodule.registerHooks, on Node.js.Older than 1.3.14 (Bun 1.3.13 does the same):
enterWith()ends with that job. Node.js runs the file with the module bodies, so the importing module sees the store.import()calls that load one module at the same time: the module runs in the store of one of them. In 8 runs it was the first call 4 times. Node.js: always the first. The order in which Bun hands transpiled modules to the loader decides (Hand transpiled modules back to the module loader in request order #42051).Since 1.3.14, and kept (Bun 1.3.13 does what Node.js does):
require()call loads and evaluates every module of the loader that becomes ready while it runs, inside the call: also modules of animport()in flight whose graph it completes, and of animport()that a required module calls. They read the caller's store, andvm.SourceTextModule#evaluate()in such a module hands that store on. Node.js evaluates them later, under the store of theirimport(). A module of such animport()that runs after the call, behind a top-level await, reads the store of itsimport(). Abun build --compile --aotexecutable runs animport()that a required module calls after the call, under the store of thatimport(), as Node.js does.New with this change:
enterWith()ends with the hook (see "What changes besides the fix").vm.SourceTextModule#evaluate()with nothing current at the call: the modules that wait read what the slot holds when their job runs, for example a store that the awaiting module entered. Node.js reads none there. (Not new: main reads the same.)The tests of the block that pin one of these outputs say in a comment what Node.js prints.
Review. The design, the rules and the measurements are in oven-sh/WebKit#941, with the account of each review. The last three reviews raised 98 concerns, and 86 are answered. What they changed here:
Module.runMainwent past it. The second answer was a reset where a checkpoint ends: it changed what listeners read in the entry point's first turn, Preserve AsyncLocalStorage context in unhandledRejection handlers #31721 has it with its other half, and a deferred task, the printer of a rejection and a--hotreload went past it. The reset is now where the host begins a load.evaluate()ofnode:vmcontinues its caller.mock.module()had the same scope for one review, and lost it.Not taken: a safeguard that lets the engine half merge alone (see "The order" in oven-sh/WebKit#941: the host's reset and its tests need no engine change and can go first). Three debug assertions of the loader fire in programs that nest
require()of ES modules, with noAsyncLocalStoragein the program. They fired on a debug build of the pin that main had then too (5a61071f79:JSModuleLoader.cpplines 869, 1147 and 1148).Reach. Module bodies have one report (#32693) and one downstream: manten skips three specs on Bun for it. With the skips removed, Bun 1.4.2 gives 117 pass and 3 fail, and the first commit of oven-sh/WebKit#941 gave 120 pass and 0 fail. One of the three specs alone still fails in 5 of 10 runs for the reason in #42051. The hooks, the factories and the CommonJS bodies have no report. They kept the store in Bun 1.3.13 and keep it in Node.js.
no test proof · iteration 10 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/plugin/plugins.test.ts, test/bundler/bundler_compile_prelinked.test.ts