Skip to content

Fix spurious "require() async module" TypeError when require() hits an ESM module the import graph is still fetching - #37185

Open
robobun wants to merge 11 commits into
mainfrom
farm/7d8432f2/require-esm-in-flight-sibling
Open

robobun wants to merge 11 commits into
mainfrom
farm/7d8432f2/require-esm-in-flight-sibling

Conversation

@robobun

@robobun robobun commented Aug 8, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #33180 (jose required by jwks-rsa / firebase-admin while the app also imports jose; also reported with eventsource in that issue and as #37557). Consolidated with #33184, the earlier PR for the same race, which is closed in favour of this one; its issue-shaped test is carried here as test/regression/issue/33180.test.ts.

Problem

  • A CommonJS module that an ESM graph imports has its body evaluated while the graph is still loading (the loader's makeModule step runs the CJS body to learn its exports). If that body require()s a plain ESM module (no top-level await) which the same graph also imports, the module's registry entry already exists in status Fetching, with its fetch promise piped from the async transpiler.
  • fetchCommonJSModule (src/jsc/bindings/ModuleLoader.cpp) transpiles the file synchronously and hands the source to JSModuleLoader::provideFetch(), but provideFetch() only accepts an entry in status New, so the source was dropped on the floor. functionEsmLoadSync then ran loadModuleSync, whose loadModule reuses the entry's still-pending fetch promise (JSModuleLoader.cpp, the status != New branch), so the synchronous drain had nothing to run, the load promise stayed pending, and that pending promise is what gets reported as TypeError: require() async module "..." is unsupported. use "await import()" instead.
  • Regressed in 1.3.14 with the module loader rewrite (Upgrade WebKit to 87fd0daba19a (module-loader rewrite) #29393); 1.3.11 and Node load the module synchronously. Whether a given run fails depends on whether the async transpile of the required module has landed yet, which is why bigger modules fail more often (the issue's measurements) and why small isolated repros were clean.

Fix

  • provideFetchForSyncLoad replaces the four provideFetch() calls in the require path (fresh transpile, isolated module cache, and both virtual-module branches). It looks up the JavaScript-typed entry for the specifier (ensureRegistered(key, Type::JavaScript), the same O(1) lookup provideFetch() itself starts with) and then does one of three things: a New entry is fed the source exactly as before; a Fetching entry whose fetch promise is still pending has that promise settled directly with the source the synchronous transpile just produced; anything else (Fetched, a failed entry, or Fetching with the fetch promise already settled) is left alone, because loadModuleSync() already handles those states itself, the last one through hostLoadImportedModule's inline replay of the queued makeModule step.
  • The lookup is deliberately typed. An earlier revision used registryEntry(), which falls back to an entry of any type, so a require() racing an in-flight import ... with { type: "ts" } (or any other loader attribute) of the same file would have fulfilled that typed entry with default-loader output while doing nothing for the require() itself, which reads the JavaScript entry; it also walked the whole module map whenever the file had no entry yet, which is the common case for a first require() of an ESM file. Typed imports keep their own entry and their own transpile, as on main.
  • That is correct because the entry's FetchSettled reaction is what drives makeModule, and it is attached to the entry's own fetch promise: settling it under the SynchronousModuleQueue the caller is about to drain runs the rest of the chain inline, so loadModuleSync afterwards finds the entry Fetched and links and evaluates it synchronously. The promise has to be settled with fulfillPromise rather than the guarded fulfill, because pipeFrom() already claimed the resolving-function flag; this is the same thing hostLoadImportedModule's existing synchronous-replay block does for dependency edges. When the async transpile finishes later it lands on an already-settled promise and is dropped by the pending-target check in PromiseFulfillWithoutHandlerJob, again the existing contract that replay block relies on.
  • Compared with settling the entry from functionEsmLoadSync (module loader: fix false "require() async module" for a module racing a concurrent import #33184, and the overlapping hunk in Fix CommonJS modules silently skipped when require() loads an ESM graph #37187): the source already transpiled synchronously is used instead of re-issuing a third transpile, and the virtual-module and isolated-cache paths get the same treatment. Fix CommonJS modules silently skipped when require() loads an ESM graph #37187's first test (a CJS module inside a require()d ESM graph requiring an in-flight sibling) also passes on this branch; its other two tests cover a separate require-cache bug that this PR does not touch, so Fix CommonJS modules silently skipped when require() loads an ESM graph #37187 stays open for that half.
  • A module that genuinely cannot complete synchronously (top-level await, async plugin) still ends with a pending load promise and still gets the async-module error; require-esm-transitive-tla, require-esm-microtask-order, issue/24387 and require.test.ts pass on this build.
  • Verified with bun bd test test/js/bun/resolve/require-esm-in-flight-sibling.test.ts test/regression/issue/33180.test.ts (5 pass), plus the require-esm-*, require.test.ts, issue/24387 and mock-module files (which exercise the virtual-module branch). Both files fail on the current release build, and the regression file's two cases failed 3/3 runs there; the cycle shape also failed 10/10 on a debug build of current main before this change and passes 10/10 after. The issue's own synthetic plain-esm package shape and a four-sibling variant pass 10/10 on this build.

Tests

  • test/js/bun/resolve/require-esm-in-flight-sibling.test.ts: deterministic. The entry imports an ESM module and a CJS package named reflect-metadata; the runtime always transpiles that package on the main thread (ALWAYS_SYNC_MODULES in src/runtime/jsc_hooks.rs), so its body runs in the microtask checkpoint right after the graph's fetches are issued, before the transpiler thread can have delivered the ESM sibling, and its require() of that sibling hits the Fetching entry on every run. Asserts Node's output e,polyfill:1,entry. Fails 10/10 on the release build and on main, passes 20/20 here; also passes 5/5 under BUN_JSC_collectContinuously=1. One spawn, no per-test timeout. Two more cases share the fixture:
    • the same graph with BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1, where the sibling's fetch promise is already settled when the require() lands and only its makeModule reaction is still queued. This is the state the helper has to leave alone; with the pending check removed it fails on ASSERTION FAILED: status() == Status::Pending (verified), and it passes on main.
    • a require() racing an import ... with { type: "ts" } of the same .js file. The file's a < b > (c) is a generic call under the ts loader and two comparisons under the default loader, so the test can see which transpile each consumer got: the typed import must still see called(3) and the require() false, with the file evaluated once per entry. Fails against the previous revision of this branch (the typed import saw false), passes here and on main.
  • test/regression/issue/33180.test.ts (carried over from module loader: fix false "require() async module" for a module racing a concurrent import #33184), two natural shapes of the issue that depend on transpiler timing to reach the bug (they fail 8/10 and 9/10 runs on the release build, 0/15 failures on this branch):
    • a cycle: a one-line CJS starter whose require chain reaches an ESM module the entry is also importing; that module imports the half-evaluated CJS module back. Asserts the CJS module ran once and both importers see the exports added after the cycle, which is the JSON Node prints.
    • the issue's shape: four large ESM siblings imported by the entry and require()d in turn by a small CJS sibling.

Background

  • Bun's require() of a file that turns out to be ESM works in two steps: fetchCommonJSModule transpiles it and registers the source with JSC's module loader, then functionEsmLoadSync (ZigGlobalObject.cpp) asks the loader to load it with loadModuleSync, which installs a VM::SynchronousModuleQueue so the loader's internal promise reactions are collected and drained inline instead of going to the microtask queue. A load whose promise is still pending after the drain is treated as a module that needs the event loop, i.e. top-level await.
  • ModuleRegistryEntry is JSC's per-module record: status (New, Fetching, Fetched, ...), a fetch promise that resolves to the JSSourceCode, and a module promise that resolves once makeModule has produced the record. Dependency loads started by import create the entry immediately and pipe its fetch promise from Bun's async transpiler result.
  • Loading a whole graph synchronously (require() of an ESM entry, BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1, NODE_COMPILE_CACHE) lets nested synchronous loads complete other edges of the referrer during one host-load call, which JSModuleLoader::innerModuleLoading makes assumptions about (needsErrorReaction, and an assertion on debug builds). Tightening that is a JSC-side change, JSModuleLoader: decide needsErrorReaction by membership in innerModuleLoading WebKit#396, independent of this fix; this PR's tests load their graphs through an ESM entry.

[review] gate passed · iteration 1 · 3 files touched

fails on main (without fix)
ASAN without fix: 3 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/resolve/require-esm-in-flight-sibling.test.ts "test/regression/issue/33180.test.ts"
bun test v1.4.0 (9ac5986c3)

test/regression/issue/33180.test.ts:
28 |     env: bunEnv,
29 |     cwd: String(dir),
30 |     stderr: "pipe",
31 |   });
32 |   const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
33 |   expect({ stdout, stderr, exitCode }).toEqual({
                                            ^
error: expect(received).toEqual(expected)

  {
-   "exitCode": 0,
-   "stderr": "",
-   "stdout": 
- "{"evaluations":1,"late":"late","lateViaESM":"late"}
+   "exitCode": 1,
+   "stderr": 
+ "1 | 
+ 2 |       globalThis.evaluations = (globalThis.evaluations ?? 0) + 1;
+ 3 |       exports.early = "early";
+                                  ^
+ TypeError: require() async module "/tmp/issue-33180-cycle_KTCjZc/b.mjs" is unsupported. use "await import()" instead.
+       at require (57:24)
+       at <anonymous> (/tmp/issue-33180-cycle_KTCjZc/a.cjs:3:23)
+       at require (51:24)
+     
... (truncated)

release without fix: 3 FAILED
bun test v1.4.0-canary.1 (da3851e57)

test/regression/issue/33180.test.ts:
28 |     env: bunEnv,
29 |     cwd: String(dir),
30 |     stderr: "pipe",
31 |   });
32 |   const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
33 |   expect({ stdout, stderr, exitCode }).toEqual({
                                            ^
error: expect(received).toEqual(expected)

  {
-   "exitCode": 0,
-   "stderr": "",
-   "stdout": 
- "{"evaluations":1,"late":"late","lateViaESM":"late"}
+   "exitCode": 1,
+   "stderr": 
+ "1 | 
+ 2 |       globalThis.evaluations = (globalThis.evaluations ?? 0) + 1;
+ 3 |       exports.early = "early";
+                           ^
+ TypeError: require() async module "/tmp/issue-33180-cycle_UlARVm/b.mjs" is unsupported. use "await import()" instead.
+       at <anonymous> (/tmp/issue-33180-cycle_UlARVm/a.cjs:3:23)
+       at <anonymous> (/tmp/issue-33180-cycle_UlARVm/starter.cjs:2:10)
+ 
+ Bun v1.4.0-canary.1+da3851e57 (Linux x64)
  "
  ,
+   "stdout": "",
  }

- Expected  - 4
+ Received  + 12

      at <anonymous> (/workspace/bun/test/regression/issue/33180.test.ts:33:40)
(fail) require() of an 
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/resolve/require-esm-in-flight-sibling.test.ts "test/regression/issue/33180.test.ts"
bun test v1.4.0 (9ac5986c3)

test/regression/issue/33180.test.ts:
(pass) require() of an ESM module the surrounding import graph is still fetching [1142.22ms]
(pass) consecutive require()s of several ESM siblings that are still transpiling [4296.23ms]

test/js/bun/resolve/require-esm-in-flight-sibling.test.ts:
(pass) require() of an ESM sibling the import graph is still fetching loads it synchronously [2390.54ms]
(pass) require() of a sibling whose fetch has settled but not been processed yet [2183.54ms]
(pass) require() racing a typed import of the same file does not overwrite the typed import [2338.14ms]

 5 pass
 0 fail
 5 expect() calls
Ran 5 tests across 2 files. [16.76s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     9ac5986c33
  features     baseline

22 deps, 123 codegen, 1176 objects in 2019ms

ninja: Entering directory `/workspace/bun/build/release'
[1/1238] fetch tinycc
[tinycc] up to date
[2/1237] gen .bind.ts → GeneratedBindings.cpp
[3/1237] gen bindgenv2
[4/1237] fetch zlib
[zlib] up to date
[5/1237] fetch libjpeg-turbo
[libjpeg-turbo] up to date
[6/1237] gen ErrorCode+*.h
[7/1237] gen ProcessBindingBuffer.lut.h
Generating /workspace/bun/build/release/codegen/ProcessBindingBuffer.lut.h from /workspace/bun/src/jsc/bindings/ProcessBindingBuffer.cpp
[8/1237] install /workspace/bun
bun install v1.4.0-canary.1 (da3851e57)

Checked 107 installs across 153 packages (no changes) [1284.00ms]
[9/1237] gen ProcessBindingConstants.lut.h
Generating /workspace/bun/build/release/codegen/ProcessBindingConstants.lut.h from /workspace/bun/src/jsc/bindings/ProcessBindingConstants.cpp
[10/1237] install /workspace/bun/packages/bun-error
bun install v1.4.0-canary.1 (da3851e57)

Checked 1 install across 2 packag
... (truncated)
diff hotspot
src/jsc/bindings/ModuleLoader.cpp                  | 36 +++++++-
 .../resolve/require-esm-in-flight-sibling.test.ts  | 96 ++++++++++++++++++++++
 test/regression/issue/33180.test.ts                | 75 +++++++++++++++++
 3 files changed, 203 insertions(+), 4 deletions(-)

gate history · 3 passed · 0 rejected · iteration 1

evidence per changed file
file                                                      reads  edits  tests
src/jsc/bindings/ModuleLoader.cpp                             7     11      0
…st/js/bun/resolve/require-esm-in-flight-sibling.test.ts      2      4      0
test/regression/issue/33180.test.ts                           1      2      0

…entry

A CommonJS module imported by an ESM graph has its body evaluated while
the graph is still loading, and that body can require() an ESM sibling
the graph already started fetching on the transpiler thread. The
sibling's registry entry is then status Fetching with a pending fetch
promise. fetchCommonJSModule re-transpiled the file synchronously but
provideFetch() only accepts a New entry, so the fresh source was
silently dropped, the synchronous load had nothing to drain, and
require() threw a spurious
  TypeError: require() async module "..." is unsupported. use "await import()" instead.
for a module with no top-level await. Intermittent on multi-file mixed
ESM/CJS graphs (28/30 runs on the repro below on a loaded machine).

Settle the entry's fetch promise with the just-transpiled source
instead. The async transpiler's own result lands later on an
already-settled promise and is dropped by the existing
PromiseFulfillWithoutHandlerJob pending-target guard.
@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

Synchronous ESM loading

Layer / File(s) Summary
Fetch coordination for synchronous loads
src/jsc/bindings/ModuleLoader.cpp
provideFetchForSyncLoad reuses pending fetch entries and registers new sources for virtual, cached, and transpiled modules.
Cyclic and concurrent loading regression coverage
test/regression/issue/33180.test.ts, test/js/bun/resolve/require-esm-in-flight-sibling.test.ts
Tests cover cyclic evaluation, in-flight ESM siblings, and consecutive synchronous requires during transpilation.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title directly and specifically describes the spurious TypeError fix for synchronous require() of an ESM module during in-flight fetching.
Description check ✅ Passed The description clearly covers the change, root cause, implementation, affected paths, regression tests, and verification results.

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. module loader: fix false "require() async module" for a module racing a concurrent import #33184 - Fixes the same false require() async module TypeError caused by a registry entry stuck at Status::Fetching during a synchronous require(esm), using the same ensureFetchPromise/fulfillPromise + synchronous-module-queue drain mechanic, just applied in functionEsmLoadSync (ZigGlobalObject.cpp) instead of at the provideFetch call sites in ModuleLoader.cpp.

🤖 Generated with Claude Code

@robobun

robobun commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

The overlap with #33184 is real but partial. Both address the same symptom (the false require() async module TypeError for a registry entry still Fetching, reported in #33180) with the same core mechanic: settle the entry's fetch promise under a synchronous module queue, relying on the pending-target guard in PromiseFulfillWithoutHandlerJob to drop the async transpiler's late delivery.

Differences:

Only one of the two should land. If maintainers prefer #33184's placement, the WebKit companion and the ordered test here still stand on their own; otherwise this PR supersedes it for the race.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and the automated bug-hunting pass found no issues. Because it touches the JSC module-loader race path (directly settling a pipeFrom()'d fetch promise via fulfillPromise and relying on the later async-transpiler result being dropped), a human look from someone familiar with the C++ loader would still be worthwhile.

What was reviewed:

  • All four provideFetch call sites in fetchCommonJSModule/fetchCommonJSModuleNonBuiltin route through the new helper; the fallthrough still calls provideFetch with the JSSourceCode* overload (matches ZigGlobalObject.cpp:741).
  • The isolation-cache and non-builtin sites now wrap SourceCode in JSSourceCode::create — consistent with what the fetch promise expects.
  • Checked the transitive mid-fetch (diamond) graph concern — ruled out; the entry-status guard applies per-specifier.
  • Test spawns 12 concurrent subprocesses with a padded ESM sibling, drains stdout/stderr/exited concurrently, and asserts exact evaluation order; 30s timeout is justified for debug/ASAN.
Extended reasoning...

Overview

This PR fixes a race in src/jsc/bindings/ModuleLoader.cpp where a CJS module require()ing an ESM sibling that is already mid-fetch on the async transpiler thread would spuriously throw require() async module ... is unsupported. The fix introduces provideFetchForSyncLoad(), which detects a Fetching-status registry entry and settles its fetch promise directly with the just-transpiled source (via fulfillPromise, bypassing the first-resolving-function guard set by pipeFrom()), so the synchronous module queue can drain it. Four call sites are rewired; two now wrap raw SourceCode in JSSourceCode::create. A new test reproduces the race with a padded ESM file across 12 concurrent runs.

Security risks

None. This is internal module-loader plumbing; no user-controlled input reaches new parsing or allocation paths, and no auth/crypto/permissions surface is touched.

Level of scrutiny

High. The module loader is on every startup path, and the fix depends on non-obvious JSC internals: that fulfillPromise can safely override a pipeFrom-armed promise, that the FetchSettled reaction lands on the caller's synchronous queue, and that the async transpiler's later resolution is harmlessly dropped by PromiseFulfillWithoutHandlerJob's pending-target check. The PR also notes a related JSC-side assertion (oven-sh/WebKit#396) that covers adjacent doors of the same graph. These are correctness claims about vendored JSC behavior that merit a maintainer's eye.

Other factors

The bug-hunting system raised and refuted two candidates (30s test timeout — reasonable for 12 debug/ASAN spawns; transitive diamond-graph coverage — the fix is per-specifier). The test follows harness conventions (tempDir, bunEnv, concurrent pipe drain, exact-output assertion, exit code last). No prior human review comments. Given the subtlety of the JSC promise/registry interaction and its position in a critical hot path, deferring to human review rather than auto-approving.

@robobun

robobun commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

For whoever takes the human look, the two contracts the review flags are both explicit in the vendored JSC:

  • fulfillPromise on a pipeFrom()'d promise: pipeFrom sets isFirstResolvingFunctionCalledFlag on the target (JSPromise.cpp, JSPromise::pipeFrom), and that flag only gates the guarded entry points (fulfill/reject/rejectAsHandled). That is exactly why the helper calls fulfillPromise directly, and it is the same call hostLoadImportedModule's synchronous-drive block makes for the nested case (JSModuleLoader.cpp, the fetchPromise->fulfillPromise(vm, promise->result()) branch under USE(BUN_JSC_ADDITIONS)). This PR extends that existing contract to the require() entry path rather than inventing a new one.
  • The async transpiler's late delivery: the PromiseFulfillWithoutHandlerJob handler in JSMicrotask.cpp opens with a pending-target check added for force-settled promises ("hostLoadImportedModule may have force-settled this promise inline while a synchronous loadModule was active; the queued pipeFrom job is now redundant"), so the piped result lands on a settled promise and is dropped. The stranded-FetchSettled copy on the normal microtask queue is likewise guarded by the pending-modulePromise check in moduleRegistryFetchSettled.

CI is green across all lanes on Buildkite build 90527.

A deterministic CJS/ESM cycle shape (a one-line CJS starter whose require
chain reaches an ESM module the entry is also importing) and the issue's
shape with several large siblings required in a row. Both fail on 1.3.14
through current main with the spurious require() async module TypeError.
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
@robobun robobun changed the title Fix spurious "require() async module" TypeError when a CJS module requires an ESM sibling mid-transpile Fix spurious "require() async module" TypeError when require() hits an ESM module the import graph is still fetching Aug 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/bun/resolve/require-esm-in-flight-sibling.test.ts`:
- Around line 10-12: Make the regression deterministic by coordinating the test
loader state so d.cjs invokes require("./e.mjs") only after e.mjs has a registry
entry in the Fetching state, exercising the production guard directly. Update
the test setup around the affected lines to expose or await that state
transition, then remove the twelve-process repetition and 30-second timeout from
the test.

In `@test/regression/issue/33180.test.ts`:
- Around line 1-8: Keep only the issue URL comment in
test/regression/issue/33180.test.ts at lines 1-8; remove the explanatory
comments at lines 13-18 and 53-56, leaving the test behavior unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9fb33487-62b6-44ff-bb0e-f01938e02871

📥 Commits

Reviewing files that changed from the base of the PR and between f426a8e and c8bf720.

📒 Files selected for processing (3)
  • src/jsc/bindings/ModuleLoader.cpp
  • test/js/bun/resolve/require-esm-in-flight-sibling.test.ts
  • test/regression/issue/33180.test.ts

Comment thread test/regression/issue/33180.test.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. It touches the JSC module loader's promise-settling internals and force-fulfills a pipeFrom()'d fetch promise — subtle enough that a human maintainer should confirm the mechanic, and there's a placement decision to make against #33184.

What was reviewed:

  • provideFetchForSyncLoad: verified the Fetching-status branch settles via fulfillPromise (bypassing the first-resolving-function guard) and falls through to provideFetch for New entries; the four call sites are the complete set that previously called provideFetch under a SynchronousModuleQueue.
  • Checked the two call sites that now wrap a raw SourceCode in JSSourceCode::create(vm, ...) — matches what provideFetch does internally.
  • Tests: 12-way concurrent spawn asserts exact evaluation order (not just absence of the error), plus a deterministic cycle test and a multi-sibling test for #33180; all drain stdout/stderr/exited concurrently.
Extended reasoning...

Overview

The PR adds a static helper provideFetchForSyncLoad in src/jsc/bindings/ModuleLoader.cpp and rewires the four provideFetch call sites inside fetchCommonJSModule / fetchCommonJSModuleNonBuiltin to use it. When a synchronous require(esm) re-transpiles a module whose registry entry is already Status::Fetching (because the async transpiler thread started it as part of a surrounding ESM graph), the helper settles the entry's fetch promise directly with the fresh source instead of calling provideFetch(), which only accepts New entries and was silently dropping the result. Two new test files exercise the 6-file doubly-nested graph and the #33180 shapes.

Security risks

None identified. This is module-loader control flow; no untrusted-input parsing, auth, or crypto is touched.

Level of scrutiny

High. This is core module-loader machinery that relies on non-obvious JSC-internal contracts: (1) fulfillPromise deliberately bypasses the isFirstResolvingFunctionCalledFlag set by pipeFrom(), and (2) the async transpiler's late delivery is expected to be dropped by the pending-target guard in PromiseFulfillWithoutHandlerJob. The author's comment cites the matching precedent in hostLoadImportedModule's synchronous-drive block, and the mechanic reads correctly against the vendored JSC, but a wrong assumption here would produce hard-to-reproduce module-graph corruption. This is not a mechanical change I can approve without a human look.

Other factors

  • There is an open competing PR (#33184) applying the same mechanic at a different layer (functionEsmLoadSync); the description explicitly says only one should land, which is a maintainer decision.
  • The fix is paired with oven-sh/WebKit#396 for the second face of the same graph (the innerModuleLoading assertion); tests for those doors are deferred to the WebKit bump.
  • The comment-cop bot has flagged four multi-line comments in the new helper. Those comments document the JSC contracts the fix depends on rather than justifying a workaround, but the author may want to trim them.
  • CI green on Buildkite build 90527; the gate evidence shows the new test fails on main (both debug/ASAN and release) and passes with the fix.

…ments

The test now imports a package named reflect-metadata, which the runtime
always transpiles on the main thread, so its body runs in the microtask
checkpoint after the graph's fetches are issued and the ESM sibling is
mid-fetch on every run instead of most runs. One spawn replaces twelve.

provideFetchForSyncLoad keeps one comment on the fulfillPromise call and
follows the neighbouring throw-scope convention.
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
@robobun

robobun commented Aug 13, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 11:27 AM PT - Aug 13th, 2026

✅ @robobun, your commit 9ac5986c3351764665269ff34662e906e7dfbb02 passed in Build #94664! 🎉


🧪   To try this PR locally:

bunx bun-pr 37185

That installs a local version of the PR into your bun-37185 executable, so you can run:

bun-37185 --bun

Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/jsc/bindings/ModuleLoader.cpp`:
- Around line 658-676: Root jsSourceCode in provideFetchForSyncLoad with a
JSC::Strong<JSC::JSSourceCode> (or equivalent JS-thread rooted handle) before
ensureModulePromise and ensureFetchPromise, and use that rooted value when
fulfilling the fetch promise. Add a GC-stress regression covering the
isolated-cache and transpiled paths that pass newly created JSSourceCode into
this flow.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bd8b9b0a-e1f6-48ce-be1a-a3ea8e7ff705

📥 Commits

Reviewing files that changed from the base of the PR and between c8bf720 and a671e79.

📒 Files selected for processing (3)
  • src/jsc/bindings/ModuleLoader.cpp
  • test/js/bun/resolve/require-esm-in-flight-sibling.test.ts
  • test/regression/issue/33180.test.ts
💤 Files with no reviewable changes (1)
  • test/regression/issue/33180.test.ts

Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it changes JSC module-loader promise settlement on the require(esm) path — force-fulfilling a pipeFrom()'d fetch promise under the synchronous module queue, racing the async transpiler's late delivery — a human look is still warranted.

What was reviewed:

  • provideFetchForSyncLoad replaces all four provideFetch sites on the sync require path; the two remaining direct calls (ZigGlobalObject.cpp:750, bindings.cpp:3303) are entry-point loads, not this path.
  • Confirmed reflect-metadata is in ALWAYS_SYNC_MODULES (src/runtime/jsc_hooks.rs:4391), so the deterministic test's ordering premise holds.
  • jsSourceCode lifetime across ensure*Promise: stack-local, conservatively scanned; same shape as upstream ModuleRegistryEntry::provideFetch.
  • Note: build #94024 shows test/cli/run/require-cache.test.ts failing on two lanes for e05032d — worth confirming that's an unrelated flake given this PR touches the require path.
Extended reasoning...

Overview

The PR adds a ~20-line static helper provideFetchForSyncLoad in src/jsc/bindings/ModuleLoader.cpp and swaps four existing provideFetch() calls on the synchronous require(esm) path to use it. When the target module's registry entry is already in Fetching status (an async transpile is in flight because the surrounding ESM import graph also imports it), the helper force-settles the entry's pending fetch promise with the source that was just transpiled synchronously, using fulfillPromise to bypass the resolving-function flag that pipeFrom() already claimed. Two new test files exercise the deterministic case (via the ALWAYS_SYNC_MODULES reflect-metadata trick) and the natural issue-shaped cases.

Security risks

None identified. This is internal module-loader plumbing with no user-controlled input reaching new parsing or allocation paths; the source code being registered is the same source the old code passed to provideFetch.

Level of scrutiny

High. This is core runtime infrastructure: the JSC module loader's registry state machine and promise-reaction ordering. The correctness of the fix depends on several JSC-internal contracts (that fulfillPromise is the right bypass for a pipeFrom()'d promise, that the async transpiler's later delivery is dropped by PromiseFulfillWithoutHandlerJob's pending-target check, and that settling under an active SynchronousModuleQueue runs the FetchSettled reaction inline). The PR description and the earlier robobun comment cite the exact vendored-WebKit lines for each of these, and the mechanism mirrors what hostLoadImportedModule's existing synchronous-replay block already does — but verifying those contracts requires someone comfortable with the JSC module loader internals, which is exactly what a human reviewer should sign off on.

Other factors

  • All prior review-thread items are resolved: the comment-cop comment-length flags were trimmed, my earlier nit about asymmetric RETURN_IF_EXCEPTION after ensure*Promise was addressed in e05032d, CodeRabbit's GC-rooting concern was withdrawn (stack-local under conservative scan), and the deterministic-test rewrite landed in e06da7d.
  • The robobun CI status for the head commit e05032d lists test/cli/run/require-cache.test.ts failing on both 🐧 13 x64-asan and 🐧 3.23 x64. That test file is adjacent to the code this PR changes (require-path behavior), and the PR description itself notes overlap with #37187's require-cache bug. A human should confirm whether that failure is a pre-existing flake on main or introduced here before merging.
  • The PR description also references a pending JSC-side change (oven-sh/WebKit#396) for a related-but-separate innerModuleLoading assumption; this PR's tests deliberately load through an ESM entry to avoid that path, so the two are independent, but it's context a human reviewer should be aware of.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

On the test/cli/run/require-cache.test.ts failures in build 94024 (flagged in the review above): they are unrelated to this change. Every failure in that file is a timeout of a leak-measurement test, and none of those tests reach the changed code. The via require() cases require CommonJS files (exports.x = ...), which fetchCommonJSModuleNonBuiltin evaluates and returns from at the isCommonJSModule check, before the provideFetchForSyncLoad call; the via import() / await import() cases go through fetchESMSourceCode, which this PR does not touch. The same source change (unchanged in logic since f6c9b25) passed this file in build 90527, and in build 94024 the debian x64-asan lane also timed out fs.test.ts, inspect-error-leak, html-rewriter-leak and two elysia vendor suites, so that lane was starved. Retriggered once (2e2870f) for a clean run.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs — my earlier nit about the ensureFetchPromise exception-check asymmetry was addressed in e05032d. Because this touches the JSC module loader's synchronous-load path and relies on internal promise semantics (fulfillPromise bypassing the resolving-function flag pipeFrom() claimed, and the later async transpile landing harmlessly on an already-settled promise), a human look would still be worthwhile.

What was reviewed:

  • The Fetching-status branch in provideFetchForSyncLoad and all four call sites it replaces; the fall-through to provideFetch() preserves prior behavior for New/absent entries.
  • GC rooting of jsSourceCode across the ensure*Promise allocations — it's a stack local under the conservative scanner, same shape as upstream ModuleRegistryEntry::provideFetch.
  • Confirmed ALWAYS_SYNC_MODULES in src/runtime/jsc_hooks.rs contains reflect-metadata, so the in-flight-sibling test's determinism claim holds.
  • The require-cache.test.ts CI failures on build 94024 look like lane starvation (same lane timed out four unrelated leak tests); worth confirming the retrigger is clean.
Extended reasoning...

Overview

This PR adds a static helper provideFetchForSyncLoad in src/jsc/bindings/ModuleLoader.cpp and swaps four provideFetch() calls in the require-of-ESM path to use it. When require() hits a module whose registry entry is already in Fetching status (because the surrounding import graph kicked off an async transpile), the old provideFetch() silently dropped the synchronously-transpiled source; the new helper fulfills the entry's pending fetch promise directly so the subsequent loadModuleSync drain completes. Two new test files exercise the deterministic case (via the reflect-metadata ALWAYS_SYNC_MODULES fast path) and the issue's natural timing-dependent shapes.

Security risks

None identified. This is internal loader plumbing with no user-controlled input reaching new parsing or allocation. The change narrows an error path (spurious TypeError) rather than widening any acceptance.

Level of scrutiny

High. ModuleLoader.cpp is a critical path — every require() and import flows through here — and the fix depends on JSC-internal invariants that are not visible from Bun's side: that fulfillPromise bypasses the already-claimed resolving functions from pipeFrom(), that the entry's FetchSettled reaction is attached to the promise being fulfilled, and that the async transpile arriving later on an already-settled promise is dropped by PromiseFulfillWithoutHandlerJob's pending-target check. The PR description traces each of these carefully and cites the analogous existing pattern in hostLoadImportedModule's synchronous-replay block, but this is exactly the kind of change where a maintainer familiar with the module-loader rewrite (#29393) should confirm the state-machine reasoning.

Other factors

  • All prior review threads (comment-cop on comment length, CodeRabbit on test determinism and GC rooting, my exception-check nit) are resolved with commits.
  • The deterministic test's mechanism (ALWAYS_SYNC_MODULES forcing reflect-metadata to transpile on-thread) checks out against src/runtime/jsc_hooks.rs:4391.
  • There is stated interaction with #37187 (which covers a separate require-cache bug and stays open) and a related JSC-side change oven-sh/WebKit#396; a human should confirm the boundary between these is drawn correctly.
  • Build 94024 had require-cache.test.ts failures on two lanes; the author's analysis that the changed code isn't reached by those tests (CJS files return before provideFetchForSyncLoad) is plausible and the same lane timed out several unrelated leak tests, but the retrigger (2e2870f) result should be confirmed clean before merge.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Retrigger result (build 94130, same diff as e05032d): test/cli/run/require-cache.test.ts passes on both lanes that timed out in 94024, as do fs.test.ts, inspect-error-leak and the other tests from the starved asan lane, so those were lane load. 176 jobs green; the one red job is test/bake/deinitialization.test.ts on Windows 2019 x64, a null-pointer segfault in the dev server teardown fixture, which does not involve the module loader and has been reported for main separately. The remaining entries are retry-passed flakes. The diff itself is green; nothing further is planned on this branch unless review asks for changes.

registryEntry() falls back to an entry of any type, so a require() racing an
in-flight import of the same file with a type attribute fulfilled that typed
entry with default-loader output (and scanned the whole module map whenever
there was no entry at all). Look up the JavaScript-typed entry directly,
which is the one provideFetch() would create and loadModuleSync() reads.
Tests: the typed-import race, and the sync-transpiler configuration in which
the sibling's fetch promise is already settled when the require() lands.
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed 5968a2f after a closer look at provideFetchForSyncLoad:

  • registryEntry() is type-agnostic (JavaScript bucket first, then any entry for the specifier), so a require() racing an in-flight import "./mod.js" with { type: "ts" } of the same file fulfilled the typed entry with the default-loader source, and did nothing for the require(), which reads the JavaScript entry. Reproduced with an a < b > (c) fixture: the typed import ended up seeing the JS parse. The helper now uses ensureRegistered(key, Type::JavaScript) and acts only on that entry (New: provideFetch; Fetching + pending: settle; otherwise nothing). That also drops the full module-map walk registryEntry() did on every first-time require() of an ESM file, where no entry exists yet.
  • The "Fetching but already settled" outcome had no test. Added one that runs the existing fixture with BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1; I checked that removing the pending check makes it fail on ASSERTION FAILED: status() == Status::Pending, so it pins that branch.
  • Added the typed-import race as a test; it fails against the previous revision of this branch and passes now (and on main, since main never touched the typed entry).

Branch is merged with current main; require-esm-*, require.test.ts, issue/24387, issue/33180 and the mock-module files pass locally. Description updated.

Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

On the typed-require point from the review of 5968a2f: agreed, and it is pre-existing on both sides of the require(esm) path (the old provideFetch() calls here and functionEsmLoadSync both use the JavaScript-typed entry, so a require(id, { type }) of an ESM file has always shared an entry with plain imports of it). 9ac5986 drops the comment that claimed otherwise. Threading the type attribute through $requireESM / functionEsmLoadSync / this helper is a separate change and is being tracked separately; this PR keeps to the in-flight fetch bug. The direction this PR does affect, a plain require racing a typed import, is what the third test covers.

Current head 9ac5986: provideFetchForSyncLoad addresses the JavaScript-typed entry directly via ensureRegistered (5968a2f; registryEntry() falls back to an entry of any type for the specifier and scans the map when there is none), with tests for the typed-import race and for the sync-transpiler configuration where the sibling's fetch promise is already settled. All five tests across the two files pass on this head; on the release build the in-flight test fails and the other two in that file pass, as intended, since they guard the new branches rather than reproduce the original bug.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and found no bugs; the two nits I left earlier (exception-check symmetry, comment wording) were addressed in e05032d and 9ac5986. Because the fix reaches into JSC module-loader internals — settling a pipeFrom()-claimed fetch promise via fulfillPromise under a SynchronousModuleQueue — a maintainer sign-off would still be worthwhile.

What was reviewed:

  • provideFetchForSyncLoad state handling (New / Fetching-pending / Fetching-settled / Fetched) against ModuleRegistryEntry semantics and the four call sites it replaces.
  • GC safety of the JSSourceCode* local across ensureRegistered/ensure*Promise — conservative stack scan keeps it live; matches upstream provideFetch.
  • Typed-entry keying: ensureRegistered(..., Type::JavaScript) vs. the earlier registryEntry() fallback; test 3 pins the typed-import direction.
  • Transitive in-flight ESM dependencies of the required module — ruled out (each dependency's own fetch goes through hostLoadImportedModule, not this helper).
Extended reasoning...

Overview

The PR adds provideFetchForSyncLoad in src/jsc/bindings/ModuleLoader.cpp and swaps four moduleLoader()->provideFetch(...) calls in the synchronous require(esm) path (two virtual-module branches, the isolated-cache branch, and the fresh-transpile branch in fetchCommonJSModuleNonBuiltin) for it. The helper looks up the JavaScript-typed registry entry via ensureRegistered, and where the old code silently no-op'd on a Fetching entry, it now fulfils that entry's still-pending fetch promise with the source the synchronous transpile just produced, so loadModuleSync's drain has work to run and the load completes instead of surfacing as "require() async module". Two new test files cover a deterministic reproduction (via ALWAYS_SYNC_MODULES's reflect-metadata), the settled-but-unprocessed state under BUN_FEATURE_FLAG_DISABLE_ASYNC_TRANSPILER=1, a typed-import race, and the two natural shapes from #33180.

Security risks

None identified. The change is confined to how already-transpiled source is handed to JSC's module registry; no new user input is parsed, no boundaries are crossed, and the entry keying is narrower than before (typed lookup instead of registryEntry()'s any-type fallback).

Level of scrutiny

High. This is C++ in the module loader, on the hot path for require() of ESM, and it relies on a specific JSC-internal contract: fulfillPromise bypasses the alreadyResolved guard that pipeFrom() set, and the later async transpile landing on the now-settled promise is dropped by PromiseFulfillWithoutHandlerJob's pending-target check. The PR description traces this precisely and cites the identical pattern in hostLoadImportedModule's synchronous-replay block, and the behaviour is pinned by a test that removes the pending check and observes the debug assertion fire. That is convincing, but it is exactly the kind of change where a maintainer who owns the module-loader rewrite (#29393) should confirm the contract holds — particularly the interaction with innerModuleLoading's assumptions the description flags as a separate WebKit-side concern.

Other factors

All prior review threads are resolved: CodeRabbit's determinism concern (fixed via the reflect-metadata fixture), its GC-rooting concern (withdrawn — stack local under conservative scan, matching upstream provideFetch), the comment-cop length flags and my comment-wording nit (9ac5986), and my exception-check-symmetry nit (e05032d). CI on the retriggered build 94130 was green for the touched paths; the one red job was an unrelated Windows dev-server teardown segfault. The remaining open inline (my typed-require note) is explicitly non-blocking and pre-existing on main. Given the depth of the JSC interaction, deferring to a human is the right call rather than auto-approving.

@leolomi

leolomi commented Sep 6, 2026

Copy link
Copy Markdown

Independent verification on a real app: NestJS 12 (ESM-only) with pino-nestjs (CommonJS, does require("@nestjs/common") at load), same regression since 1.3.14 as #33180.

Build from this PR (bunx bun-pr 37185, darwin-aarch64, 9ac5986) vs Bun 1.4.2, with our workaround (an explicit import '@nestjs/common' first) removed:

  • integration suite under bun test, 18 files: 1.4.2 fails all 18 with require() async module ".../@nestjs/common/index.js" is unsupported; this PR passes 439/439.
  • two-line reproduction, import '@nestjs/core' then a module that loads pino-nestjs, 20 runs each: 1.4.2 fails 20/20, this PR 0/20.

Adds a fourth affected stack to jose/jwks-rsa, eventsource and uncrypto. Happy to test another build if needed.

@robobun

robobun commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

Status after #43022 (WebKit bump to 000c48997255, merged as b52d513).

The bump includes oven-sh/WebKit#662. Its second part fixes this bug inside JSC. When a synchronous load finds a registry entry in Fetching with a pending fetch promise, JSModuleLoader::loadModule now calls the new JSModuleLoader::fetchSynchronously(). That function runs the fetch hook again and settles the fetch promise of the entry. provideFetchForSyncLoad in this PR does the same job from the require() side. The lookup in loadModule is typed (getRegisteredMayBeNull(specifier, type)), so a typed import keeps its own entry.

I ran the two test files of this PR without the ModuleLoader.cpp change:

Build Result
Release canary before the bump (1.4.3-canary.1+c6b7fcb5b) 3 of 5 tests fail with TypeError: require() async module "..." is unsupported
Debug build with ASAN of b52d513 5 pass in 10 of 10 runs
Release canary after the bump (1.4.3-canary.1+b52d51348) 5 pass in 10 of 10 runs

No commit between c6b7fcb and the bump changes the module loader sources.

The repros from #33180 agree. The plain-esm repro with a 252 KB index.js fails 44 of 50 runs before the bump and 0 of 50 after. The eventsource@3.0.7 repro fails 0 of 50 runs after.

The source change in this PR is no longer needed to fix #33180. It still does one thing: it reuses the source that fetchCommonJSModule already transpiled. On main, provideFetch() drops that source for a Fetching entry, and fetchSynchronously() runs the fetch hook, which transpiles the file one more time.

main has no test for this behavior, because #43022 changed only scripts/build/deps/webkit.ts. The two test files here give that coverage. If a maintainer wants only the tests, say so and I will drop the ModuleLoader.cpp change and merge main.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun 1.3.14 regression: require() async module error for jose via jwks-rsa/firebase-admin (works fine on 1.3.11)

3 participants