Skip to content

module loader: fix false "require() async module" for a module racing a concurrent import - #33184

Closed
robobun wants to merge 5 commits into
mainfrom
farm/e6efe5d7/require-esm-concurrent-import-race
Closed

robobun wants to merge 5 commits into
mainfrom
farm/e6efe5d7/require-esm-concurrent-import-race

Conversation

@robobun

@robobun robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

Fixes #33180

Problem

Under Bun 1.3.14+, require() of a plain ESM module (no top-level await) throws a false positive when that module is concurrently loaded by a static import:

TypeError: require() async module ".../node_modules/jose/dist/webapi/index.js" is unsupported. use "await import()" instead.
      at <anonymous> (.../node_modules/jwks-rsa/src/utils.js:1:7)

The reporter hits this on startup with firebase-admin (whose jwks-rsa does require('jose')) while the app also imports jose directly. jose's webapi entry has zero top-level await, so Node and Bun 1.3.11 both return it; 1.3.14+ throws. It regressed in the module-loader rewrite (#29393).

Cause

When the require() lands, a concurrent static import has already created the registry entry and kicked off an async fetch on the transpiler thread, so the entry is at status Fetching with no module record yet. require(esm) calls JSModuleLoader::loadModuleSync, whose top-level loadModule reuses the existing pending async fetch promise (status != New) instead of forcing a synchronous fetch. The synchronous-fetch handling only exists in the dependency path (hostLoadImportedModule), not this top-level path, so the drain completes nothing and the load stays pending, which functionEsmLoadSync reports as an unsupported async module.

Instrumented state at the throw (releases, synthetic repro): the entry is Fetching, record null, and fetch/module/load promises are all still pending both before and after loadModuleSync, i.e. the call advances nothing.

Fix

functionEsmLoadSync now detects an entry that is still Fetching with no record and drives the fetch synchronously (set up the synchronous module queue, re-issue loader->fetch which routes to fetchESMSourceCodeSync, fulfill the entry's fetch promise, drain the queue), mirroring the dependency-load handling in hostLoadImportedModule. The entry then reaches Fetched and the existing loadModuleSync path links and evaluates it.

A module that genuinely cannot complete synchronously (real top-level await, async onLoad plugin) leaves the fetch promise pending and still reaches the async-module throw, so the existing guardrail behavior is preserved.

Verification

  • New regression test test/regression/issue/33180.test.ts: a plain ESM module statically imported alongside a CJS sibling that require()s it. Fails on the unfixed debug build with the exact issue error on every attempt; passes with the fix.
  • Guardrails stay green: require-esm-transitive-tla, require-esm-microtask-order, issue/24387 (TLA still throws ERR_REQUIRE_ASYNC_MODULE), plus require, concurrent-dynamic-import, esModule.
  • Manual: a large synthetically-slow (sync-completable, no TLA) target went from 20/20 throws to 0/20 on a release build; a macro-task-TLA target still throws 5/5.

…ncurrent import

When a CommonJS require() targets an ESM module that a concurrent static
import already started fetching, the registry entry exists with status
Fetching and no record yet (its transpiler-thread fetch is still in
flight). loadModuleSync's top-level loadModule path reuses that pending
async fetch promise instead of forcing a synchronous fetch, so the load
never completes synchronously and require() wrongly reports the plain
(non-top-level-await) module as an unsupported async module:

    TypeError: require() async module "..." is unsupported. use "await import()" instead.

This regressed in the module-loader rewrite (#29393); 1.3.11 returned the
module. It reproduces with firebase-admin's auth path, where jwks-rsa
require()s jose while the app also imports jose directly.

functionEsmLoadSync now forces the synchronous fetch for an entry that is
still Fetching with no record, mirroring the dependency-load handling in
JSModuleLoader::hostLoadImportedModule, so the entry reaches Fetched
before loadModuleSync links and evaluates it. A module that genuinely
cannot complete synchronously (real top-level await, async onLoad plugin)
leaves the fetch promise pending and still reaches the async-module throw.
@github-actions github-actions Bot added the claude label Jul 1, 2026
@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:53 AM PT - Jul 1st, 2026

❌ @robobun, your commit 8ff98bf has 7 failures in Build #67577 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33184

That installs a local version of the PR into your bun-33184 executable, so you can run:

bun-33184 --bun

@github-actions

github-actions Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Found 2 issues this PR may fix:

  1. Async Bun plugin onLoad for .ts makes require() treat the module as async #30014 - Async Bun plugin onLoad for .ts makes require() treat the module as async — same false-positive require() async module error caused by async plugin handler putting the registry entry into Fetching status before require() resolves it
  2. Trying to import react native by striping flow types using a runtime plugin #10083 - Importing react native with a runtime plugin that uses async onLoad causes require() to throw the same false async module error

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #30014
Fixes #10083

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 0609ade7-490d-4137-b710-133e235fb4aa

📥 Commits

Reviewing files that changed from the base of the PR and between 4a6c22f and a2be884.

📒 Files selected for processing (1)
  • test/regression/issue/33180.test.ts

Walkthrough

This PR adds a synchronous recovery path in functionEsmLoadSync for a Fetching module registry entry without a record, and adds a regression test that reproduces the concurrent require() and static import race.

Changes

Sync ESM load race fix and regression test

Layer / File(s) Summary
Sync fetch completion for in-flight Fetching entries
src/jsc/bindings/ZigGlobalObject.cpp
Detects registry entries in Fetching status without a record(), temporarily swaps in a SynchronousModuleQueue, calls loader->fetch, transfers the resulting promise state into the existing fetchPromise, drains the queue, and restores the prior queue pointer.
Regression test for concurrent require()/import race
test/regression/issue/33180.test.ts
Adds makeTarget() to generate a large ESM module, in-memory CJS/ESM fixture files that trigger a concurrent require() and static import, and four spawn attempts asserting no false "async module" error and successful completion output.

Possibly related PRs

  • oven-sh/bun#29393: Modifies functionEsmLoadSync in the same file and related module-loading flow.
  • oven-sh/bun#31407: Also updates functionEsmLoadSync to address concurrent module registry state handling.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the module-loader race fix and false async-module error being addressed.
Description check ✅ Passed The description covers the problem, cause, fix, and verification, even though it uses different headings than the template.
Linked Issues check ✅ Passed The code and regression test address #33180 by fixing the concurrent import/require race and preserving the async-module guardrail.
Out of Scope Changes check ✅ Passed The PR stays focused on the module-loader race and its regression test, with no unrelated changes.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/jsc/bindings/ZigGlobalObject.cpp`:
- Around line 770-780: Shorten the comment in ZigGlobalObject::loadModuleSync to
durable local intent only. Remove the bug-history and control-flow narrative,
and keep just the invariant that a module must be driven to Fetched before
synchronous linking/evaluation, plus the fallback that genuinely async modules
(top-level await or async onLoad) remain pending and hit the existing throw
path. Keep it to at most 3 lines.

In `@test/regression/issue/33180.test.ts`:
- Around line 5-12: Shorten the regression rationale comments in the test so
they stay within the 3-line limit and avoid the long bug-history narrative.
Update the explanatory block near the top of the file, keeping only the
essential scenario around the plain ESM module, the CommonJS require() sibling,
and the concurrent transpile fetch; preserve the intent without the detailed
loadModuleSync history.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bce75e83-2f00-479e-826f-60a197d3dcb4

📥 Commits

Reviewing files that changed from the base of the PR and between d816daf and 856e46d.

📒 Files selected for processing (2)
  • src/jsc/bindings/ZigGlobalObject.cpp
  • test/regression/issue/33180.test.ts

Comment thread src/jsc/bindings/ZigGlobalObject.cpp Resolved
Comment thread test/regression/issue/33180.test.ts Resolved
@robobun

robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks, but I don't think this PR fixes #30014 or #10083. Those are a different mechanism: an async onLoad plugin handler. The plugin's onLoad returns a promise, so the module's source genuinely cannot be produced without yielding to the event loop, and require() has no synchronous way to complete it.

This PR only rescues modules that are synchronously completable (a plain on-disk module whose transpiler-thread fetch is merely still in flight). It drives that fetch via fetchESMSourceCodeSync, which cannot run an async plugin handler, so a plugin-backed module's fetch promise stays pending and it still reaches the async-module throw by design.

Verified against the exact #30014 repro on this branch, it still throws:

$ bun --preload ./preload.ts ./entry.ts
TypeError: require() async module ".../repro.ts" is unsupported. use "await import()" instead.
      at require (...)
      at <anonymous> (.../entry.ts:3:28)

So I'm leaving those two out of the description. They need separate handling for the async-plugin path (Node surfaces that case as ERR_REQUIRE_ESM/a race-condition error rather than loading it).

Comment thread test/regression/issue/33180.test.ts Resolved
@robobun

robobun commented Jul 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Update: the full run (build 67577) settled with a broad, scattered set of red lanes, and it's unrelated to this diff.

Each platform fails on a different subsystem:

  • ubuntu 25.04 x64: test/js/bun/util/v8-heap-snapshot.test.ts SIGKILL (OOM, no core)
  • alpine 3.23 x64-baseline: test/js/node/test/parallel/test-net-connect-memleak.js (gc-timing assertion)
  • darwin 26 aarch64: test/js/bun/http/bun-serve-file.test.ts + test/js/bun/http/fetch-file-upload.test.ts (sendfile) timeouts, test/js/node/http/node-http-uaf.test.ts
  • darwin 14 x64: test/js/bun/terminal/terminal.test.ts (90s timeout)
  • darwin 14 aarch64: test/js/web/websocket/autobahn.test.ts (docker-based)
  • plus a set of soft "flaky" retries (bake dev server, bun-link, napi, zlib/leak, etc.)

Different tests per platform, dominated by timeouts / OOM / GC-timing / docker, is the signature of CI infra flake rather than a code defect. This change is a narrow module-loader fix (functionEsmLoadSync); it cannot plausibly break heap snapshots, terminals, websockets, and sendfile at once, and a real regression in it would fail the same module-loading test on every platform. None of the red tests touch require/import/ESM loading, and the added regression test (test/regression/issue/33180.test.ts) is green where it ran.

I've already spent my one retrigger, so I'm not pushing more. Flagging for a maintainer: the diff is ready; a re-run on healthier agents should clear these.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found and my earlier feedback was addressed, but this directly manipulates JSC module-registry promise state on a hot require(esm) path while a concurrent async fetch is still in flight — worth a look from someone close to the #29393 module-loader rewrite.

Extended reasoning...

Overview

This PR adds a ~30-line recovery path inside functionEsmLoadSync in src/jsc/bindings/ZigGlobalObject.cpp: when require(esm) finds the registry entry already at Fetching with no record (because a concurrent static import kicked off an async transpiler-thread fetch), it swaps in a SynchronousModuleQueue, re-issues loader->fetch (which routes to fetchESMSourceCodeSync while the queue is set), transfers the resulting promise state into the entry's existing fetchPromise, drains the queue, and restores the prior queue pointer. A new regression test (test/regression/issue/33180.test.ts) reproduces the race with 4 concurrent spawn attempts.

Security risks

None identified. The change is confined to internal module-loading control flow; no user-controlled input reaches new parsing/allocation logic, and no auth/crypto/permissions surfaces are touched.

Level of scrutiny

High. The module loader is one of the most correctness-critical paths in the runtime — bugs here manifest as user startup failures or hangs. This change:

  • Directly calls fulfillPromise/rejectPromise on a registry entry's fetchPromise while the original async fetch for that same entry is still in flight on the transpiler thread. Whether the later async completion racing against an already-settled internal promise is fully benign (JSC internal promises should ignore double-settlement, but the entry's status transition and any reactions attached by the concurrent import path also matter) is the kind of question that needs eyes from whoever owns the #29393 rewrite.
  • Issues a duplicate fetch (sync path) alongside the in-flight async one — presumably the async result is discarded, but that's an implicit contract worth confirming.

The SynchronousModuleQueue push/drain/pop shape matches the four existing sites in ModuleLoader.cpp exactly, and queue.prev is restored before the RETURN_IF_EXCEPTION, so the stack-scoped queue lifetime looks correct on all exit paths.

Other factors

  • The bug-hunting system found no issues.
  • My earlier inline suggestion (use test.concurrent for the 4 independent spawn attempts) was applied in 4a6c22f.
  • The regression test follows harness conventions (tempDir, bunEnv, concurrent pipe drain, exitCode asserted last) and the PR description reports guardrail tests (require-esm-transitive-tla, issue/24387, etc.) still pass, so the TLA/async-plugin throw path is preserved.
  • CodeRabbit's comment-length nits were addressed and resolved.

Given the critical path and the subtle concurrent-fetch/promise-settlement interaction, I'm deferring rather than approving.

@robobun

robobun commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the careful read. To confirm the two contracts you flagged:

Double-settlement of the fetch promise is benign. The concurrent import attaches its async fetch to the entry's fetchPromise via pipeFrom, which sets isFirstResolvingFunctionCalledFlag. I settle it with the unguarded fulfillPromise/rejectPromise specifically so it takes effect past that flag. This is the same shape the dependency-load path already uses in JSModuleLoader::hostLoadImportedModule (its vm.m_synchronousModuleQueue && status == Fetching block), including the same comment about the flag. When the in-flight async transpile later resolves, its target is already settled, so the pipe's resolve is a no-op (JSPromise ignores settling an already-settled promise).

The fetch-settled reaction runs once. ensureModulePromise attaches a single ModuleRegistryFetchSettled reaction to the fetch promise. My fulfillPromise fires it (diverted onto the synchronous queue, run during the drain): makeModule + fetchComplete build the record and flip status to Fetched. The async completion neither re-attaches nor re-runs it, so makeModule runs once and the entry isn't double-initialized.

The duplicate transpile is discarded. The synchronous fetch (fetchESMSourceCodeSync) produces the source that's actually used; the in-flight async transpile's result targets the already-settled fetch promise and is dropped. The two reads run on separate threads and touch no shared entry state; makeModule/fetchComplete only run on the JS thread.

Guardrail is unchanged: a module that cannot complete synchronously (real top-level await, async onLoad plugin) leaves the fetch promise pending, so this block no-ops and the existing async-module throw still fires. Covered by require-esm-transitive-tla and issue/24387.

@robobun

robobun commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

Independently hit the same root cause from a different entry point and converged on the same fix (settle the Fetching entry's fetch promise through a synchronous fetch under a SynchronousModuleQueue before loadModuleSync), so not opening a second PR. Branch with the diff, in case anything here is useful: farm/73b69977/require-esm-fetching (main...farm/73b69977/require-esm-fetching).

Two things from that branch that might be worth folding in:

A deterministic, race-free reproduction. The shape in test/regression/issue/33180.test.ts depends on the large target's off-thread transpile still being in flight when the sibling's require() runs, and it hedges with 4 concurrent attempts. The same Fetching state can be forced purely by the entry graph's own evaluation order, with no timing dependence and no large file: the entry imports a one-line CJS starter whose require chain reaches a require(esm) of a module the entry also imports. The starter's synthetic-source body runs at the first transpiler-store delivery, before the other entries' results have been delivered, so the target is Fetching on every run. 20/20 failures on an unfixed build; Node 26 runs it fine.

// entry.mjs
import "./starter.cjs";
import a from "./a.cjs";
import { check } from "./b.mjs";
if (a.late !== "late" || check() !== "late") throw new Error("lost");
// starter.cjs
require("./a.cjs");
// a.cjs
exports.early = "early";
require("./b.mjs");   // <- throws require() async module "b.mjs" before the fix
exports.late = "late";
// b.mjs
import a from "./a.cjs";
export function check() { return a.late; }

The ESM<->CJS cycle contract. Once the false error is gone, b.mjs re-enters the still-evaluating a.cjs mid-require, so its default binding is taken from a partial module.exports. The test on my branch additionally asserts that binding is the live object (a.late === "late" through the ES importer after a.cjs finishes), matching Node, which issue/33180.test.ts doesn't exercise. That test lives next to the other functionEsmLoadSync Pending-path coverage in test/js/bun/resolve/require-esm-transitive-tla.test.ts.

@robobun

robobun commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

Verified another independently reported shape of this race against current main (f972c28): a CommonJS module evaluated eagerly during module-graph loading, requiring several ESM siblings of its importer while they are still on the transpiler pool.

// e1.mjs..e4.mjs: plain ESM, no top-level await
// k.cjs:  const a = require("./e1.mjs"); ...through... require("./e4.mjs");
// m0.mjs: import "./e1.mjs"; ...; import k from "./k.cjs";

With 1000-export siblings, an unpatched debug build at f972c28 fails 10/10 with TypeError: require() async module "/tmp/ccgraph/e1.mjs" is unsupported. use "await import()" instead. (which of e1..e4 throws varies with transpiler job completion order; the small-file variant also fails most runs on the 1.4.0-canary.1 release build). With this PR's ZigGlobalObject.cpp change applied onto that same commit, 15/15 runs pass and print the expected sum, so consecutive require() calls on multiple still-Fetching entries each complete synchronously.

@robobun

robobun commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator Author

Cross-link: #37187 fixes the same underlying gap (a top-level synchronous load reusing a mid-flight entry's pending fetch promise) reached from a different, deterministic reproduction: a CJS module inside a require()d ESM graph requiring an in-flight ESM sibling, which ended in a CommonJS module being silently skipped (no error, exit 0) rather than the TypeError. I verified this PR's change alone also fixes that silent-skip repro, so the two pre-settle hunks in functionEsmLoadSync overlap and will conflict textually; whichever lands second rebases that hunk. #37187 additionally fixes the require-cache half (a replayed makeModule fabricating an empty module after eviction, which swallows real module errors), which is independent of this PR.

@robobun

robobun commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator Author

Another report of the same race: #37557 (Nuxt 4 + NestJS, uncrypto resolved via its bun export condition, closed as duplicate of #33180).

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Consolidated into #37185, which fixes the same race one layer earlier: fetchCommonJSModule settles the mid-fetch entry with the source it has already transpiled synchronously, instead of re-fetching from functionEsmLoadSync, so there is no third transpile and the virtual-module and isolated-cache paths are covered as well. #37185 now also carries test/regression/issue/33180.test.ts with this PR's issue-shaped case (four large siblings required in a row, from the comment thread here) plus the deterministic starter.cjs cycle shape from the farm/73b69977/require-esm-fetching branch linked above. Verified on a debug build of current main that both cases still fail without the fix and pass with #37185's change. Closing.

@robobun robobun closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun 1.3.14 regression: require() async module error for jose via jwks-rsa/firebase-admin (works fine on 1.3.11)

1 participant