Skip to content

Module mocks: decode the key of a builtin's original at the builtin lookup - #48305

Open
robobun wants to merge 2 commits into
mainfrom
robobun/7cb718ad/import-original-loaded-builtin
Open

robobun wants to merge 2 commits into
mainfrom
robobun/7cb718ad/import-original-loaded-builtin

Conversation

@robobun

@robobun robobun commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • importOriginal() in the factory of a mocked builtin rejects with ENOENT reading "node:fs". It needs a loaded builtin and no registered mock. Example: mock.module("node:fs", factory) below the import.
  • The original has the key node:fs?actual. runVirtualModule (src/jsc/bindings/BunPlugin.cpp:5280) and fetchBuiltinModuleWithoutResolution (src/jsc/bindings/ModuleLoader.cpp:656) decode it only while virtualModules exists. The first registered mock allocates that map. mockModule() calls the factory of a loaded module before it registers the mock.

Fix

  • fetchBuiltinModuleOrItsOriginal wraps both call sites of Bun__fetchBuiltinModule. When the key misses, it checks for the query and asks for the builtin. It has no gate, so Bun.plugin.clearAll() and bun run work.
  • runVirtualModule renames the key only for an onLoad answer. Its rewrite before the lookup made require("bun:sqlite") in its own factory throw Circular import.
  • builtinModuleOfOriginalKey() uses withoutOriginalModuleQuery(), so node:os?actual&actual decodes.
  • Verified: test/js/bun/test/mock/mock-module.test.ts: 8 new tests fail on main. Also plugins.test.ts. Self-reviewed: 11 concerns raised, 9 addressed, 2 in part (Notes).

Background

  • importOriginal returns the real module behind a mock (bun test: Vitest compatibility #44833, unreleased). For a builtin, Bun loads it again under the key <builtin>?actual.
  • Considered a decode above the gate in runVirtualModule: it leaves bun run and CommonJS failing.

Downsides

  • Each load of a module that is not a builtin costs 14 more instructions. A gate on isBunTest || hasVirtualModules() avoids 12 of them and keeps bun run failing.
  • Binary: +1,024 bytes of text.
  • Under bun run, an onLoad plugin in the node namespace is not asked for that original.
Notes

Origin. No GitHub issue. #48218 lists this failure under "Not changed here". #44833 is in no release, so this should merge before the first release that has it.

Main and this PR, debug builds of main 2596ac5 and of this branch:

Case main this PR
mock.module("node:fs", async importOriginal => ...) below the import, one file ENOENT reading "node:fs" pass
hoisted vi.mock and a --preload that imports the builtin, also with --isolate and --isolate=modules ENOENT pass
hoisted vi.mock in a second test file, after one that imported the builtin ENOENT pass
vi.doMock in a test. A factory that calls require("./fake.cjs") first. A factory that does not await importOriginal() ENOENT pass
os with no prefix, bun:sqlite, node:buffer, node:process ENOENT pass
importOriginal() after Bun.plugin.clearAll(): in a pending factory, or kept and called later ENOENT pass
two mocks of one builtin in one tick the debug build aborts: ASSERTION FAILED: m_status == Status::Fetching pass
two mocks of one builtin a microtask apart, with a mock registered before (key node:os?actual&actual) ENOENT pass
require("bun:sqlite") in the factory of its own mock, which the first import calls Circular import: "bun:sqlite" is imported by a module that its own mock factory loads while it runs pass
bun run: importOriginal(), and vi.importActual() of a mocked loaded builtin ENOENT pass
bun run: require("node:os") in a pending registered factory (works in 1.4.3) Failed to fetch builtin module pass
a file named node:fs in the working directory it runs and is the original not read

These pass on main too: a run where another mock is registered first, and a factory that spreads vi.importActual(), jest.requireActual(), require() or import().

The three causes and their hunks.

  • No table of mocks, under bun test: before the first mock is registered, or after Bun.plugin.clearAll(). Fixed by the decode on a miss of the builtin lookup.
  • bun run: the builtin lookup runs before runVirtualModule, so the key was never a builtin. Fixed by the same hunk.
  • A registered mock, and require() of a builtin whose id has no node: prefix, in a factory that the first import calls. runVirtualModule rewrote the caller's key to bun:sqlite, and the mock lookup in fetchBuiltinModuleWithoutResolution then found the running mock. Fixed in runVirtualModule: it renames the key only after an onLoad plugin answers, which is after the mock lookup.

Why the decode reads no state of running factories. lldb at importOriginalModule: when the factory calls require("./fake.cjs") before importOriginal(), runningModuleMocks.size() is 0 and virtualModules is null. keyOfImportWhileModuleMocksRun() detaches a running mock that is not registered yet. A table made before the factory runs does not cover Bun.plugin.clearAll().

Measurements. Release builds (LTO) of main 2596ac5 and of this branch.

  • Miss, instructions of one thread from the entry of a function to its return, callees included (gdb single steps). fetchBuiltinModuleWithoutResolution for require() of a file: 603 on main, 615 here. runVirtualModule with no plugin and no mock: 232 on main, 234 here. The ES module site runs the same 12 instructions at the lookup (llvm-objdump). 0 calls, 0 lock instructions, no allocation. A key that ends in l runs 5 more.
  • Hit: the same instructions as main (call Bun__fetchBuiltinModule, test, je). The helper is inlined. The decode is one function out of line.
  • 1,000,000 calls of require("node:os"), user-mode instructions per call from a perf_event_open counter, median of 5 runs. bun run: 2,679 on main, 2,679 here. In a bun test file after one mock.module(): 3,364 on main, 3,251 here. Main decoded the key at every call there.
  • 4,000 loads of one-line files (2,000 import(), 2,000 require()), whole process, median of 7 runs: 553.50 M instructions on main, 553.10 M here. Runs of one binary differ by 0.7 %. The counts above predict a difference of 56,000.
  • Calls of Bun__fetchBuiltinModule, lldb breakpoint counts on debug builds: 1,002 on both for 1,000 loads with no mock. 4 on main and 5 here for one importOriginal() of a builtin.
  • Size, from size: text 137,931,057 to 137,932,081. Data and bss are equal. By symbol, from nm -S: fetchESMSourceCode<true> +429, fetchESMSourceCode<false> +350, fetchBuiltinModuleWithoutResolution +45, runVirtualModule with OnLoad::run +225, esmExportsBeforePatches -169, and the new builtinOfOriginalModuleKey is 330.
  • Host functions +0. extern "C" symbols +0.
  • perf, valgrind, strace and bloaty are not in the build container. test/bundler/compile-aot-size.test.ts skips its 42 tests on a local build.

Behaviour outside the symptom.

  • bun run: the builtin lookup runs before plugins. An onLoad plugin in the node or bun namespace is not asked for the original of a builtin. It is not asked for the builtin either. The new test pins asked: [].
  • A <builtin>?actual key that reaches the loader with no mock now loads the builtin. A replaced Module._resolveFilename or an onResolve plugin can return one. The resolver still rejects such a specifier in a script: No such built-in module: node:fs?actual.
  • A module that build.module() or a mock registered under exactly such a key. Outside bun test it keeps its answer: the new test uses node:zlib?actual, and fails without that check. Under bun test, import() asks it first and require() of such a node: name gives the builtin, both as on main. A new test pins both.
  • The second module keeps the builtin's name as the name of its source, as on main when a mock was registered.
  • bun test and an onLoad plugin that serves a builtin: the plugin is asked first, is shown the same path, and what it returns for the original has the builtin's name in stack frames and coverage rows. On main that name held only once a mock was registered. In the first mock of a run the frame was node:os?actual:1:93. It is node:os:1:93 in both cases now, and a new test pins both.

Not changed here. All four behave the same on main and with this PR.

Self-review. 11 concerns. Addressed: a test that reaches node:os?actual&actual, one strip function for the query, tests that call what the factory was given after the mock applies, an assertion that could not fail, tests for node:buffer, node:process and require("bun:sqlite") under bun run, the bun run plugin order named and tested, the three causes named, a one-line comment in place of four, the counts and the gate question, derived constants. In part:

Review of fee07ab. The review left three optional findings. All three are addressed in d72c5ea:

  • The coverage name of an original that a plugin serves. runVirtualModule renames the key again where a plugin answered.
  • require() of a key that something is registered under threw Failed to fetch builtin module under bun test. The check for a registered key now applies outside bun test only, where runVirtualModule is asked after the builtins.
  • esmExportsBeforePatches() built the key of the original and decoded it again. It asks isBuiltinModuleKey() now, which gives the answers of main for every specifier.

Open PRs near this code. #48178 and #47819 edit the findModuleMock line in fetchBuiltinModuleWithoutResolution. #48218 edits withoutOriginalModuleQuery(). git merge-tree of d72c5ea with each of the three heads is clean.

Suites run with the debug build. mock-module.test.ts (315 pass, 1 todo), plugins.test.ts (270), mock-hoist.test.ts (134), vitest-flavor.test.ts (191), mock-fn.test.js (848), builtin-esm-lazy-exports.test.ts (25), import-attributes.test.ts (58), transpiler-cache.test.ts (34), isolation.test.ts (151 pass, and 1 test that times out at 5 s on main too), mock-module-non-string.test.ts, mock-module-resolve-log.test.ts, 6874, 6879, missing-module.test.js, disallow-code-generation-from-strings.test.ts, regression tests 11664, 12548, ENG-24434. The new tests also pass with BUN_JSC_validateExceptionChecks=1. test/aot/mock-actual-builtin.test.ts fails in setters.test.mjs on both builds.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/test/mock/mock-module.test.ts

…ookup

The original of a mocked builtin is the builtin loaded once more, under the
key `<builtin>?actual`. Two places turned that key back into the builtin:
runVirtualModule() and fetchBuiltinModuleWithoutResolution(). Both did so only
while a table of mocks existed.

Three failures came from that:

- A mock of a loaded builtin calls its factory before the mock is registered.
  With no table yet, or after Bun.plugin.clearAll(), importOriginal() rejected
  with `ENOENT reading "node:fs"`.
- Outside `bun test` the builtin lookup runs before runVirtualModule(), so the
  key was never a builtin there.
- runVirtualModule() rewrote the caller's key. require("bun:sqlite") in the
  factory of its own mock then found that mock and threw `Circular import`.

The two call sites of Bun__fetchBuiltinModule now share one helper. When the
key misses, the helper checks it for the query and asks again for the builtin.
It is not gated on `bun test` or on a registered mock. The only mock state it
reads is whether a module is registered under that exact key. The rewrite in
runVirtualModule() and the gated arm in fetchBuiltinModuleWithoutResolution()
are removed.

builtinModuleOfOriginalKey() takes the builtin's name from
withoutOriginalModuleQuery(), so it also accepts the `&actual` that a copy of
the original carries.
@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 2596ac5 and d72c5ea.

Files selected for processing: 3

  • src/jsc/bindings/BunPlugin.cpp
  • src/jsc/bindings/ModuleLoader.cpp
  • test/js/bun/test/mock/mock-module.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added the claude label Nov 9, 2026
@robobun

robobun commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

How this was reproduced, on a debug build of main (2596ac5):

// fs.test.ts
import { expect, mock, test } from "bun:test";
import * as fs from "node:fs";

const patched = mock.module("node:fs", async importOriginal => ({ ...(await importOriginal()), readFileSync: () => "mocked" }));

test("importOriginal of a loaded builtin", async () => {
  await patched;
  expect(fs.readFileSync("x")).toBe("mocked");
});
  • bun test ./fs.test.ts on main prints error: ENOENT reading "node:fs" and the test fails.
  • The same file passes on main when another module mock is registered first.
  • The same file passes with this branch.

PR: #48305

CI on d72c5ea (build 139138, at 06:10 UTC on 2026-11-09)

  • 184 jobs passed. This includes every test lane on Linux, on Windows and on macOS aarch64.
  • test/js/bun/test/mock/mock-module.test.ts passed on each of those lanes. On debian 13 x64-asan: 315 pass, 1 todo, 0 fail.
  • The 4 darwin any x64 - test-bun jobs did not start, because no agent took them. The same 4 jobs did not start on the other builds of the same hours, for example 139137 and 139139.
  • 4 test files failed one time and passed on the retry: test/aot/heavy-test-files-run-apart.test.ts, test/napi/napi.test.ts, test/js/bun/util/filesink.test.ts and test/js/web/fetch/fetch-leak.test.ts. The diff touches none of them.

No change to the diff is pending. The PR is ready for a maintainer to review.

@robobun

robobun commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:29 AM PT - Nov 9th, 2026

✅ @robobun, your commit d72c5ea61d008d47f9e42391cd2a30984fd979b9 passed in Build #139138! 🎉


🧪   To try this PR locally:

bunx bun-pr 48305

That installs a local version of the PR into your bun-48305 executable, so you can run:

bun-48305 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked the lifetime of the original BunString that specifier/builtinKey is rebound to in both callers of fetchBuiltinModuleOrItsOriginal — it is a function-scope local whose EncodedSlice points at the static builtinModuleKeys literal, so the ZeroCopy moduleKey view in fetchESMSourceCode does not dangle; and endsWithOriginalModuleQuery branches on is8Bit() before reading bytes. The new tests were not run as part of this review.

Extended reasoning...

The change rewires the two Bun__fetchBuiltinModule call sites (CJS require and ESM fetch) through a helper that decodes a <builtin>?actual key on a miss, removes the specifier rewrite in runVirtualModule, and broadens builtinModuleOfOriginalKey to strip any &actual/&spy tail; it touches no security-sensitive surface. Three confirmed findings are posted inline (coverage naming under an onLoad plugin, a registered virtual module under the literal ?actual key now failing on the CJS path, and spy/patch mocks under a ?actual key skipping their exports snapshot), so a human look is already signalled; this note only records the pointer-lifetime and encoding checks that were examined and ruled out.

Comment thread src/jsc/bindings/BunPlugin.cpp Resolved
Comment thread src/jsc/bindings/ModuleLoader.cpp Resolved
Comment thread src/jsc/bindings/BunPlugin.cpp Resolved
…erves

runVirtualModule() again names what an onLoad plugin answers for
`<builtin>?actual` by the builtin, as it did while a mock was registered. It
now does so for the first mock of a run too, so the name no longer depends on
the table of mocks.

Under `bun test` a key that something is registered under is decoded where
only the builtins are asked: require() of such a `node:` name gives the
builtin, as before. Outside `bun test` the registered module keeps its answer.

esmExportsBeforePatches() asks isBuiltinModuleKey() about the mock's own
specifier. It no longer builds the key of the original to decode it again.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the updated PR (through d72c5ea) and found no bugs; the follow-up commit addresses the three inline comments from my earlier review. Because this changes the gating of the builtin lookup in the module loader and the rename step in runVirtualModule, a human look is still worthwhile.

What was reviewed:

  • Lifetimes: original in both call sites holds a view of a static builtin-key literal, and specifier/builtinKey only point at it inside the calling function; createCommonJSModule and the registry still use the unrewritten key.
  • The rename in the fallback lambda runs after onLoadPlugins.run with no check macro inside the lambda, and builtinOfOriginalModuleKey enters no JS; endsWithOriginalModuleQuery reads span8() of the WTFStringImpl only, so the "makes no String" claim holds.
  • A &spy-suffixed original of a builtin is still not decoded at the builtin lookup, as on the base branch (pre-existing, not introduced here).
  • Not run: the new fixtures in mock-module.test.ts and BUN_JSC_validateExceptionChecks=1.
Extended reasoning...

The change touches src/jsc/bindings/ModuleLoader.cpp and src/jsc/bindings/BunPlugin.cpp, moving the "?actual" to builtin translation into a helper used by both the CommonJS and ESM builtin lookups without the bun-test/registered-mock gate, and moving the specifier rename in runVirtualModule into the onLoad fallback; it adds about 360 lines of subprocess fixtures. It touches no auth, crypto, or injection surface. The second commit was pushed after my earlier review and resolves each of the three inline comments (plugin-served originals keep the builtin's name, esmExportsBeforePatches uses isBuiltinModuleKey, and the virtual-module gate is skipped under bun test). No CODEOWNERS entry covers the changed files and the hunt ran dry, but the module loader is core runtime with a new check on every non-builtin load and subtle test-vs-run gating, so a maintainer should still look rather than relying on automated approval.

@robobun

robobun commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

The review above lists two items as not run. Both were run locally on d72c5ea, with debug builds (ASAN):

  • The 8 new tests in test/js/bun/test/mock/mock-module.test.ts pass on this branch. All 8 fail on a debug build of main (2596ac5).
  • The same 8 tests pass with BUN_JSC_validateExceptionChecks=1. The test harness passes that variable to the child processes that the tests spawn.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant