Skip to content

watcher: count node_modules from the package folder that holds the start folder - #46910

Draft
robobun wants to merge 5 commits into
robobun/c6f8fb9b/watch-node-modules-segmentfrom
robobun/6daa48d1/watch-project-inside-node-modules
Draft

robobun wants to merge 5 commits into
robobun/c6f8fb9b/watch-node-modules-segmentfrom
robobun/6daa48d1/watch-project-inside-node-modules

Conversation

@robobun

@robobun robobun commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Draft, stacked on #46809. Its bun_paths commits are older than #46909.

Problem

  • A project that runs from node_modules/<pkg> gets no reload for its imports under --watch and --hot, and bun test --watch runs nothing again.
  • The cause: the watcher gates take every path with a node_modules component for a package path (ImportWatcher::add_file, Watcher::is_eligible_directory, check_command::watching).

Fix

Background

  • The Watcher has three owners: the runtime, bun build --watch and the dev server.
  • Considered a (root, path) test at each gate: it finds the package folder again per hit.
  • Considered a flag from the resolver: it would apply to every process that starts in a package folder.

Downsides

Notes

Why a draft. #46809 has an open thread on ImportWatcher::add_file: decide a package file by the spelling of its path, or carry the verdict from the resolver. This change builds on the first answer. #45594 and #42749 add other entry points to ImportWatcher with a whole-path test of their own. They need to ask node_modules_scope() for this change to hold after they land.

#46809 has moved since this draft was pushed. It now also takes a path component named node_modules.nosync for a package store (ec504f6), and it has a merge of main. This draft is older than both and conflicts with them. Its rebase follows when #46809 lands: NodeModulesScope must then match that name too.

#46909 has moved too. The rebase has these steps from it:

  • NodeModulesScope has a lifetime parameter now. The field of Watcher and the return type of VirtualMachine::watcher_for_threads become NodeModulesScope<'static>.
  • Main has a new caller of the watcher that this draft does not know: bun run --aot --watch (WatchedFiles in src/runtime/cli/run_compiled.rs). It calls Watcher::init with two arguments, and add_read_file (line 1320) has the whole-path test that check_command::watching had. In node_modules/my-pkg, an edit to an import does not restart the program there. It must ask node_modules_scope() too.

Repro. bun 1.4.3-canary.1 (367d939), main 179664e and #46809 (9c6c8ee), Linux x64.

mkdir -p t/node_modules/my-pkg && cd t/node_modules/my-pkg
printf 'import "./util.js";\nsetInterval(() => {}, 1000);\n' > index.js
printf 'console.log("v1");\n' > util.js
bun --watch index.js     # prints v1
# other shell: printf 'console.log("v2");\n' > util.js    -> no reload, no message

An edit to index.js reloads: the entry is added by path after the first run, with no rule. The same project in t/packages/my-pkg reloads. node --watch 26.3.0 and tsx watch 4.23.15 reload in this layout, nodemon 3.1.14 does not.

The rule is NodeModulesScope of #46909, built once in Watcher::init from the folder bun started in. The project folder is compared byte for byte on POSIX. On Windows either separator and either case of a letter match.

Start folder Project
no node_modules component none: every path is tested whole
node_modules, node_modules/@scope, node_modules/.store none: every folder there is a package
node_modules/my-pkg, node_modules/my-pkg/src node_modules/my-pkg
node_modules/.bun/my-pkg@1.0.0/node_modules/my-pkg that folder
node_modules/.store/<id>/package the start folder

Where the rule is read.

  • ImportWatcher::add_file: the module loader offers each file it reads. It declines a package file before the watcher mutex.
  • The file-loader arm in jsc_hooks.rs: it asks first, because on kqueue it opens a descriptor before add_file.
  • Watcher::is_eligible_directory: the folder of each added file, and each folder in which a resolver lookup failed. bun build --watch gets it through the same Watcher::init call.
  • check_command::watching: VirtualMachine::watcher_for_threads hands the scope out with its closure. A path with /node_modules/ in a start folder outside a package folder returns with no join, as before.
  • Not given the rule, as before: the entry, the files of bun test --changed, the dev server's own directory watches.
  • bun_paths loses the two root-less predicates of watcher: skip packages by a node_modules path component, not by the letters #46809. Their three callers ask the scope.

What the tests pin (watch.test.ts, block "a project in a package folder of node_modules"). A negative claim is read from BUN_WATCHER_TRACE or from open descriptors, never from a timeout.

  • --watch in node_modules/my-pkg, --hot in node_modules/@scope/my-pkg and in node_modules/.store/<id>/package: an edit to an import reloads. A nested package, a hoisted package, a file added to node_modules and the sibling folder my-pkg-2 see no event.
  • In the same cases, a second process starts in node_modules, node_modules/@scope and node_modules/.store: only the entry is watched.
  • --hot: a file of the file loader, a module loaded by require, a module saved by rename.
  • --hot after process.chdir(): a program that moves into a package folder does not get that package watched, and a project that moves out of its package folder stays watched.
  • bun test --watch in the physical folder of the isolated linker: an edit to an import and an edit to the test file.
  • --watch --check: a fix in lib/imported.ts, which only the type checker has read, is checked again, and the declarations of a package see no event.
  • --watch, started in my-pkg/src: an edit to ../lib/util.js reloads. Not on Windows: that watcher takes no file outside the folder bun starts in, for any project (Bun doesnt pick changes in monorepos packages #9547).
  • bun build --watch: a save by rename rebuilds, and the folder of the project reports it.
  • --hot: a file that is added to a folder in which a lookup failed is found on the next reload.
  • The dev server does not watch that folder, the runtime does. This pins the rule per owner.

Measurements, Linux x64, base #46809 (9c6c8ee):

  • Gate cost per call against the predicate of watcher: skip packages by a node_modules path component, not by the letters #46809 (perf_event_open, 1e6 calls, 3 identical runs, scalar memmem stub of the unit tests, so differences only). Start outside a package folder: project file 1,114 to 1,117 instructions, package file 731 to 737, project folder 804 to 807, package folder 731 to 737. Branches are equal: 280, 185, 200, 186. Start in node_modules/my-pkg: own file 843, nested package 900, hoisted package 771, the project folder 789.
  • node_modules scans, gdb tally on highway_memmem, debug builds: 4,003 for 2,000 project modules and the entry under --watch (base 4,002), 2,003 for 2,000 package modules (base 2,002), 0 with no watch flag (base 0). The one more is the scan of the start folder in Watcher::init.
  • Scopes built, gdb tally: 0 with no flag, 1 per Watcher::init with 2,001 modules, 1 with --watch --check, 1 for --hot with a dev server (0 for the watcher of the dev server).
  • size_of::<Watcher>(): 35,936 to 35,952 bytes.
  • Release binary (size): text 107,557,956 to 107,560,004 (+2,048), data +0, bss +0.
  • Start outside a package folder, release builds, inotify watches / descriptors, equal on base and here with 0 paths that differ: --watch 2,003 / 2,009, --hot 2,003 / 2,009, test --watch 5 / 13, build --watch 7 / 13, dev server 7 / 18.
  • Start in node_modules/my-pkg (the same in node_modules/@scope/my-pkg): --watch and --hot 1 to 3 watches and 6 to 9 descriptors, test --watch 0 to 3 and 8 to 11, build --watch 2 to 3 and 8 to 9. No watched path is under <cwd>/node_modules or above the package.
  • Start in node_modules/my-pkg/src with an import of ../lib/util.js: 1 to 3 watches, 6 to 9 descriptors.
  • Start in node_modules, node_modules/@s and node_modules/.store with a 192-module package: 1 watch and 6 descriptors, equal to base.
  • Dev server in node_modules/my-pkg: 0 inotify watches and 11 descriptors on base and here.
  • --watch --check with 161 files read, start outside a package folder: 4 joins and 4 watch calls (base 4 and 4, one more of each in both builds when the checker reads package.json twice). In node_modules/my-pkg: 161 joins and 4 watch calls (base 0 and 0).
  • perf, valgrind, strace and bloaty are not installed here.

Not tested. bun bake in this layout: its VM watcher takes the new rule, the watcher of its dev server does not. macOS and Windows were run by CI only. bun run rust:check-all passes for x86_64-pc-windows-msvc, aarch64-apple-darwin and x86_64-unknown-freebsd.

Other places that take such a project for a package are listed in #46909: the dev server's bundler, bun ./index.html, tsconfig extends, the extension order of the resolver, macros.

Open pull requests on the same lines. #35596 replaces only the Windows arms of Watcher.rs and does not overlap. #39832 adds a second spelling of the root on Windows, which the scope does not have yet.

Self-review, the 3 concerns that are not addressed.

  • A tracking issue for the places above. This account opens no issues.
  • A real-path root on Windows. It needs a Windows machine to run it.
  • Edits to four older tests of watch.test.ts.

Suites run on a debug build. test/cli/watch/watch.test.ts (57 pass, 1 skip), watcher-trace.test.ts, test/cli/hot/watch.test.ts, watch-many-dirs.test.ts, hot.test.ts, test/cli/test/test-changed.test.ts, test/bake/dev/html.test.ts, test/bake/dev/hot.test.ts, the source lints watcher-node-modules-needle and byte-search, cargo test -p bun_paths, cargo clippy for the four crates.


no test proof · iteration 1 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/watch/watch.test.ts

…ct's own package folder

A path with a node_modules component is a package file, except for a
project that is itself a package folder of node_modules: each of its own
paths has that component.

NodeModulesScope::below_root(start) finds the package folder that holds
the start folder (through node_modules/<name> or node_modules/@scope/<name>).
At or below that folder only the components after it count. Every other
path is tested whole, and so is every path when the start folder is
node_modules itself, a scope folder or a dot folder.
The watcher took every path with a node_modules component for a package
file. A project that runs from node_modules/<pkg> has that component in
each of its own paths, so --watch, --hot, bun test --watch and --check
watched none of its imports, and a folder of the project was never read
again after a failed lookup.

Watcher::init now takes the rule of its owner and turns it, once, into a
NodeModulesScope. ImportWatcher::add_file, the file-loader arm,
Watcher::is_eligible_directory and check_command::watching read that one
scope. The runtime and bun build --watch count node_modules from the
package folder that holds the folder bun started in. The dev server keeps
the rule of the whole path, as its bundler does.
@robobun

robobun commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:58 PM PT - Oct 25th, 2026

✅ @robobun, your commit 0bc550c9bb75f687edddf38b9f27b93afeebfe6d passed in Build #132793! 🎉


🧪   To try this PR locally:

bunx bun-pr 46910

That installs a local version of the PR into your bun-46910 executable, so you can run:

bun-46910 --bun

@robobun

robobun commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: draft. It waits for #46809, which it is stacked on, and for #46909, which has its bun_paths commits.

How it was reproduced, on bun 1.4.3-canary.1 (367d939), on main 179664e and on #46809 (9c6c8ee), Linux x64:

  1. Make a package folder: mkdir -p t/node_modules/my-pkg && cd t/node_modules/my-pkg.
  2. Write index.js (import "./util.js"; setInterval(() => {}, 1000);) and util.js (console.log("v1");).
  3. Run bun --watch index.js. It prints v1.
  4. Write console.log("v2"); to util.js. Nothing happens. bun --hot does the same, and bun test --watch does not run again after an edit.
  5. Put the same files in t/packages/my-pkg. The edit of step 4 prints v2.

With this change step 4 prints v2.

Pull request: #46910

NodeModulesScope asked resolve_path::is_parent_or_equal whether a path
is in the project folder. That helper folds ASCII case on every system
but Linux and takes a backslash for a separator everywhere, so on a
case-sensitive macOS volume a sibling package that differs by case
counted as the project.

The scope now compares the folder itself: byte for byte on POSIX, and on
Windows with either separator and either case of a letter.
… case for a start below it

The Windows watcher takes no file outside the folder bun starts in, for
any project. The --watch --check case started in my-pkg/src and waited
for a change of ../lib/imported.ts, so it could not pass there.
@robobun

robobun commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

The base branch of this PR gained a commit: has_node_modules_component now also accepts a component named node_modules.nosync (#46809, ec504f6). Main skipped that store by the letters, and the component test alone watched it.

For the rebase of this PR:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant