Repository navigation
install: link a transitive file: dependency of a local file: package - #33159
Conversation
The hoisted installer sourced a transitive Resolution::Folder from dirname(node_modules.path), i.e. the declaring package's install directory, and turned the resulting ENOENT into Success. That base is only correct when the declaring package came from an npm registry manifest (Package::from_npm keeps the file: path verbatim). A local file: package's dependency list is parsed by Lockfile::Package::parse, which stores the folder path relative to the top-level dir, so its transitive file: dependencies were resolved against a directory that never exists and silently skipped: the lockfile recorded the resolution, bun printed "N packages installed", exited 0, and the package was missing from node_modules. Branch on the declaring package's resolution: a Folder parent installs its transitive folder dependencies from the cwd (the top-level dir), and a missing source folder is now an error instead of being swallowed. npm parents keep the existing behavior, which the transitive file dependency registry tests rely on (missing-file-dep).
|
Reproduced with a three-package local project: Status: fix and tests are pushed ( CI on
The design question from @dylan-conway above is still open; this diff is the targeted installer fix either way, and I am happy to follow up with the structural |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
WalkthroughThis PR changes how Bun's installer handles transitive ChangesTransitive folder dependency install handling
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/install/PackageInstaller.rs`:
- Around line 1986-1994: The folder-resolution error in
PackageInstaller::install is using the resolved package name from
self.names[package_id as usize] instead of the user-declared dependency alias.
Update the error path in the resolution::Tag::Folder and
cause.is_package_missing_from_cache() branch to use the original dependency
key/specifier that appears in package.json (the alias like "foo"), so the
message points users to the entry they need to fix.
In `@test/cli/install/bun-install.test.ts`:
- Line 9593: The install tests are over-specifying the physical
`node_modules/lib/node_modules/nested` layout for transitive local `file:`
dependencies. Update the affected assertions in
`test/cli/install/bun-install.test.ts` to verify the behavior that matters
instead: successful install and resolution for `require("lib")` on the happy
path, and the existing missing-folder failure path, without checking for nested
folder materialization. Use the nearby install/resolve test cases and
`require("lib")` expectations to keep the tests stable across different package
layout strategies.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7197fd85-b262-4a73-8c10-41ffa12d676c
📒 Files selected for processing (3)
src/install/PackageInstaller.rssrc/install/lockfile.rstest/cli/install/bun-install.test.ts
Use the key the user wrote in package.json rather than the resolved
package name, and stop asserting the physical node_modules nesting in
the new tests: require("lib") already proves the transitive file:
dependency was materialized.
Match the file's convention instead of asserting an exactly-empty stderr (debug/ASAN builds can emit benign warnings), and normalize path separators before asserting on the error message (the stored Resolution::Folder path uses backslashes on Windows).
There was a problem hiding this comment.
Both test-portability nits from my earlier pass are addressed and I didn't find further issues, but this changes install-path resolution and error semantics in the hoisted linker based on a two-parser path-normalization invariant, so it's worth a human look from someone familiar with src/install/.
Extended reasoning...
Overview
This PR fixes transitive file: dependencies of local file: packages in the hoisted linker. It touches src/install/PackageInstaller.rs (adds a branch in the transitive-folder install path that sources from Fd::cwd() instead of dirname(node_modules.path) when the parent tree is itself a Folder resolution, and adds a new hard-fail error branch for missing local folders), src/install/lockfile.rs (new is_folder_tree_id helper mirroring is_workspace_tree_id), and adds two integration tests to bun-install.test.ts.
Security risks
None identified. No untrusted-input parsing, no path-traversal surface introduced (the folder path was already being used as an install source; only the base directory changes). The related ..-escape concern is explicitly deferred to #33106.
Level of scrutiny
Medium-high. The change is small and well-tested, but it lives in the package manager's hoisted linker — a critical path where subtle regressions silently corrupt node_modules layouts. The correctness of the fix hinges on two invariants that a maintainer should confirm: (1) Package::parse always normalizes Resolution::Folder paths relative to the top-level dir while Package::from_npm leaves them verbatim, including when the resolution is loaded from a lockfile rather than freshly parsed; and (2) Fd::cwd() is reliably the top-level dir at this point for every install entry point (bun install, bun add, bun update, filtered installs). The PR also changes user-visible behavior — a previously-silent success now hard-fails with exit 1 — which is almost certainly the right call but is a UX decision.
Other factors
I previously flagged two test-portability issues (Windows path separator in the error assertion, and expect(stderr).toBe("") under ASAN); both were addressed in c11d7af and the diff reflects the fixes. The bug-hunting system found no issues this round. The PR description is thorough, cites the relevant registry-fixture regression tests as verified byte-identical, and notes the one-line conflict with #33106. CI on the latest commit shows musl build failures and earlier Windows test failures on the pre-fix commit; the Windows failures should be resolved by c11d7af but a green run would confirm.
|
To save the human reviewer the two invariant checks: (1) The two formats survive the lockfile round trip, so a lockfile-loaded install behaves the same as a fresh resolve. The text lockfile stores the (2) |
|
@robobun is this happening because we need |
|
Not for the symptom in this PR, but you have put your finger on the real design gap right next to it, and the duplication you are describing does happen. Why this bug is not caused by duplicate registration. In the minimal repro ( But yes, the same local folder is registered more than once today. The same folder as a root dependency and as a transitive one becomes two distinct packages, one fully resolved and one an empty stub: And because the stub never reads the folder's package.json, the third level of a What routing the transitive side through the map would buy. For a dependency declared by a local I kept this PR at the installer because the enqueue change also changes the lockfile shape (the nested |
…33106) ### Problem - A root `file:` package with relative `file:` dependencies of its own fails to resolve: `error: Could not find package.json for "file:../packages/bun-types" dependency "@types/bun"`. From a lockfile: `error: refusing to install dependency shared-lib with unsafe folder path "../packages/shared-lib"`. - The escape check from #31417 (`PackageManagerEnqueue.rs`, `PackageInstaller.rs`) rejects any transitive folder path that leaves the project root, whoever declared it. Worked in 1.3.14, broke in 1.4.0. Fixes #40314. ### Fix - Both sites call `Lockfile::is_trusted_folder_dependency(id)`: the declaring package is the root, a workspace, or a `Folder` package the root or a workspace depends on directly, or a plain override names the dependency (#32452, unchanged). The `package-lock.json` migration (`npm_lock.rs`) applies the same rule. - A `file:` path in one of those package.jsons is user authored, like a root `file:` dependency, which may already point anywhere. Registry, git, and tarball packages stay constrained, as does a folder one of them ships. - The anchor is checked, not assumed: a migrated or hand-written lockfile can carry dependencies for a folder a registry package ships. - Verified: `bun-install.test.ts` (3 tests fail on main), `migration/migrate.test.ts` (1 fails on main), and the other install suites (Notes). ### Background - A `file:` dependency becomes a `Folder` package. Its lockfile path is root relative, so an outside package starts with `../`. - `bin_target_escapes_package_dir` detects a path that leaves its base directory. #31417 applied it to transitive folder paths so a registry package cannot link local directories. - Overrides come from the root package.json only. A flat rule makes every edge of that name root authored (#32452 whitelists it). A nested rule covers one edge, so `contains_name` ignores it. <details><summary>Notes</summary> This repo's own `test/` directory hits the bug on `bun update` (`bun-plugin-svelte` is a `file:` dependency outside the project and declares `"@types/bun": "../bun-types"`). Minimal repro: a root package.json with `{ "dependencies": { "plugin": "file:../packages/plugin" } }` where `packages/plugin/package.json` declares `"shared-lib": "../shared-lib"`. The two sites that picked `local_package_features` with the same three-tag match (`Tree.rs`, `PackageManagerResolution.rs`) now call `is_local_package()` too. The overlong-path check and the #32452 carve-out are unchanged. Rebase on current main (requested in the PR comments): main already has `Lockfile::get_parent_pkg_of_dependency` (#38867, returns `Option<PackageID>`), so this branch uses it instead of its own copy. Visibility follows main (`pub(crate)`). One test on main conflicts in behavior, not in text. #38333 added `rejects a nested "<field>" rule pointing at a file: path outside the project`. Its parent is `pkg-a: file:./pkg-a`, a local folder package, so this change exempts the path and the nested rule installs. The test is re-anchored on a registry parent (`baz` from the dummy registry declaring `shared: 1.0.0`), which the name-based check still rejects, with the same two error lines. A second test asserts the local-parent case installs, from package.json and from the lockfile. The same re-anchoring was already done for the two tests from #31417/#32452 that pinned the old behavior with a local folder parent. Test changes in `bun-install.test.ts`: - New: `installs transitive file: dependencies of a local file: package that point outside the project`. Runs `bun install` (fresh), `bun update`, and `bun install` from the saved lockfile with an empty `node_modules`. Asserts both transitive packages are linked under the declaring package. - New: `applies a root "<field>" file: path to a registry package's dependency` (the #32452 carve-out, with a remote parent, at both sites). - New: `installs a nested "<field>" rule pointing at a file: path outside the project for a local file: package's dependency`. - The `different name` reject test now points at an existing folder with a package.json (absolute path), so a bypassed check would link `loot` and fail the test. - Re-anchored on a registry parent: `does not install a registry package's transitive file: dependency that points outside its package`, `still rejects a registry package's transitive file: dependency that escapes when a different name is in "<field>"`, `rejects a nested "<field>" rule pointing at a file: path outside the project for a registry package's dependency`. Trust anchor: `is_dependency_of_local_package` accepts a `Folder` parent only when `is_workspace_declared_package(parent)` finds it in the root or a workspace resolution list. Without that, a lockfile with `evil (registry) -> inner (Folder, node_modules/evil/inner) -> loot (file:/outside)` was installed, because the migration writes dependencies for `inner` while bun's resolver never would. `refuses to install an escaping file: dependency that a registry package's own folder declares in the lockfile` (hand-written `bun.lock`) and `a file: dependency declared by a registry package's own folder is still skipped` (migration) cover both layers. Migration: in `npm_lock.rs` the gate at `link_package` only exempted `Root | Workspace` parents. A blanket `Folder` exemption would also migrate `o -> p: ""` in the `external-link--root` fixture (a registry spec that npm resolved to a bare folder beside `o`, which does not exist as a package), so the exemption is limited to a `Folder` parent with a `file:` spec. That is the case this PR is about: the path is written in that package.json. The existing out-of-tree migration tests are unchanged. Fail-before was checked with main's `src/install/` swapped in: the three positive tests fail with the errors above, and everything else in the file has the same result as on main (the remaining failures there are the network-dependent bitbucket/gitlab/external URL tests). Suites run with the fix: `bun-install`, `overrides`, `nested-overrides`, `bun-workspaces`, `bun-update`, `bun-add`, `bun-dedupe`, `isolated-install`. #33159 (merged) materializes transitive `file:` dependencies of local `file:` packages at install time. This PR removes the resolve-time and install-time rejection of their paths, and the positive test asserts the combined result. </details>
Repro
A three-package, all-local project where a
file:dependency has its ownfile:dependency (the standard vendored-monorepo shape):The lockfile records the resolution (
"lib/util2": ["util2@file:vendor/util", {}]), butnode_modules/lib/node_modulesis left empty. The install reports success, and every downstream consumer (runtimerequire,--frozen-lockfilereinstalls, lifecycle scripts for that package) fails in confusing ways. The isolated linker handles the same project correctly; only the hoisted (default) linker is affected.Cause
Resolution::Folderpaths are produced by two parsers with different bases, and the hoisted installer only handles one of them:Lockfile::Package::parse(used for the root, workspace members, and localfile:packages) normalizes eachfile:dependency relative to the top-level dir. Afile:../utildeclared invendor/lib/package.jsonis stored asvendor/util.Package::from_npm(registry manifests) stores thefile:path verbatim, so it stays relative to the declaring package (for example./the-filesinsidefile-dep).PackageInstaller(src/install/PackageInstaller.rs) sourced every transitive folder dependency fromdirname(node_modules.path), the declaring package's install directory. That is only correct for the npm case. For a localfile:parent it looked fornode_modules/lib/vendor/util, which never exists, and theENOENT -> InstallResult::Successrewrite on the next lines turned the failure into a successful, counted install.Fix
Lockfile::is_folder_tree_id(tree_id): the package whosenode_modulesa tree represents is resolved from a localfile:folder.In the installer's transitive-folder branch, when the declaring package is a
Folderresolution, install fromFd::cwd()(the top-level dir, whichbun installchdirs to) instead of the parent's install dir, and do not swallow the error. A genuinely missing source folder now fails the install withinstead of exiting 0 with the package missing.
npm-declared transitive folder dependencies keep the existing behavior, including treating a
file:path absent from the published tarball as nothing to link.test/cli/install/bun-install-registry.test.ts(transitive file dependencies, themissing-file-dep/self-file-dep/file-depfixtures) depends on both.Related: #33106 lifts the
..-escape rejection for transitivefile:dependencies declared by local packages. That PR and this one address different halves of the same user story (resolve-time rejection vs install-time materialization) and do not depend on each other; they conflict on one comment line.Tests
test/cli/install/bun-install.test.ts:installs the transitive file: dependency of a file: dependency: assertsnode_modules/lib/node_modules/nested/package.jsonexists andrequire("lib")works, for both a fresh resolve and a--frozen-lockfileinstall from the saved lockfile.fails when a transitive file: dependency's folder does not exist: the source folder is deleted; asserts the new error message and a nonzero exit instead of a silent success.Both fail on the released build and pass with this change. Verified no regression in the npm path by diffing
bun.lockand thenode_modulestree produced by the released build vs this build for all seven*file-depregistry fixtures: byte-identical.bun-install.test.ts(file:andfolder), the fullbun-install-registry.test.ts,isolated-install.test.ts, andbun-workspaces.test.tspass (the only failures are pre-existing tests that need network access).