Repository navigation
bundler: store a path external that ends in a separator with and without it
#48164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -169,7 +169,14 @@ pub(crate) fn init_external_modules( | |
| let normalized = validate_path(log, cwd, external, b"external path"); | ||
|
|
||
| if !normalized.is_empty() { | ||
| result.abs_paths.insert(&normalized).expect("unreachable"); | ||
| // The resolver's key for an import keeps the separator that the import ends in, | ||
| // so a path that ends in one is stored with and without it. | ||
| let trimmed = | ||
| bun_paths::string_paths::without_trailing_slash_windows_path(&normalized); | ||
| result.abs_paths.insert(trimmed).expect("unreachable"); | ||
| if trimmed.len() != normalized.len() { | ||
| result.abs_paths.insert(&normalized).expect("unreachable"); | ||
| } | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟣 pre-existing, not blocking: pre-existing: users who pass Why this was flaggedA user runs Verification: pre-existing. An
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes. The description has it as an open question (Notes, "Open question: the import side"). Both mechanisms that you name work. I ran the first one by passing both spellings to this build: rows 07, b3, c4, e8 and eB become external. It has two side effects. Every Bun release bundles these pairs, so I wait for a maintainer's decision before I change them. It is a change of a few lines in this same place. |
||
| } | ||
| } | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -190,18 +190,20 @@ describe.concurrent("--metafile with --aot", () => { | |
| import type { Shape } from "./shape"; | ||
| import { unused } from "./unused"; | ||
| import * as marked from "marked-external"; | ||
| import * as outside from "./outside"; | ||
| import test from "node:test"; | ||
| let missing: unknown; | ||
| try { | ||
| missing = require("./not-there"); | ||
| } catch {} | ||
| const shape: Shape = { name: process.argv[2] ?? "b" }; | ||
| const chosen = require("./chosen/" + shape.name + ".ts"); | ||
| console.log(typeof marked, typeof test, typeof missing, chosen.name, import.meta.resolve("./resolved.ts").endsWith("resolved.ts")); | ||
| console.log(typeof marked, typeof outside, typeof test, typeof missing, chosen.name, import.meta.resolve("./resolved.ts").endsWith("resolved.ts")); | ||
| `, | ||
| "marked-external.d.ts": `declare module "marked-external";\n`, | ||
| "shape.ts": `export interface Shape {\n name: string;\n}\n`, | ||
| "unused.ts": `export const unused = 1;\n`, | ||
| "outside/index.ts": `export const outside = 1;\n`, | ||
| "resolved.ts": `export const resolved = 1;\n`, | ||
| "chosen/a.ts": `export const name = "a";\n`, | ||
| "chosen/b.ts": `export const name = "b";\n`, | ||
|
|
@@ -211,12 +213,14 @@ console.log(typeof marked, typeof test, typeof missing, chosen.name, import.meta | |
| await bun( | ||
| // Only the metafile is read. An executable that loads code at run time would hold every built-in module. | ||
| ...[folder, "build", "--compile", "--aot", "--no-builtin-modules", "--external", "marked-external"], | ||
| ...["--metafile=meta.json", "main.ts", "--outfile", "app"], | ||
| // A folder is external with a separator at the end of its path too. | ||
| ...["--external", "./outside/", "--metafile=meta.json", "main.ts", "--outfile", "app"], | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 nit (optional): the existing Why this was flaggedtest/aot/cli/metafile.test.ts:193 adds Verification: nit. Triggering condition: a future regression in trailing-separator external matching, which would then fail the pre-existing assertions of this test rather than a dedicated sibling. The diff for test/aot/cli/metafile.test.ts edits the single existing "--compile --aot" metafile test in place: line 217 adds
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed. I extended the existing test because |
||
| ), | ||
| ); | ||
| const metafile = read(join(folder, "meta.json")); | ||
| expect(Object.keys(metafile.inputs).sort()).toEqual(["chosen/a.ts", "chosen/b.ts", "main.ts", "resolved.ts"]); | ||
| expect(metafile.inputs["main.ts"].imports.map(line).sort()).toEqual([ | ||
| "import-statement ./outside external", | ||
| "import-statement marked-external external", | ||
| "import-statement node:test external", | ||
| "require-call ./not-there external", | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 Users with a top-level
external = ["./lib/"]in bunfig.toml now getEISDIRfrombun runonimport "./lib", which runs fine on the base branch. The trimmed key inserted at src/bundler/options.rs:176 is also what the runtime resolver matches at src/resolver/resolver.rs:2438, and bunfig'sexternalreaches every command (src/bunfig/bunfig.rs:1214). This follows from the stated purpose. Fix: keep external path matching out of the runtime resolver for every spelling, e.g. land #47504 (clearsexternalfor VMs andbun test --changed) with or before this change, or gate theabs_pathscheck on bundling. The PR lists this as a downside; it is a break for that population.Why this was flagged
A project has
external = ["./lib/"]at the top level of bunfig.toml, parsed intoctx.args.externalfor every command at src/bunfig/bunfig.rs:1214.init_external_modulesis the only writer ofabs_paths(src/bundler/options.rs:2054) and now also inserts<cwd>/libat src/bundler/options.rs:176. Underbun run,import "./lib"reachescheck_relative_pathwith key<cwd>/lib, which now matches at src/resolver/resolver.rs:2438 and returns IS_EXTERNAL with the directory path, so the runtime tries to read the directory and fails withEISDIR;Bun.resolveSync("./lib")returns<cwd>/libinstead of<cwd>/lib/index.js, andbun test --changedstops seeing tests that import./lib. On the base branch the key is<cwd>/lib/, nothing matches, and the import resolves tolib/index.jsand runs. The PR description names this as a downside and defers to #47504, which is not in this diff, so nothing in this checkout prevents it.Verification:
src/bunfig/bunfig.rs:1214-1233parses the top-levelexternalkey unconditionally. The new code atsrc/bundler/options.rs:174-179inserts the trimmed<cwd>/lib.src/resolver/resolver.rs:2438now matches and returnsIS_EXTERNAL, so the directory<cwd>/libis loaded as a file and the read fails (EISDIR). On base the stored key<cwd>/lib/does not equal<cwd>/lib, so the program runs.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is correct, and the description lists it under Downsides. The facts for the maintainer who decides:
external = ["./lib"]givesEISDIRon 1.3.14 and on main, andexternal = ["./lib/"]gave it on 1.3.14.externalofbunfig.tomlreaches the resolver of every VM. runtime: clear the build-onlyexternalandpackagesfor every VM and thebun test --changedscan #47504 (open) clears the list ininit_runtime_stateand for thebun test --changedscan, for every spelling. A gate in this PR would be that same change.externalandpackagesfor every VM and thebun test --changedscan #47504 merges first, or in the same release, this PR changes nothing at run time.I did not add a gate here. The thread stays open for the decision on the merge order.