Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion src/bundler/options.rs
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,14 @@ pub(crate) fn init_external_modules(
let normalized = validate_path(log, cwd, external, b"external path");

if !normalized.is_empty() {
result.abs_paths.insert(&normalized).expect("unreachable");
// The resolver's key for an import keeps the separator that the import ends in,
// so a path that ends in one is stored with and without it.
let trimmed =
bun_paths::string_paths::without_trailing_slash_windows_path(&normalized);
result.abs_paths.insert(trimmed).expect("unreachable");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Users with a top-level external = ["./lib/"] in bunfig.toml now get EISDIR from bun run on import "./lib", which runs fine on the base branch. The trimmed key inserted at src/bundler/options.rs:176 is also what the runtime resolver matches at src/resolver/resolver.rs:2438, and bunfig's external reaches every command (src/bunfig/bunfig.rs:1214). This follows from the stated purpose. Fix: keep external path matching out of the runtime resolver for every spelling, e.g. land #47504 (clears external for VMs and bun test --changed) with or before this change, or gate the abs_paths check on bundling. The PR lists this as a downside; it is a break for that population.

Why this was flagged

A project has external = ["./lib/"] at the top level of bunfig.toml, parsed into ctx.args.external for every command at src/bunfig/bunfig.rs:1214. init_external_modules is the only writer of abs_paths (src/bundler/options.rs:2054) and now also inserts <cwd>/lib at src/bundler/options.rs:176. Under bun run, import "./lib" reaches check_relative_path with key <cwd>/lib, which now matches at src/resolver/resolver.rs:2438 and returns IS_EXTERNAL with the directory path, so the runtime tries to read the directory and fails with EISDIR; Bun.resolveSync("./lib") returns <cwd>/lib instead of <cwd>/lib/index.js, and bun test --changed stops seeing tests that import ./lib. On the base branch the key is <cwd>/lib/, nothing matches, and the import resolves to lib/index.js and runs. The PR description names this as a downside and defers to #47504, which is not in this diff, so nothing in this checkout prevents it.

Verification: src/bunfig/bunfig.rs:1214-1233 parses the top-level external key unconditionally. The new code at src/bundler/options.rs:174-179 inserts the trimmed <cwd>/lib. src/resolver/resolver.rs:2438 now matches and returns IS_EXTERNAL, so the directory <cwd>/lib is loaded as a file and the read fails (EISDIR). On base the stored key <cwd>/lib/ does not equal <cwd>/lib, so the program runs.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct, and the description lists it under Downsides. The facts for the maintainer who decides:

I did not add a gate here. The thread stays open for the decision on the merge order.

if trimmed.len() != normalized.len() {
result.abs_paths.insert(&normalized).expect("unreachable");
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟣 pre-existing, not blocking: pre-existing: users who pass --external ./lib and write import "./lib/" still get lib/index.js bundled, while esbuild keeps the import external. The fix at src/bundler/options.rs:174-179 only widens the stored keys when the external ends in a separator. When the import ends in the separator and the external does not, the resolver key <abs>/lib/ still misses the single stored key <abs>/lib. Fix: match an import and an external regardless of which side carries the trailing separator, for example by storing every path entry with and without it, or by trimming the import's key once in the three readers, so ./lib, ./lib/, ./lib// and ./lib/. on either side all pair up.
A small fix can ride a push you are already making; otherwise a short reply is enough.

Why this was flagged

A user runs bun build entry.js --external ./lib with import v from "./lib/" and lib/index.js present. init_external_modules at src/bundler/options.rs:169 normalizes ./lib to <cwd>/lib; without_trailing_slash_windows_path leaves it unchanged, so src/bundler/options.rs:177 inserts nothing else. check_relative_path at src/resolver/resolver.rs:2431 builds the key <cwd>/lib/ from the import as written, and src/resolver/resolver.rs:2438 compares bytes, so the import is not external and the folder is bundled. The same holds for the absolute-import compare at src/resolver/resolver.rs:2160 and for ./lib//, ./lib/index.js/ and <abs>/lib/ on the import side. The base branch bundles these too, so this is pre-existing. The PR description calls the import side an open question and says the same lines can store every entry both ways; the change as merged leaves this half of the class unmatched.

Verification: pre-existing. An --external entry without a trailing separator while the source import specifier ends in one. src/bundler/options.rs:169-179: only the single key <cwd>/lib is inserted. resolver.rs:2438 does self.opts.external.abs_paths.contains(abs_path), so <cwd>/lib/ misses <cwd>/lib and lib/index.js is bundled. The base branch fails the same pair by the same route.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The description has it as an open question (Notes, "Open question: the import side"). Both mechanisms that you name work. I ran the first one by passing both spellings to this build: rows 07, b3, c4, e8 and eB become external.

It has two side effects. import "<abs>/lib/" with --external=./lib becomes external, which no Bun release and not esbuild does. One more bunfig.toml case under bun run moves to EISDIR until #47504 lands.

Every Bun release bundles these pairs, so I wait for a maintainer's decision before I change them. It is a change of a few lines in this same place.

}
}
}
Expand Down
4 changes: 2 additions & 2 deletions src/paths/resolve_path.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1552,7 +1552,7 @@ pub fn join_abs<'a, P: PlatformT>(cwd: &'a [u8], part: &[u8]) -> &'a [u8] {

/// Convert parts of potentially invalid file paths into a single valid filpeath
/// without querying the filesystem
/// This is the equivalent of path.resolve
/// Unlike `path.resolve`, the result keeps the platform's separator at the end of the last part.
Comment thread
robobun marked this conversation as resolved.
Resolved
///
/// Returned path is stored in a temporary buffer. It must be copied if it needs to be stored.
// result borrows the thread-local buffer ('static) OR returns `cwd`
Expand Down Expand Up @@ -1596,7 +1596,7 @@ pub fn join_abs_string_spill<'a, P: PlatformT>(

/// Convert parts of potentially invalid file paths into a single valid filpeath
/// without querying the filesystem
/// This is the equivalent of path.resolve
/// Unlike `path.resolve`, the result keeps the platform's separator at the end of the last part.
///
/// Returned path is stored in a temporary buffer. It must be copied if it needs to be stored.
pub fn join_abs_string_z<'a, P: PlatformT>(cwd: &'a [u8], parts: &[&[u8]]) -> &'a ZStr {
Expand Down
8 changes: 6 additions & 2 deletions test/aot/cli/metafile.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,18 +190,20 @@ describe.concurrent("--metafile with --aot", () => {
import type { Shape } from "./shape";
import { unused } from "./unused";
import * as marked from "marked-external";
import * as outside from "./outside";
import test from "node:test";
let missing: unknown;
try {
missing = require("./not-there");
} catch {}
const shape: Shape = { name: process.argv[2] ?? "b" };
const chosen = require("./chosen/" + shape.name + ".ts");
console.log(typeof marked, typeof test, typeof missing, chosen.name, import.meta.resolve("./resolved.ts").endsWith("resolved.ts"));
console.log(typeof marked, typeof outside, typeof test, typeof missing, chosen.name, import.meta.resolve("./resolved.ts").endsWith("resolved.ts"));
`,
"marked-external.d.ts": `declare module "marked-external";\n`,
"shape.ts": `export interface Shape {\n name: string;\n}\n`,
"unused.ts": `export const unused = 1;\n`,
"outside/index.ts": `export const outside = 1;\n`,
"resolved.ts": `export const resolved = 1;\n`,
"chosen/a.ts": `export const name = "a";\n`,
"chosen/b.ts": `export const name = "b";\n`,
Expand All @@ -211,12 +213,14 @@ console.log(typeof marked, typeof test, typeof missing, chosen.name, import.meta
await bun(
// Only the metafile is read. An executable that loads code at run time would hold every built-in module.
...[folder, "build", "--compile", "--aot", "--no-builtin-modules", "--external", "marked-external"],
...["--metafile=meta.json", "main.ts", "--outfile", "app"],
// A folder is external with a separator at the end of its path too.
...["--external", "./outside/", "--metafile=meta.json", "main.ts", "--outfile", "app"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): the existing --compile --aot metafile test's fixture and flags are changed to the new case rather than adding a variant alongside it (REVIEW.md: never mutate an existing test's input to the new case). Fix: add a sibling test that passes --external ./outside/ and asserts import-statement ./outside external, leaving the original fixture, console.log, and expectations untouched; the added typeof outside output is never read, per the test's own comment.

Why this was flagged

test/aot/cli/metafile.test.ts:193 adds import * as outside from "./outside" and test/aot/cli/metafile.test.ts:217 adds --external ./outside/ to the one existing --compile --aot metafile test, and test/aot/cli/metafile.test.ts:201 adds typeof outside to a console.log that the test never reads (the comment at line 214 says only the metafile is read). If the trailing-separator external matching regresses, the pre-existing assertions on metafile.inputs and the other import lines fail together with the new one, so the original coverage no longer reports independently. The base branch has the original test unchanged; REVIEW.md asks for new variants alongside existing tests.

Verification: nit. Triggering condition: a future regression in trailing-separator external matching, which would then fail the pre-existing assertions of this test rather than a dedicated sibling. The diff for test/aot/cli/metafile.test.ts edits the single existing "--compile --aot" metafile test in place: line 217 adds "--external", "./outside/" to the build flags. No new test case is added alongside, which REVIEW.md forbids.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I extended the existing test because test/aot/README.md asks for one build for many cases, and all earlier assertions are still there. A sibling test reports by itself, so I will split it out with the next push to this branch.

),
);
const metafile = read(join(folder, "meta.json"));
expect(Object.keys(metafile.inputs).sort()).toEqual(["chosen/a.ts", "chosen/b.ts", "main.ts", "resolved.ts"]);
expect(metafile.inputs["main.ts"].imports.map(line).sort()).toEqual([
"import-statement ./outside external",
"import-statement marked-external external",
"import-statement node:test external",
"require-call ./not-there external",
Expand Down
66 changes: 65 additions & 1 deletion test/bundler/bundler_edgecase.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, expect, test } from "bun:test";
import { bunEnv, bunExe, tempDir } from "harness";
import { bunEnv, bunExe, isWindows, tempDir } from "harness";
import { readdirSync, writeFileSync } from "node:fs";
import { join } from "node:path";
import { unsupported } from "../aot/harness/commands";
Expand Down Expand Up @@ -1870,6 +1870,70 @@ describe("bundler", () => {
api.expectFile("/out.js").toContain("h(");
},
});
// A path in `external` names the same file or folder with and without a separator at its end.
// Each row: the external, what the entry point imports, and whether those imports stay imports.
const externalPathRows: [id: string, external: string, specifiers: string[], kept: boolean][] = [
["Folder", "./lib/", ["./lib"], true],
["AbsoluteFolder", "{{root}}/lib/", ["./lib"], true],
["TwoSeparators", "./lib//", ["./lib"], true],
["File", "./lib/index.js/", ["./lib/index.js"], true],
["ImportEndsInDot", "./lib/", ["./lib/."], true],
["MissingTarget", "./absent/", ["./absent"], true],
["ImportEndsInSeparator", "./lib/", ["./lib/"], true],
["BothImportSpellings", "./lib/", ["./lib", "./lib/"], true],
// The match is on the whole path: a folder does not cover the files inside it.
["FileInsideFolder", "./lib/", ["./lib/index.js"], false],
["ParentFolder", "./", ["./lib"], false],
];
if (isWindows) {
externalPathRows.push(["Backslashes", ".\\lib\\", ["./lib"], true]);
} else {
// An import by absolute path has a compare of its own in the resolver. On Windows that compare
// is on the import as written, which has no drive letter or has `/` where the list has `\`.
externalPathRows.push(
["AbsoluteImport", "./lib/", ["{{root}}/lib", "{{root}}/lib/"], true],
["RootedImport", "/api/", ["/api", "/api/"], true],
);
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
}
for (const backend of ["cli", "api"] as const) {
for (const [id, external, specifiers, kept] of externalPathRows) {
const imports = specifiers.map((specifier, i) => `import v${i} from ${JSON.stringify(specifier)};`);
itBundled(`edgecase/ExternalPathEndsInSeparator/${id}/${backend}`, {
backend,
files: {
"/entry.js": `${imports.join("\n")}\nconsole.log(${specifiers.map((_, i) => `v${i}`)});`,
"/lib/index.js": `export default "LIB";`,
},
external: [external],
onAfterBundle(api) {
const out = api.readFile("/out.js");
expect({
imports: out.match(/^import .+$/gm) ?? [],
bundlesLib: out.includes(`"LIB"`),
}).toEqual({
imports: kept ? imports.map(line => line.replaceAll("{{root}}", api.root)) : [],
bundlesLib: !kept,
});
},
});
}
}
// bunfig.toml gives `bun build` the same list.
itBundled("edgecase/ExternalPathEndsInSeparator/Bunfig", {
backend: "cli",
files: {
"/entry.js": `import v from "./lib";\nconsole.log(v);`,
"/lib/index.js": `export default "LIB";`,
"/bunfig.toml": `external = ["./lib/"]`,
},
onAfterBundle(api) {
const out = api.readFile("/out.js");
expect({
imports: out.match(/^import .+$/gm) ?? [],
bundlesLib: out.includes(`"LIB"`),
}).toEqual({ imports: [`import v from "./lib";`], bundlesLib: false });
},
});
// `bun build --compile --aot` resolves entry points with a resolver of its own, to the same answers.
const itCompiledAheadOfTime = (id: string, opts: BundlerTestInput) =>
(unsupported ? itBundled.skip : itBundled)(`edgecase/aot/${id}`, {
Expand Down
Loading