Repository navigation
Conversation
…46969) ### Problem - `mordant-baseline.toml` accepts more findings than main has for fourteen (lint, file) lines. The `mordant` job accepts findings up to the baseline count. A new finding in one of those files then passes with no report. - Commits on main removed 21 findings in nine crates. The counts did not follow. ### Fix - Delete ten lines and lower four counts to what `bun run rust:mordant:baseline` records on main at 271b65e. The baseline goes from 494 findings to 473. The Notes list each line. - Six more stale lines are left out. #46244 removes five of them. #46528 rewrites the sixth, `unused_pub:src/sys/lib.rs` (55, main has 53). - Verified: `bun run rust:mordant` over the three CI targets exits 0 with this baseline. No source file changes. ### Background - mordant is the lint pack that the `mordant` job in `.github/workflows/rust-lints.yml` runs through `scripts/rust-mordant.ts`. `mordant-baseline.toml` holds, for each crate, the count of findings that the job accepts for each lint and file. - The job fails only when a file has more findings than its count. It reports nothing when a file has fewer, so a count stays too high until someone lowers it. - A count is the largest one that any of the three targets (Linux, Windows, macOS) records. The `unused_pub` counts come from one run over all three targets together. <details><summary>Notes</summary> | Line | Before | After | | --- | --- | --- | | `bound_checked_only_in_debug:src/css/properties/custom.rs` | 2 | deleted | | `unused_pub:src/glob/lib.rs` (the only line of `[bun_glob]`) | 1 | deleted | | `generic_body_not_generic:src/js_parser/visit/visit_expr.rs` | 1 | deleted | | `unused_pub:src/jsc/host_fn.rs` | 2 | deleted | | `unused_pub:src/jsc/lib.rs` | 1 | deleted | | `generic_body_not_generic:src/resolver/fs.rs` | 1 | deleted | | `field_valid_only_when:src/runtime/shell/subproc.rs` | 1 | deleted | | `field_valid_only_when:src/runtime/timer/mod.rs` | 2 | deleted | | `generic_body_not_generic:src/runtime/webcore/Blob.rs` | 2 | deleted | | `unused_pub:src/windows_sys/externs.rs` (the only line of `[bun_windows_sys]`) | 1 | deleted | | `safe_fn_returns_shared_mut:src/bundler/transpiler.rs` | 2 | 1 | | `safe_fn_returns_shared_mut:src/http/HTTPThread.rs` | 3 | 2 | | `generic_body_not_generic:src/jsc/ConsoleObject.rs` | 6 | 5 | | `generic_body_not_generic:src/runtime/socket/socket_body.rs` | 8 | 4 | Main has 197 lines, 494 findings and 38 sections. This branch has 187 lines, 473 findings and 36 sections. How the counts were measured, with mordant at the pinned revision 564c8da5 and `nightly-2026-09-01`: - `bun run rust:mordant:baseline` on main at 271b65e writes a baseline of 184 lines and 464 findings. It differs from the committed one in these fourteen lines, in the five lines of #46244, and in `unused_pub:src/sys/lib.rs` (55 to 53). - `bun run rust:mordant` with the baseline of this branch exits 0 and writes no `target/mordant/over-baseline.txt`. It is one `cargo mordant --workspace --keep-going` run over `x86_64-unknown-linux-gnu`, `x86_64-pc-windows-msvc` and `aarch64-apple-darwin`. Two lines with a known cause: - `[bun_glob]`: #47559 added the line on 2026-10-31, because `bun_glob::walk_bun_glob` (#47513) had no caller. After that, #47316 put `#[cfg_attr(mordant, allow(unused_pub))]` on the function, and #44833 added a caller in `src/js_parser/lower/import_meta_glob.rs`. - `custom.rs`: #47025 removed the two `debug_assert!(*d <= 0x7F)` checks that the lint reported. Changes from the first version of this PR (nine lines, measured at 449877f): - `generic_body_not_generic:src/runtime/jsc_hooks.rs` stays at 1. The baseline that main regenerates has that line again. - #47238 deleted the `IOReader.rs` and `IOWriter.rs` lines on main, so this PR does not change them. - Eight lines are new: `custom.rs`, `resolver/fs.rs`, `timer/mod.rs`, `Blob.rs`, `[bun_glob]`, `[bun_windows_sys]`, `transpiler.rs` and `HTTPThread.rs`. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · no src or test change; test-proof not applicable <!-- robobun:evidence:end -->
### Problem - `mordant-baseline.toml` accepts more findings than `main` has for five `(lint, file)` keys. The `mordant` job tolerates an over-count, so a new finding of the same lint in one of these files passes unseen until the count is used up. - Four of these files changed after their count was recorded, and `main` now has fewer findings in them. The fifth, `WebSocketUpgradeClient.rs`, has not changed since #45223 set its count to 2, and mordant reports one finding there. Nothing regenerated the baseline. ### Fix - Lower each count to the number of findings that `main` has: - `"generic_body_not_generic:src/http_jsc/websocket_client/WebSocketUpgradeClient.rs"`: 2 to 1 - `"generic_body_not_generic:src/sql_jsc/mysql/JSMySQLConnection.rs" = 1`: removed - `"generic_body_not_generic:src/sql_jsc/mysql/MySQLConnection.rs"`: 4 to 1 - `"generic_body_not_generic:src/sql_jsc/mysql/protocol/DecodeBinaryValue.rs" = 1`: removed - `"options_as_enum:src/lint/driver/lint.rs" = 1`: removed, with the `[bun_lint_driver]` header of its now empty section - The baseline goes from 198 lines and 495 accepted findings to 195 lines and 488. No code changes. - Verified: `bun run rust:mordant` exits 0 with the new file (three targets, pinned `MORDANT_REV`). ### Background - `mordant-baseline.toml` is a ratchet. Each line is `"<lint>:<file>" = <count>`. The `mordant` job hides that many findings of the lint in the file and fails only when the file has more. - A count is the largest number that any one of the three targets reports. `unused_pub` is the exception: one run over all three targets counts it (`scripts/rust-mordant.ts`). - No other line is stale: runs with an empty baseline matched every other count, `unused_pub` included. <details><summary>Notes</summary> Findings per target for the five keys: | key | old count | Linux | Windows | macOS | | --- | --- | --- | --- | --- | | `generic_body_not_generic:src/http_jsc/websocket_client/WebSocketUpgradeClient.rs` | 2 | 1 | 1 | 1 | | `generic_body_not_generic:src/sql_jsc/mysql/JSMySQLConnection.rs` | 1 | 0 | 0 | 0 | | `generic_body_not_generic:src/sql_jsc/mysql/MySQLConnection.rs` | 4 | 1 | 1 | 1 | | `generic_body_not_generic:src/sql_jsc/mysql/protocol/DecodeBinaryValue.rs` | 1 | 0 | 0 | 0 | | `options_as_enum:src/lint/driver/lint.rs` | 1 | 0 | 0 | 0 | How the counts were taken: `cargo +nightly-2026-09-01 mordant --workspace --target <triple> --keep-going` for each of `x86_64-unknown-linux-gnu`, `x86_64-pc-windows-msvc` and `aarch64-apple-darwin`, with `MORDANT_TOML` set to the text of `mordant.toml` with its `baseline` key pointed at an empty file, so that every finding prints. Then one run with all three `--target` flags for `unused_pub` (160 findings in 22 files, the same as the baseline). The finding that remains in `MySQLConnection.rs` is `MySQLConnection::process_split_packet` (generic over `C`). The finding that remains in `WebSocketUpgradeClient.rs` is `HTTPClient::<SSL>::connect`. When each count was recorded: - `MySQLConnection.rs`, `JSMySQLConnection.rs` and `DecodeBinaryValue.rs`: in #43669. #45304 changed all three files after that. - `lint/driver/lint.rs`: in #45478. #45800 and #46001 changed the file after that. - `WebSocketUpgradeClient.rs`: in #45223, which is also the last change to that file. #45720 later changed two other files of the same crate (`websocket_client.rs` and `WebSocketProxyTunnel.rs`). </details>
) Lint-ratchet maintenance, no reachable defect: `main` already refuses a graph for a module with more than 65535 requests. #44968 left out #39165 and #40312 of this category ("refactor nothing needs"). Merged #39169 has this shape. @alii decides whether this is wanted. ### Problem - `serialize()` in `src/bundler/prelinked_module_graph.rs` packs a `u32` request index into 16 bits of each import and export entry, bounded there only by a `debug_assert!`. - The release bound is one function up, in `build()`, on the request count. mordant reports `bound_checked_only_in_debug`. ### Fix - `parse_module` converts the request count once (`u16::try_from(count).ok()?`) and takes every index from `0..request_count`, as a `u16`. The const, the count test and the assertion are gone. - Removes `"bound_checked_only_in_debug:src/bundler/prelinked_module_graph.rs" = 1` from `mordant-baseline.toml`. - Verified: `bun run rust:mordant` (564c8da, 3 targets) exits 0. Empty baseline: 0 findings in the file (`main`: 1). `test/bundler/bundler_compile_prelinked.test.ts`: 83 pass (debug build). Same bytes as `main` on 4033 graphs in a model of the file. - Self-reviewed: 9 concerns raised, 9 addressed. ### Background - `bun build --compile --bytecode` embeds a pre-resolved module graph for JavaScriptCore. A request is one module that a chunk imports. `PrelinkedModuleGraph::tryCreate` refuses more than 65535 per module. - Considered an `assert!` at the shift: it costs a compare and a branch per entry. ### Downsides - Two new test cases build executables with 65535 and 65536 imports: 1 to 3 s each on release, 13 to 35 s on debug with ASAN, no CI timing yet. - They pass on `main` too, so they hold the limit and prove no fix. The 65536 case builds the `module_info` fallback executable and does not run it. Behaviour change: none <details><summary>Notes</summary> **Why nothing observable changes** - On `main`, `build()` returns `None` when `module.requests.len() > MAX_REQUESTS_PER_MODULE` (65535), and the only call of `serialize()` comes after that test. So the shift never sees an index above 65534 in any build. - The limit was stated twice. The const and its count test are one statement. The `debug_assert!` on the index is the second, with a `u32` between them, and it is a different inequality (it allows index 65535). - mordant reports the shift because the lint looks at one function, and the release bound is in another one. **Measurements** - mordant (564c8da, 3 targets): `bun run rust:mordant` exits 0 with the baseline line deleted. Findings in `prelinked_module_graph.rs` with an empty baseline: 0 (`main`: 1). The baseline goes from 495 accepted findings to 494. - Limit on a real binary (debug build of this PR, `bun build --compile --bytecode --format=esm --packages=external`): 65535 requests give a graph with `requestCount` 65535 and largest packed index 65534. 65536 requests give exit 0 and no graph. A release build with `main`'s version of the file gives the same two results. - Serialized bytes: a model that compiles the real file text against stubbed dependencies gives byte-identical output for `main`'s file and this PR's file on 33 boundary graphs and 4000 random graphs (0 differences). The model is not a bun build. - Release-profile object of `bun_bundler` (one codegen unit, no LTO), `main` against this PR: `.text` 1,893,032 to 1,892,349 bytes (-683), `.rodata` 83,806 to 83,806. Symbols that name `prelinked_module_graph`: 34,043 to 33,360 bytes. `build` (with `parse_module` and `serialize` inlined): 3267 to 3175 instructions, 336 to 328 conditional branches. Limit compares against `0xffff`: 1 per module and 0 per request in both. - Not measured: the linked release binary. The numbers above are from the crate's object before the link. - Struct sizes (`-Zprint-type-sizes`): `Import` 28 to 24 bytes, `Export` 28 to 24 bytes, a star-export element 4 to 2 bytes. These exist only during the build. The serialized layout does not change, and `VERSION` stays 2. - Over-limit module (65536 requests, `build` returns `None`), same model with a counting allocator: 84 to 11 allocator calls, 14.7 MB to 5.2 MB requested. The refusal now happens before the module's requests are parsed. - Diff: `prelinked_module_graph.rs` +21/-26, `mordant-baseline.toml` +0/-1, test +68/-0. **The new test cases** - `compile/prelinked/RequestLimit65535` and `RequestLimit65536` build an entry with that many external imports and read the executable. They do not run it. - They pass before and after this change (checked on a release build of `367d939d9`, which has `main`'s version of the file). They hold a limit that had no test and that now depends on one line. A build with that line changed to `.unwrap_or(u16::MAX)` fails the 65536 case. - Time per case: 0.5 to 2.6 s on a release build (three runs). On a debug build with ASAN: 12.7 s and 25.1 s in a run of the whole file, 18.0 s and 34.7 s when only these two run. An existing case of this file takes about 30 s on that debug build. - The entry is built as a `Buffer`. `itBundled` sends a string through `dedent()`. With the entry as a string, the two cases took 42 s and 57 s in a run of the whole file on the debug build. - The build subprocess of `itBundled` has a 60 s limit. The build step of these cases takes 12 to 17 s on the debug build. - A module over the limit is the only way an executable gets per-module `module_info` bodies in place of a graph. No other bundler test builds a module over the limit. `RequestLimit65536` has no `run`, so nothing executes that path. **What moved** - The refusal of a module over the limit now happens at the head of `parse_module`'s request section, before the requests are parsed. `build()` interns the same strings before that point as it did before. - The check reads the length of `requested_modules_keys`. The old check read the length of the zipped request list. The three request vectors of a `ModuleInfo` have one writer and the same length, so the two are equal. - The four `as usize` reads of the index became `usize::from`, because the type now allows it. **Left as they are** - `bound_checked_only_in_debug:src/bundler/Chunk.rs = 3` (`Query::new`, `EntryPoint::non_entry_point`, `set_entry_point_id`): same lint and crate, no shared type with this file. - The `kind` and `resolution` fields of the same packed word are `u32` values from a `repr(u8)` enum and a five-value table. mordant does not flag them. - A local export keeps the placeholder `request: 0`. Nothing reads it as an index. </details>
…#47523) ### What does this PR do? On Linux arm64, code compiled with `-mbranch-protection=pac-ret` saves its return addresses signed: on a processor with pointer authentication, the bits above the address hold a signature. The crash handler walks the frame pointers and read that word as it is. A signed word is the address of nothing, so the frame went into the report as unknown. Who that hits: - A Bun that is itself compiled that way loses every frame. #47517 (closed, never for merging) ran the tests with such a build, on top of oven-sh/WebKit#945, and a crash there reported the trace `AA___________________`. - In a Bun as it is built today, only the caller of a function that signs: a native addon or a system library compiled with the flag. The fix is `strip_return_address_signature` in `src/bun_core/debug.rs`, at the places that read a return address: - `StackIterator::next`, the walk. - `return_address_of_frame`, which is the read that `return_address()` (`src/bun_core/lib.rs`) did itself, moved out so that a test can give it a frame. `capture_current` looks for that address in the walk to trim the handler's own frames. One without the other would never match, and the trim falls back to the whole trace without saying so. - x30 in the `Registers:` line (`src/crash_handler/fault_registers.rs`), which the report of an executable with compiled code has. From the first instruction of a function that signs to its first call, the link register is signed. That line prints a value that may be an address by what it points into, so that it does not say where anything was loaded. A signed word is above the range that it takes for addresses, so all of it was printed, the address included. Between calls the register may hold anything, so the stripped word is taken only where it is in that range. It is `XPACLRI`, written as `hint #7`: a no-op on a processor without pointer authentication, and an address without a signature comes out as it went in. It needs no key and checks nothing. So it is compiled for every arm64 target, and there is none to leave out: I have not looked at whether anything signs on macOS or Windows, where Bun is tested but nothing here is. Left as they are: - The walk on Windows, which is `RtlCaptureStackBackTrace`. - Frames of ahead-of-time compiled code, which are walked by their tables: that code does not sign. ### How did you verify your code works? **The new test**, `a stack trace takes the signature off a return address` in `test/cli/run/run-crash-handler.test.ts`, on Linux arm64. No function of an ordinary build signs, so the new `crash_handler.stackTraceStartsAtSignedReturnAddress()` of `bun:internal-for-testing` does in inline assembly what a function compiled with the flag does: `PACIASP`, save the frame record, call, restore, `AUTIASP`. The callee takes the return address of that signed frame with `return_address_of_frame`, captures the stack trimmed to it, and the hook says whether the trace starts at the address that was signed. That takes both strips. It fails without the fix only where something is signed, so it is skipped unless `/proc/cpuinfo` has `paca`. **Nothing tests x30 in the `Registers:` line.** It takes a fault in the few instructions in which the register is signed, in an executable with compiled code. `test/aot/crash-in-compiled-code.test.ts`, which reads that line, is for x64, and plants its fault in compiled code, which does not sign. I gave the expression a signed return address under the emulator, which came out as the address, and an unsigned one, `0xfffe000000000005`, `0x7ff8123456789abc`, 42, all ones and `0xffff800012345678`, which came out as they went in. **Under emulation, each strip taken out in turn.** I have no arm64 hardware, so I compiled the helpers and the hook as they are in this PR, around a walk and a trim that do what `StackIterator::next` and `capture_current` do, for `aarch64-unknown-linux-gnu`, and ran it with `qemu-aarch64`, 100 times each at `opt-level` 0 and at 3. How many of the 200 pass: | `-cpu` | both strips | not in the walk | not in `return_address_of_frame` | neither | |---|---|---|---|---| | `neoverse-v1` (pointer authentication) | 200 | 1 | 1 | 1 | | `neoverse-n1` (none) | 200 | 200 | 200 | 200 | The few that pass without a strip are the runs in which the signature is all zeros. That can let a broken build through now and then. It cannot fail a sound one. That is of 2a073ce, and b3b1f95 changes neither. What follows, to the end of the section on CI, is of the commits before it, which had the same two strips, with the asm and the hook in a longer form. **On hardware, in a Bun that signs.** #47517 had this PR's first commit on top of its Linux arm64 build, which is compiled with `-mbranch-protection=pac-ret` (C, C++ and Rust) and links the WebKit of oven-sh/WebKit#945. The test machine printed `paca` from `/proc/cpuinfo`. The addresses in the reports of the same three crashes, without that commit ([build 135982](https://buildkite.com/bun/bun/builds/135982)) and with it ([build 135994](https://buildkite.com/bun/bun/builds/135994)); `_` is a frame that is in no image: | crash | without | with | |---|---|---| | `bun fmt` panics (`cli/format/conformance.test.ts`) | `___________________`: 0 of 19 | `u8nghGu5t/gG2iv/gGm7r7gG+nw7gG…`: 19 of 19 | | `crash_handler.panic()` | `__________________`: 0 of 18 | `mq/xwC+t7uwC+molwC_u903jGm93tjG…`: 17 of 18 | | `crash_handler.segfault()` | `29nlwC___________________`: 1 of 20, the faulting instruction | `29nlwC_u903jGm93tjG2x7mnG…`: 19 of 20 | - One frame stays unknown in the last two, which JavaScript calls. I have not looked at which it is. The trace of `crash_handler.panic()` on x64 has one too. - Each trace has as many frames as before, so `capture_current` still finds `return_address()` in the walk. - `test/cli/run/run-crash-handler.test.ts`: 33 pass, 8 skip, 0 fail, with the test in its first form among them. With this PR's second commit on top as well ([build 136000](https://buildkite.com/bun/bun/builds/136000), on a machine that printed `paca` too), so with the test as it is now: 33 pass, 8 skip, 0 fail. **In this PR's CI** ([build 136672](https://buildkite.com/bun/bun/builds/136672), at b3b1f95, the head, and the same in [build 135999](https://buildkite.com/bun/bun/builds/135999), at 113095a), `test/cli/run/run-crash-handler.test.ts` has no failure on any Linux lane, and each arm64 lane has one pass more and one skip less than its x64 twin, which is this test: | | arm64 | x64 | |---|---|---| | Debian 13 | 33 pass, 8 skip | 32 pass, 9 skip | | Ubuntu 25.04 | 33 pass, 8 skip | 32 pass, 9 skip | | Alpine 3.23 | 30 pass, 11 skip | 29 pass, 12 skip | Since 2a073ce the test is skipped where `/proc/cpuinfo` has no `paca`. It ran on all three arm64 lanes of build 136672, so all three machines have pointer authentication. 40 test files fail in that build, 39 of them in `test/aot`. They are the 40 that fail on `main`: a comment below has the builds. **On Linux x64**, with a debug build: - `test/cli/run/run-crash-handler.test.ts`: 31 pass, 10 skip, 0 fail. One of them failed before, on `main` too: `crash trace starts at the crash site, not inside the crash handler` expected the first frame to be named `js_panic`, and nothing in `src/` has had that name since #46540. It runs only in a debug build on Linux with `llvm-symbolizer`. It is a test of what `return_address()` is for, so it now expects the name that the crash site has, `crash_handler_jsc::panic`. - `crash_handler.segfault()` still reports a trace with resolved frames, through Rust, C++ and LLInt. **`bun run rust:check-all`**: 12 ok, 0 failed, 0 skipped (of 12). `cargo clippy -p bun_core -p bun_crash_handler` for `x86_64-unknown-linux-gnu` and `aarch64-unknown-linux-gnu`: nothing. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…47779) `getAzure` in `scripts/ci-image.ts` now requests an OIDC token with `buildkite-agent oidc request-token` and sends it to Azure as the `client_assertion`. The Packer template takes `client_jwt` in place of `client_secret`. - Lookups read the client ID from the Buildkite secret `AZURE_LIST_IMAGES_CLIENT_ID`, and bakes from `AZURE_BAKE_IMAGES_CLIENT_ID`. - The script no longer reads `AZURE_CLIENT_ID` or `AZURE_CLIENT_SECRET`. - The template change renames both Windows images.
With a finite maxDepth, DepthLimiter asked path.resolve() and path.relative() about every match and every directory, and kept one cache entry per path, as the text of Node v26.12.1 does. The limiter now reads the depth from the text of a path when the text says it: - plainDepth() counts the names of a relative path with no ".." segment. It is PlainDepth() of the native walker on Node's main branch (nodejs/node#65392). - #plainDepth() also counts what follows cwd in an absolute path that starts with cwd. The test of the start is the private endsWith of JSC, which user code cannot replace. - The limiter remembers the directory that canTraverse() last allowed by its depth. A name in that directory is one level deeper at most, so includes() does not ask about it. On Windows and macOS only a directory that the text placed is remembered: there Node's text asks the file system about a path that can be cwd in another spelling, and the limiter asks in the same order. A path with a ".." segment and, on Windows, a path with ":" keep the route and the awaits that they had. internal/fs/glob.ts does not change, so a walk with no maxDepth is the walk that it was. The tests run walks in a child process that counts the calls of path.relative(): the module of the limiter loads at the first walk with maxDepth, so a count that is in place before that walk sees every call. One test asks the limiter itself, through exposedInternals of bun:internal-for-testing, and compares its five methods with path.resolve() and path.relative() on every platform. bench/snippets/fs-glob-max-depth.mjs has each spelling of a pattern.
|
Status of this PR. How the cost was reproduced. A release build of #47750 (f3d2cd6) walks a tree of 121 directories and 484 files with State. A parked draft, with no request for review. Nobody has asked for this change: no issue reports the cost, and CI. |
CI builds a branch only when it has this commit.
DepthLimiter calls depthFromRoot(path, root, foldsCase) for each match and directory. The function answers from the characters of the two strings: a count of names, and on POSIX resolve() and relative() of an ASCII path with no string made. When the characters do not settle the depth, the limiter takes the route of Node.js's text, as before. The limiter has no scanner, prefix or remembered directory of its own any more.
…lasses The 8-bit body of depthFromRoot() is in the host function alone: a rope is resolved in a cold function that hands the two strings back. The path that resolve() has to put together is a function of its own, as for relative(). A 16-bit string takes the count by names with the root in either width. bench/snippets/fs-glob-max-depth.mjs has a row for each way that a pattern names its paths, with names of 2, 49 and 200 characters and 16-bit names.
… at the limit The walker calls includesAsync() only for a path that includesNow() could not answer, so a call of depthFromRoot() there is never of use. The test of a name in another case has two rows with a wildcard at the limit: there the walk reads the directory only where the file system says that it is cwd. The test of the 512 characters has an absolute path. The bench has one group for each way that a pattern names its paths.
With names of 200 characters the deepest file of the tree is over the 1,024 bytes that macOS allows for a path.
Stacked on #47750. It must merge after that PR (order: #47308, #47750, this).
Parked draft: nobody asked for it, and one engine test is red with it (State and Open items in Notes).
Problem
maxDepth,fs.globcallspath.resolve()andpath.relative()for every match and directory.**/*.txtwalk of 121 directories costs 6.44M instructions, 4.08M with nomaxDepth(0.4 ms more). The calls are inDepthLimiter.#relativeCandidate(src/js/internal/fs/glob-depth-limiter.ts:155).Fix
DepthLimiterasks one native function,depthFromRoot()(src/runtime/node/path.rs:1196). It makes no string.glob.tsis byte-identical.test/js/node/fs/glob.test.ts. Its 77 new cases fail on fs.glob: add the maxDepth option, with the match limit in the result set #47750. It passes in CI on Windows, macOS and Linux. Self-reviewed: 23 concerns raised, 10 addressed, 13 open in Notes.Background
cwd.DepthLimiter(Node v26.12.1's text) tests it againstmaxDepth.Downsides
bungrows by 12,288 bytes and 2 host functions.cwdwhencwdor a name is not ASCII (+0.6% to +2.4%), outsidecwdon Windows (up to +1.4%),cwdin another case on macOS (+2.3% to +4.7%). The last two are emulated.Notes
State. A parked draft, with no request for review. A self-review said not to queue it. The reasons:
maxDepthis in no Bun release and not on main, and fs.glob: add the maxDepth option, with the match limit in the result set #47750, which adds it, has no review yet.glob.tssays to replace it withBun.Glob, and Node replaced its own with a native walker, where the limit costs nothing per entry.maxDepth. It is older and goes first.It stays a draft until the reviewer of #47750 or a user report asks for faster walks with a limit. The question is on #47750. If the answer is yes, it comes back as one commit on main, measured again, with the open items below done. If the answer is no, close it: the branch
robobun/cdfe2c19/fs-glob-plain-depthkeeps the work.The branch has main merged up to a4c3a5e, because CI does not build an older branch. That merge brings 11 of the 18 files in the diff. They are not part of this change.
What the function answers.
depthFromRoot(path, root, foldsCase)has one body forpath.posixand one forpath.win32.rootis whatresolve()answered forcwd.>= 0: the depth of the path. It is whatpathDepth(relative(root, resolve(root, path)))counts in the limiter.~k: the path is the directoryklevels aboveroot. The limiter needs this for an absolute pattern, whose walk starts at the root of the volume.UNPLACED): the characters do not settle the depth. The limiter then runs the line of fs.glob: add the maxDepth option, with the match limit in the result set #47750.The arms, in the order of the code:
a/b): the count of the names. This isPlainDepth()of Node's native walker.resolve()would not change, and a path of..segments and then names.join()writes a path that leavescwdin this second form. The answer isrelative()of the two with no string made, from the kernel thatpath.relative()has (relative_posix_latin1)..,./a,a//b,a/../b): the function joins it torootand resolves it in 512 bytes on the stack, aspath.resolve()does for a short path. A longer one isUNPLACED. Each walk asks about.this way.foldsCase, which the limiter passes on macOS: a path besiderootisUNPLACEDwhen in lower case it isroot, or above it, or below it. Node's text asks the file system about such a path on macOS.rootas it is spelled. The function resolves nothing there. On Windows a:makes the pathUNPLACED.cwd, or through.., with such a name isUNPLACEDon each platform.Four methods of the limiter call the function:
includes(),includesNow(),canTraverse()andcanTraverseAsync().includesAsync()is the text of #47750. The walker calls it only for a path thatincludesNow()could not answer.Why a path with
:is not counted on Windows. With Node v26.12.1's ownpath.win32(Bun's gives the same three answers):resolve("C:\\root", "x:y")is a path on drivex:, so it is not belowcwd.resolve("C:\\root", "C:")isC:\rootitself: depth 0, and the count would say 1.relative()givesC:\afor.\C:\abelowcwd, andpathDepth()takes that for an absolute path: no depth, and the count would say 2.An absolute path that starts with
cwdhas its:in the part that is compared, not counted.Measurements, Linux. linux-x64 release builds of #47750 (f3d2cd6) and of this PR (18d3e33), from one build directory. A tree has 121 directories and 484 files, and a copy of itself beside
cwd. Names have 2 characters where no other length is named. A number is the user instructions of the main thread per walk (perf_event_open) after 300 walks, withmaxDepth: 100, which cuts nothing, where no limit is named. It is the median of 5 rounds in which the builds alternate. "Twice" is a copy of the #47750 binary against the #47750 binary in the same rounds. "Spread" is the highest round less the lowest, over the median, for the wider of the two builds.**/*.txt*/*/*/*/f0.txt**["**/*.txt", "*/*/*"]*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt../other/*/*.nomatch../*./**/*.txt**/*.txt*/*/*/*/f0.txt*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt**/*.txt......, no match./**/*.txt**/*.txt......, no match../*../*.txt<parent>/*<parent>/*.txt../*,maxDepth: 1../*,maxDepth: 2<parent>/*,maxDepth: 0<parent>/*,maxDepth: 1<parent>/*.txt,maxDepth: 1<parent>/*.txt,maxDepth: 2**/*.txt,maxDepth: 2maxDepth: 2..,maxDepth: 2maxDepth: 4..,maxDepth: 4**/*.txt,maxDepth: 2..,maxDepth: 4..,maxDepth: 4**/*.txt, nomaxDepthmaxDepth.., nomaxDepth.., nomaxDepth**/*.txt, nomaxDepth**/*.txt, nomaxDepth**/*.txt...., no match...., no match["**/*.txt", "*/*/*"]**/*.txt,maxDepth: 2..,maxDepth: 4**/*.txt, nomaxDepth**/*.txt....**/*.txt..cwdor through..: the function does not place such a path.cwdwith a character outside ASCII gives the same class, with ASCII names too. Measured with acwdnamedjosé: below cwd -46.3%, absolute below cwd -38.0%, through..+2.4%, beside cwd by an absolute path +2.1%, beside cwd through..+2.4%, and +0.8% with no match.maxDepthare within -0.2% to +0.7%. The same binary twice is within -0.7% to +0.9% over all rows.below cwd, no match,below cwd, a path with no wildcardand16-bit names, below cwd. In their first run other work was on the machine, and the same binary twice differed by -2.2%, -1.5% and +1.6%......., no match....maxDepthbench/snippets/fs-glob-max-depth.mjs(mitata, the average of a sync walk in µs, two runs each, with other work on the machine):globSync, names of 2 charactersmaxDepth: 100maxDepthmaxDepth: 100maxDepth..,maxDepth: 100.., no match.., a limit that cuts.., nomaxDepthmaxDepth: 100maxDepth..,maxDepth: 100.., no match.., a limit that cuts.., nomaxDepthWhere no JIT runs.
BUN_JSC_useJIT=0, user instructions of the whole process per walk (walks 4 to 13, a counter around the process). Of 35 kinds of walk with a limit, 34 cost 3.1% to 22.8% less. Below cwd: 22.39M to 18.86M (-15.8%). Names of 200 characters: -20.0% to -21.1%. The one that costs more has 16-bit names beside cwd: +1.3%. NomaxDepth: -0.1%.bun build --compile --aot), main-thread instructions per walk (walks 4 to 23). "From a file on disk" is a compiled host that loads the walking script at run time (--aot-allow-code-from-disk). "Compiled" is the walking script itself compiled. Of 17 kinds with a limit, 16 cost 2% to 31% less in both shapes. 16-bit names beside cwd: +1.9% and +2.0%.*/*/*/*/f0.txt...., no match...., no match../*......maxDepth......, no match......, no match..maxDepthOne question. Instructions for one call of the function, and for the route of #47750 for the same path, with the cost of the loop taken off. That route is
resolve(),relative(),isAbsolute()andsplit(). The path has three directory names and a file name of the length given.8-bit names:
.., below cwd.., beside cwd./and names16-bit names:
.., below cwd.., beside cwd./and names./is over 512 bytes when it is joined, so the function does not place it. The walker does not write such a path: it drops the./of a pattern.Calls and memory, per
**/*.txtwalk of the tree withmaxDepth: 100:maxDepthpath.relative(), below cwdpath.relative(), absolutepath.relative(), through..path.relative(), beside cwd, absolutepath.relative(), beside cwd, through..path.resolve(), below cwdpath.resolve(), absolutepath.resolve(), through..path.resolve(), beside cwd, absolutepath.resolve(), beside cwd, through..path.isAbsolute(), below cwdpath.isAbsolute(), absolutepath.isAbsolute(), through..path.isAbsolute(), beside cwd, absolutepath.isAbsolute(), beside cwd, through..path.relative(), 16-bit names, below cwdpath.relative(), 16-bit names, absolutepath.relative(), 16-bit names, through..path.relative(), 16-bit names, beside cwd, absolutepath.relative(), 16-bit names, beside cwd, through..path.join(): 1,208 to 1,223 calls per walk on both builds, with and withoutmaxDepth. With nomaxDepthevery count is equal on both builds.**walk of 100,110 entries, after a full collection: below cwd 46.2 MB on fs.glob: add the maxDepth option, with the match limit in the result set #47750, 32.0 MB here and 32.0 MB with nomaxDepth. Absolute: 44.0 and 34.4 MB. Through..: 47.5 and 32.9 MB. fs.glob: add the maxDepth option, with the match limit in the result set #47750 keeps one entry of#relativeCacheper path until the walk ends, as Node's text does. An unplaced path keeps its entry here too. The heap is a side effect, not the reason for the change.BUN_JSC_logGC, 2,000 walks): below cwd 545.2 KB to 392.1 KB, absolute 539.7 to 393.7, through..546.8 to 396.5. NomaxDepth: 388 and 390 KB.straceis not in this container, so aptracecounter gave them.getdents64242,openatandclose121 each,statx1 to 7 by the spelling, on both builds. The collector reads/proc/self/statmat each collection: 0.4 to 0.55 times per walk on fs.glob: add the maxDepth option, with the match limit in the result set #47750 and 0.3 to 0.35 here.maxDepthloads 0 more modules on both builds. The first walk with a limit loads 1, the limiter..text+9,728 bytes (110,632,857 to 110,642,585),.bun_builtins+944 (2,803,207 to 2,804,151), the file +12,288.internal/fs/glob-depth-limiter.jsgoes from 6,200 to 6,905 bytes.internal/fs/glob.jsis identical (cmp, 83,697 bytes).--trimmable=on),bunmaps 16,496 bytes more. A--compile --aothello maps 8,304 more, and a--compile --trimhello 20,592 more. With--compile --aot --trimthe sizes move by pages, downward in this pair of builds: hello 25,964,935 to 25,956,855 bytes, a program that callspath.relative()26,091,911 to 26,087,927, and one that callsfs.globSync()withmaxDepth29,860,231 to 29,848,055.Windows and macOS (emulated). The texts of the walker and of the limiter for
win32anddarwinrun as user code on Linux, over a tree in memory. They usepath.win32orpath.posixand the native function of this build. The table has the change in the instructions per walk against the text of #47750, withmaxDepth: 100and names of 2 and of 49 characters:win32, 2win32, 49darwin, 2darwin, 49**/*.txt*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt...., no match..,*/*/*/*/f0.txt../*..**/*.txt...., no match...., no matchpath.win32here is the one of a Linux build, whose scratch buffer has 32,768 elements against 98,303 on Windows, and the tree is in memory. So the size of thewin32gain is not a prediction for Windows.win32: names below cwd and an absolute path below cwd as it is spelled cost 13.1% to 67.7% less. Every other kind pays the one call: +0.3% to +1.4% where the rounds do not overlap.darwin: every kind costs 4.8% to 57.4% less, but a pattern that spellscwdin another case: +2.3% to +4.7%. There the function compares the two paths in lower case before it saysUNPLACED.cwdof these trees are ASCII. With acwdor a name outside ASCII, a path outsidecwdor through..isUNPLACEDon each platform.win32text no match of an async walk below cwd waits forincludesAsync(). On fs.glob: add the maxDepth option, with the match limit in the result set #47750 all 484 wait. The walk takes 3,635 microtask turns, against 5,329. By an absolute path below cwd: 3,682 against 5,497. Through..it is as on fs.glob: add the maxDepth option, with the match limit in the result set #47750: 484 matches wait, and 5,342 turns.darwintext the async walks take 3,635 to 3,667 turns, against 3,756 to 3,912. The lists and the fs calls are the same.Equal to #47750's text.
resolve()andrelative()ofpath.posixandpath.win32, and against a JS text of the same arms: 0 differences in 5,000,000 random cases per platform (ASCII, Latin-1, 16-bit, a rope and a substring of each). An answer is equal orUNPLACED.linux,darwin,win32texts). A row is one question, or a directory and then names in it (sync, async and mixed), or 400 questions in a random order.path.win32andpath.posix: 0 differences in 142,884, 142,884 and 158,436 walks (linux,darwin,win32). The matches, their order, the order of the fs calls and the calls made when each async match arrived are compared.cwd, 6 depths and 3 kinds of consumer. The same four things are identical.win32anddarwinruns are emulations on Linux. On Windows and macOS themselves, CI ran the tests of this form, and they pass: see Tests.Mutants.
darwinandwin32texts each ran against ports of the count tests and of the limiter test. 15, 10 and 36 pass, and none of these changes an answer on its platform. They are of four kinds:depth <= 0ordepth < 1fordepth < 0, which give the same answer,~karms on Windows, where the function gives no~k,takes a name in another case for cwdwith a wildcard at the limit come from this: two mutants that passed thedarwinports before.Tests.
path.relative():asks path.relative() only about a path that its characters do not place, the 27 rows ofthe depth of a name below cwd(18 on Windows), andtakes a name in another case for cwd only where the file system says that it is cwd. Each runs its walks in a child process that replacespath.relativebefore the first walk withmaxDepth. Only the limiter callspath.relative(), and its module loads at that first walk, so the count sees every call. A row gives the list of Node v26.12.1 and what the limiter asked about: no path, no path below cwd, or a path below cwd. Two rows with a 16-bit name outside cwd must ask on every platform. They fail if the count stops seeing the limiter.depthFromRoot()calls both bodies of the function on every platform, through an export ofbun:internal-for-testing. One test compares 11,715 generated cases withresolve()andrelative()ofpath.posixandpath.win32. An answer must be equal orUNPLACED, and it must not beUNPLACEDwhere the function has a rule. 45 rows name one rule each. One test has the paths around 512 characters.gives the depth that path.resolve() and path.relative() giveasks the limiter itself, through a getter inexposedInternals. It asks the five methods about more than 800 paths and about names below each. The limit is the depth of the path, one less, and one more.bun:internal-for-testingas a namespace. On a build without the export the file loads, and the 77 tests fail.glob.test.tswith this form on 11 lanes, with 0 fail on each. Windows 2019 x64 and Windows 11 aarch64: 203 pass. macOS x64 and aarch64: 213 pass. debian 13, ubuntu 25.04 and alpine 3.23, each on x64 and aarch64: 196 pass. debian 13 x64 with ASAN: 195 pass. Node'stest-fs-glob.mjsis not among the failures of any lane. This is the first run of this form on Windows and macOS.test/js/node/path/(170 pass), Node's 16test-path-*.jsfiles,test/internal/builtins-table/andtest/internal/source-lints/.Open items, for the day this is wanted. The last self-review left these. None is done, because each needs another round of code and nobody has asked for the change.
test/aot/engine/ir-6.test.ts, forvalueflow/modules/create-require-read-late-on-a-copy-of-node-module-returns-a-require-function-with-flow-rules-delayed.ir. It is red on five lanes: debian 13 x64, aarch64 and x64 with ASAN, Windows 2019 x64 and Windows 11 aarch64. It passes on main at the commit that this branch has merged (a4c3a5e). This change adds two host functions to the table of built-ins, and the ids of the entries after them move. With the new table, and withtesting=DelayFlowRules, the engine gives the effect of this program without its site. The defect is in the engine, not in the table. With the table of fs.glob: add the maxDepth option, with the match limit in the result set #47750 (a release build),testaot runof the same program alone with--option=testing=DelayFlowRules:59fails too, and so do:76,:175,:190and:199. With this branch's table (a debug build) 13 of the 256 orders fail, and one of them is order 0, the order of the fixture. This needs a cure in the engine before the change can merge.foldsCaseis off. It needs a form of the two arms with no buffer that reads 16-bit strings (about 60 to 90 lines). The 6 slower rows and the rows under acwdoutside ASCII then go... The count of names gives 0 for it. The review found no other such path in 14,976 walks, so the arm can go, with its 1,546 bytes.root, which is what acwdoutside ASCII is. The case rule of macOS is pinned for 8-bit ASCII only. Rows with such roots in the table ofdepthFromRoot()close both.cwd. They fail when the temporary directory has a character outside ASCII.src/node_pathacargo testand Miri can run them. Insrc/runtime/node/path.rsthey cannot.win32anddarwinnumbers here are an emulation.resolved_latin1()(src/jsc/bindgen.rs) makes a view of a flat 16-bit string that it then drops, andpath.win32.resolve()fills a stack buffer with zeros on each call. Both are on main.Not in this PR.
cwdor through..keeps the route of Node's text. The function resolves no Windows path: that needs the drive, UNC and case rules ofpath.win32, and nothing here runs Windows.cwdor through..keeps that route on every platform: see the first open item.canTraverseAsync()at each directory. That is inglob.ts.Pattern.isAbsolute(glob.ts) joins two strings and callsisAbsolute()at each directory test.resolve()725 times andjoin()1,208 times per walk, with or withoutmaxDepth. node:fs glob: read directories ahead in the async walk, less work per entry #42963 is about those calls.The first form of this PR. It counted the names of a path in JS with a
$charCodeAtloop. It also remembered the directory that the walk may read. A review found three things:cwdcost 0.4% to 4.3% more.A second design review picked the native function. The
endsWithprivate name of the first form is gone.Credit. The shape is that of
DepthFromRoot()andIsAncestorOfRoot()in Node's native walker (nodejs/node#65392,src/glob/glob_walker.cc). The rows of odd names in the tests come from an earlier form of #47750 (7898b35).no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/fs/glob.test.ts, test/cli/run/run-crash-handler.test.ts, test/bundler/bundler_compile_prelinked.test.ts