Skip to content

test(install): share the starting trees of bun-prune.test.ts and assert full layouts - #40639

Closed
robobun wants to merge 5 commits into
mainfrom
farm/b3d30f02/prune-test-shared-trees
Closed

robobun wants to merge 5 commits into
mainfrom
farm/b3d30f02/prune-test-shared-trees

Conversation

@robobun

@robobun robobun commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • bun-prune.test.ts took 17.1s on the x64 ASAN lane (build 106656), about 7s of it the registry start. Under ASAN bun test runs 5 cases at a time (src/options_types/context.rs:512), so the file costs its summed per-case work divided by 5.
  • 111 cases spawned 381 processes: 202 prune runs, 169 installs. 33 installs rebuilt a tree another case also built. Every spawn of a case waited for the previous one, even on a separate copy.
  • End states were spot checks (287 existsSync, 44 lstatSync throws, 25 toEndWith). stderr was unread on most prune runs.

Fix

  • shared() installs a tree once, on first use, and gives each case a copy, cache included, through copyTree (the algorithm of the harness copyTreeSync in test(install): share the starting trees between bun-update-transitive cases and assert every update summary #39741) and the harness writeBunfig. 33 fewer installs: 381 spawns become 347. Independent halves of a case run in parallel: 28 more spawns leave the serial chain.
  • layout(dir) lists every package folder, file, bin, store entry and link under a node_modules, nested folders and store included. 81 of 93 test blocks compare it before and after each prune, assert stderr on every prune run, and pin each refusal message.
  • Correct because a copy is what the case's own install had just produced: --frozen-lockfile and --production installs in a copy report no changes.
  • Verified: bun bd test test/cli/install/bun-prune.test.ts, 110 pass, 5 runs; Windows x64 debug, 109 pass. On the x64 ASAN lane the cases take 8.1s and 8.4s after the registry start (builds 106846 and 106841) against 10.1s for main (build 106656). Interleaved pairs, local debug+ASAN: main 17.43s, 17.45s, 17.06s; branch 16.66s, 17.30s, 16.03s.

Background

  • The local host is CPU bound by other tenants, so the parallel halves show no gain there. The ASAN lane has spare cores for them.
  • fs.cpSync resolves a relative link against the source and keeps an absolute one: both point into the original tree. bun writes absolute links for the cache index, and for workspace and store links on Windows.
  • The isolated linker's hidden hoist folder, node_modules/.bun/node_modules, holds one link per package name.
Notes

Spawns (preload that wraps Bun.spawn, one run each). Before: 381, 71.7s summed wall with the debug binary: prune 202, install 121, install --production 28, install --lockfile-only 10, install --frozen-lockfile 5, --cpu/--os 4, --omit=peer 1, link 2, add 1, run 1, git 6. After: 347, 66.5s: install 88, --lockfile-only 9, the rest unchanged. Summed per-case wall in the interleaved pairs: main 69.1s, 68.8s, 67.5s; branch 64.7s, 68.8s, 62.7s. The 52 copies cost 0.62s in total and the 289 layout calls 0.65s with the debug runner, so about 1.3s of case time is spent against the 7s of installs removed.

Shared trees. noDeps (14 cases), oneDepNoDeps2 (4), noDepsScopedBin (2), devBins (3), prodAndDev (2), devRootProdNested (2), native (2), isolatedNoDeps (2), isolatedDevOneDep (4), isolatedNative (2), optional and peerDepsFixed per linker (2 each), hoistedRootAndA (2), appLinksToolTree (2), prunedCheckoutTree per linker (2 each). packageTree(pkgJson, linker?) and workspaceTree(linker, workspaces) build them the way setup, setupWithLinker and setupWorkspaces do, through the same installInto and installWorkspacesInto. A tree is built lazily inside the first case that needs it, so there is no beforeAll stall. A copy's bunfig is written by VerdaccioRegistry.writeBunfig with the tree's options. Every spawn in this file also sets BUN_INSTALL_CACHE_DIR, which is what the cases relied on before.

copyTree and #39741. #39741 adds copyTreeSync to test/harness.ts and wires it into makeTreeSync, so registry.createTestDir({ files: <path> }) becomes link-aware. copyTree here is that algorithm, kept file-local because this PR is scoped to the one file: a relative link is recreated with its own kind, an absolute link into the source is pointed at the copy, an absolute directory link becomes a junction. Once #39741 lands, shared() becomes (await registry.createTestDir({ bunfigOpts, files: await source })).packageDir and copyTree is deleted; nothing else in this file changes. On Linux the only absolute links are the cache's index entries. The whole file passes on Windows x64 with the debug build after this change, 109 pass and 2 skips (the two skipIf(isWindows) cases).

CI. The x64 ASAN lane logs a timestamp per line. Main (build 106656): bun test start to the first case result 7.0s (the registry start), first result to the end 10.1s, 17.14s reported. This branch: 8.1s and 8.4s, 16.64s reported (build 106841); 6.5s and 8.1s, 14.60s reported (build 106846). The file ran third in its job there, during the job's service warm-up (coordinator: postgres_plain ready lands in the middle of its output), which is where the extra second before the first result in build 106841 comes from; main's run was eighth. The case portion is the part this PR touches: 10.1s to 8.1s and 8.4s. The registry start is the larger share of the lane time and is a harness matter: #40635's in-process FixtureRegistry is the lever for it, and this file can switch to it with the same createTestDir/writeBunfig calls once it lands.

Windows. Interleaved pairs on the Windows x64 debug build: main 11.57s, 11.65s; branch 10.98s, 10.77s.

layout. A symlink is shown as name -> target, the target made relative to the link's parent when it is absolute, separators normalized, so the same listing holds on Windows. .bin entries are shown as .bin/<name>; on Windows that row stands for the whole <name>.exe + <name>.bunx shim pair and half a pair keeps its file name. .bun lists each store entry and the contents of its node_modules, and .bun/node_modules (the hidden hoist). A scope, .bin or store folder that is left empty is a row of its own, so a folder prune leaves behind is visible. A dot folder that is not one of those is listed by name only. Peer-hash store keys are variable, so the cases about them keep storeEntries and build the expected rows from the observed key. The harness nodeModulesPackages skips links, .bin and the store, which is why the listing is local to this file; lifting it next to nodeModulesPackages is a follow-up once a second file wants it.

Assertion changes. toEndWith(NOTHING(...)) became the full three lines (24 sites). expect(() => lstatSync(p)).toThrow() became a kind(p) value check or is covered by a layout (43 sites). Pinned outputs: the global-store case's stdout (was toContain + toEndWith), both linker-mismatch refusals in each direction (4 sites), the missing-workspace dependency error and its note, the corrupt bun.lock parser output, the node_modules-is-a-file error (regex on the errno text), bun run prune (SCRIPT_RAN\n on stdout, $ echo SCRIPT_RAN on stderr). The lifecycle case now checks the install ran PRE, POST, PREPARE in order before asserting prune runs none. The listings record two things prune leaves in place that no case asserted before: an emptied .bin folder and an emptied hidden hoist folder (both are dot entries). expect() calls: 1834 before, 1613 after; inline snapshots 95 to 138.

Observations. In the alias case (aliased: npm:no-deps@1.0.0 plus dev no-deps: 2.0.0, isolated) the hidden-hoist link no-deps points at 1.0.0 or 2.0.0 depending on the install's order. Prune removes it only when it pointed at the removed 2.0.0, and the production install that follows points a new link at 1.0.0, so the case accepts both. After bun prune --production on the devRootProdNested tree a bun install --production leaves the layout unchanged.

Kept on purpose. Every prune run, every --production install check, every --frozen-lockfile reinstall, and the isolated: store entries of file:, tarball and git dependencies case (#38856 edits it). expectBinInstalled/expectBinRemoved are unused here but #40045 adds cases that call them. git merge-tree merges #39216, #40396, #40045, #40239, #39443, #30659 and #38797 cleanly onto this branch; #38856 and #39232 conflict with main in other files, not in this one. tsc reports the same pre-existing error in this file as on main (the .each callback type at the --production/--prod/--omit=dev table).

Self-review. A review of this PR asked to split off the sharing half and to build it on #39741 instead, since the copier duplicates that PR's harness helper and the registry start, not the cases, is most of the lane time. The sharing half stays: the task for this file asked for it, it is where 33 of the 58 spawns that left the serial chain come from, and it is confined to shared, copyTree, two builders and the trees table. What changed from that review: copyTree now follows copyTreeSync line for line (it created every directory link as a junction before), the copy's bunfig comes from writeBunfig instead of a Bun.TOML rewrite of the copied file, and the lane numbers above separate the registry start from the cases.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/cli/install/bun-prune.test.ts

…rt full layouts

Install each tree that several cases start from once and give every case
its own copy, cache included. A link-aware copier keeps relative links
and points absolute links into the copy. Cases with independent halves
run them in parallel.

Assert the whole node_modules layout before and after each prune with a
sorted listing of package folders, files, bins, store entries and links,
check stderr on every prune run, and pin the refusal messages.
@coderabbitai

coderabbitai Bot commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

💤 Files selected but had no reviewable changes (1)
  • test/cli/install/bun-prune.test.ts
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9cf0bff5-a7a4-4307-b8d8-b76ef2e33832

📥 Commits

Reviewing files that changed from the base of the PR and between 65362b5 and 5eb005a.

📒 Files selected for processing (1)
  • test/cli/install/bun-prune.test.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: the branch is pushed and the file passes locally, as a non-root user, and on Windows (5eb005a).

How the numbers were taken:

  • bun bd test test/cli/install/bun-prune.test.ts with the debug+ASAN build, main's file and this branch's file run in alternation on the same host, 3 pairs. The Ran 111 tests line is the figure quoted in the description.
  • Spawn counts come from a --preload that wraps Bun.spawn and logs each command and its wall time.
  • Windows x64: fetched this branch, bun bd, then the same alternating runs on the debug build.

Second commit (7e7610b): the first CI run failed one case on every Linux and macOS lane, a failed deletion is reported, the rest is removed, exit code 1. It is skipped as root, which is how I ran it locally, so the first push never exercised it. The case makes junk-b read-only; prune deletes junk-b/inner/package.json and then fails on the rmdir of inner. My first version checked the file, the original checked the folder. The folder check is back, and the case passes locally under runuser -u nobody.

Third commit (db3d92d), from the review thread below: layout() listed nothing for an emptied scope folder, so it could not tell one left behind from one removed. A scope, .bin or store folder with no rows is now a row of its own. Prune removes the emptied scope folders in every case, and the listings now also record the emptied .bin and hidden hoist folders it leaves in place.

Fourth commit (74a1eb3), from the second review thread: on Windows a bin is a <name>.exe + <name>.bunx pair and layout folded any of them into one row. Now only a whole pair folds into .bin/<name>; half a pair keeps its file name. Verified on Windows x64.

Fifth commit (5eb005a), from a self-review: copyTree now follows the harness copyTreeSync of #39741 line for line (a relative link keeps its own kind, only an absolute directory link becomes a junction), and a copy's bunfig is written by VerdaccioRegistry.writeBunfig with the tree's options instead of a Bun.TOML rewrite of the copied file. packageTree/workspaceTree build the shared trees through the same helpers as setup/setupWithLinker/setupWorkspaces. Once #39741 lands, shared() can call createTestDir({ files }) and copyTree goes. Verified locally and on Windows x64.

CI (build 106929, 5eb005a): bun-prune.test.ts passes on every lane. On the x64 ASAN lane (build 106846) it takes 14.60s, 8.1s of it the cases, against 17.14s and 10.1s for main (build 106656). The red lanes are not this diff's: test/bundler/transpiler/macro-test.test.ts (a LeakSanitizer report in node_fs_binding, red on x64-asan in every build of this PR), test/cli/run/require-cache.test.ts (its leak check, Windows x64 this time) and test/js/web/url/url.test.ts (TypeError: Invalid URL, macOS x64). All three are reported for main separately; everything else passed on retry. The diff is ready as it stands.

The CodeRabbit notices are status messages (no reviewable changes for it in a test-only diff). Nothing to act on there.

@robobun

robobun commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 10:19 AM PT - Aug 27th, 2026

❌ @robobun, your commit 5eb005a has 3 failures in Build #106929 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 40639

That installs a local version of the PR into your bun-40639 executable, so you can run:

bun-40639 --bun

…ile inside it

The failing rmdir is what leaves junk-b/inner behind; the file inside was
already removed. Check the folder, as the case did before.
Comment thread test/cli/install/bun-prune.test.ts Resolved
A scope folder that prune leaves behind empty was invisible in the
listing, so the checks that the emptied scope folder goes with its
package had no teeth. An empty folder of those kinds is now a row of its
own. Prune leaves an emptied .bin and an emptied hidden hoist folder in
place, which the cases now record.
Comment thread test/cli/install/bun-prune.test.ts Resolved
…yout

On Windows a bin is a <name>.exe and <name>.bunx pair. One row for the
pair hid a pair with one half missing; now only a whole pair folds into
the bin's name.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

…like copyTreeSync

shared() takes the bunfig options of its tree and writes the copy's
bunfig through VerdaccioRegistry.writeBunfig instead of patching the
copied one. packageTree and workspaceTree build the trees the way setup,
setupWithLinker and setupWorkspaces do, from one helper each.

copyTree recreates a relative link with its own kind and only an
absolute directory link as a junction, the same as the harness's
copyTreeSync in #39741, so the two cannot drift apart.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

This is Claude (an AI assistant) writing on Jarred's behalf.

Not part of #44968: 3,900 lines, no gain, stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants