Skip to content

install: make bun update re-resolve git dependencies to the current remote - #36689

Closed
robobun wants to merge 12 commits into
mainfrom
farm/6ee2ddd9/bun-update-git-deps
Closed

robobun wants to merge 12 commits into
mainfrom
farm/6ee2ddd9/bun-update-git-deps

Conversation

@robobun

@robobun robobun commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Fix

  • The refresh is now git fetch --quiet --prune origin +refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/*.
    • Explicit refspecs update the refs find_commit reads; + follows force-pushes and moved tags.
    • --prune removes refs deleted or renamed upstream. Without it a rename into a nested name (release to release/1.0) collides with the stale loose ref and every later refresh of that repository fails, including lockfile-pinned installs whose checkout is missing from the cache.
    • Fetching origin by name (not the URL the caller passed) keeps using whichever transport the original clone succeeded with; the clone now passes -o origin so clone.defaultRemoteName in the user's gitconfig cannot change the name.
  • bun install with a lockfile still checks out the pinned commit: the refresh only moves refs in the bare clone, and the pinned SHA is what gets checked out.
  • Verified by the six cases added to test/cli/install/bun-install-git-deps.test.ts (these scenarios never worked, so they are not regressions):
    • bun update on a HEAD-tracking dep moves to the new upstream HEAD (with clone.defaultRemoteName set in the global gitconfig)
    • bun update <name> on a #branch dep moves to the new branch tip
    • bun update with an unchanged upstream leaves bun.lock byte-identical
    • pinned bun install after upstream moved (node_modules and the @G@<sha> checkout removed, bare clone kept) still installs the locked commit
    • upstream branch renamed into a nested name, then a pinned reinstall
    • a new tag pushed without moving any branch, then switching the dep to #<tag>
  • On the released build, the two bun update cases and the tag case fail; the other three are guards and pass. Removing any one of --prune, -o origin, the heads refspec or the tags refspec from the fix fails at least one case.

Background

  • Bun caches each git dependency as a bare clone (<cache>/<hash>.git) and each resolved commit as a separate checkout directory (<cache>/<name>@G@<sha>). Resolving a dependency means refreshing the bare clone, running git log -1 <committish> in it to get a SHA, then checking that SHA out.
  • A fetch refspec (+refs/heads/*:refs/heads/*) tells git fetch which remote refs to copy to which local refs. git clone normally writes one into the repo config; git clone --bare does not, so git fetch with no arguments in such a repo only records what it saw in FETCH_HEAD and updates nothing git log can see.
  • git fetch --prune deletes local refs that no longer exist on the remote, in the namespaces covered by the refspecs. A ref named release and a ref named release/1.0 cannot coexist (one is a file, the other needs a directory of the same name), so a rename upstream needs the stale ref pruned before the new one can be created.
  • Tests serve a bare repository over git's dumb HTTP protocol from Bun.serve({ port: 0 }), so no network is involved.

Follow-up

Two warm-cache edge cases are left for a separate PR; both recover with bun pm cache rm, and cold clones and lockfile-pinned installs are unaffected by either:

  • The bare clone's HEAD symref is still the one written at clone time. If upstream renames its default branch, --prune removes the old head and a dependency with no committish fails with no commit matching "" (it previously resolved the stale tip silently). Fix: repoint HEAD after the fetch (git ls-remote --symref origin HEAD, then git symbolic-ref HEAD ...).
  • Bare clones created by earlier releases on a machine with clone.defaultRemoteName set have no remote called origin, so the new refresh fails there (the old one worked because git falls back to the only configured remote). -o origin covers every clone made from now on; repairing old caches means discovering the remote name on failure, which belongs with the HEAD repoint above.

Fixes #13769
Fixes #11548
Fixes #18947

Same underlying fix as #35566; whichever lands first, the other can be closed.

Earlier revisions

The first version of this PR also skipped the get_package_id short-circuit in enqueue_dependency for update targets. Tracing showed that lookup already misses for git dependencies (their realname() is empty until a checkout has run), so the stale result came entirely from the un-refreshed bare clone, and that change was dropped. The tests originally lived in test/regression/issue/13769.test.ts; review pointed out these paths never worked, so they moved to the existing git-deps test file.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 7 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: dbc8da30-244a-4dea-b33a-208e065f5731

📥 Commits

Reviewing files that changed from the base of the PR and between b7a777e and 7949481.

📒 Files selected for processing (2)
  • src/install/repository.rs
  • test/cli/install/bun-install-git-deps.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Reproduced with a local dumb-HTTP git server. On the released build, bun update keeps the first commit and switching to a newly pushed tag fails with "no commit matching"; with this branch all six cases in test/cli/install/bun-install-git-deps.test.ts pass.

bun bd test test/cli/install/bun-install-git-deps.test.ts

PR: #36689

@github-actions github-actions Bot added the claude label Aug 1, 2026
@robobun

robobun commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 7:12 PM PT - Aug 12th, 2026

⏳ @robobun, your commit 7949481 is still building in Build #93858, but has 3 failures so far (All Failures):

Comment thread src/install/repository.rs Resolved
@github-actions

github-actions Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Found 3 issues this PR may fix:

  1. bun update --force doesn't update GitHub dependencies #8590 - bun update --force doesn't update GitHub dependencies — same root cause: the lockfile short-circuit in enqueue_dependency skips re-resolving git deps during bun update
  2. bun requires clearing cache when updating package that is using a git tag #18947 - bun requires clearing cache when updating package that is using a git tag — the missing fetch refspecs mean the bare clone cache never sees new tags/refs without bun pm cache rm
  3. bun requires clearing cache when updating package that is using a git tag #11548 - bun requires clearing cache when updating package that is using a git tag — same stale-cache root cause as bun requires clearing cache when updating package that is using a git tag #18947; changing a git tag in package.json fails with "no commit matching X found" until cache is cleared

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #8590
Fixes #18947
Fixes #11548

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🟡 src/install/PackageManager/PackageManagerEnqueue.rs:1182-1195 — The new should_update predicate in the Tag::Git arm gates only on is_root_dependency(id), dropping the || dependency.version.tag == Tag::Catalog disjunct that the npm sibling it's modeled on carries (lines ~2002-2011, comment: "catalogs are root-scoped"). A catalog entry that resolves to a git+... spec reaches this arm with dependency.version.tag == Catalog (see the catalog resolution at ~728-743), so when it's consumed by a workspace member rather than the root, is_root_dependency is false and bun update still short-circuits to the stale lockfile SHA — the same bug this PR fixes, left unfixed for the catalog-via-member sibling. One-line fix: mirror the || Tag::Catalog disjunct from the npm path.

    Extended reasoning...

    What the bug is

    The PR introduces a should_update predicate in the Tag::Git arm of enqueue_dependency_with_main_and_success_fn (PackageManagerEnqueue.rs:1182-1195) so that bun update skips the get_package_id short-circuit and re-resolves the committish against the remote. This predicate is copied from the pre-existing npm path at lines ~2002-2011, but drops one disjunct. The npm path reads:

    this.to_update
        // Update direct deps of the current workspace; catalogs are root-scoped.
        && (dependency.version.tag == dependency::version::Tag::Catalog
            || unsafe { &*(*this_ptr).lockfile }.is_root_dependency(unsafe { &mut *this_ptr }, dependency_id))
        && (this.update_requests.is_empty() || ...)

    whereas the new git path reads:

    this.to_update && dep.resolved.is_empty() && {
        unsafe { &*(*this_ptr).lockfile }.is_root_dependency(unsafe { &mut *this_ptr }, id)
            && (this.update_requests.is_empty() || ...)
    }

    The || Tag::Catalog clause is missing.

    Why the git arm is reachable with a Catalog tag

    At lines ~728-743, when dependency.version.tag == Tag::Catalog, the catalog entry is looked up and its resolved version is assigned to the local version binding (break 'version v). The subsequent match version.tag at line ~753 then dispatches on the catalog entry's tag — so a catalog entry whose value is git+https://... lands in the Tag::Git arm at line 1171. Crucially, dependency itself is untouched: dependency.version.tag is still Catalog. This is exactly why the npm sibling checks dependency.version.tag == Tag::Catalog explicitly, with the comment "catalogs are root-scoped".

    Why is_root_dependency alone is insufficient

    is_root_dependency (lockfile.rs:854-864) checks whether the dependency id is in the current root package's dependency list (via manager.workspace_name_hash / root_package_id). Catalogs are declared in the workspace root's package.json but consumed by workspace members: when member packages/foo declares "lib": "catalog:", the resulting dependency id belongs to packages/foo's dependency list, not the root's. So is_root_dependency(id) returns false for that dep when bun update is run from the workspace root.

    Step-by-step proof

    1. Workspace root package.json defines "workspaces": { "catalog": { "lib": "git+http://.../lib.git" } } and a member packages/foo.
    2. packages/foo/package.json declares "lib": "catalog:". The root does not depend on lib directly.
    3. bun install resolves lib to commit A and pins it in the lockfile.
    4. Upstream pushes commit B.
    5. User runs bun update from the workspace root. this.to_update is true.
    6. Enqueue reaches packages/foo's lib dependency: dependency.version.tag == Catalog, so lines 728-743 assign the catalog's git version to version; match version.tag enters Tag::Git at 1171.
    7. dep.resolved.is_empty() is true (the spec has no #sha). is_root_dependency(id) checks the root package's dep list — lib's id is in packages/foo's list, not the root's — returns false.
    8. should_update is therefore false; get_package_id matches the lockfile-loaded package via committish (empty resolved fallback in Repository::eql) and short-circuits to commit A's stale SHA. The clone/fetch path never runs.

    Result: bun update from the root does not move a catalog-defined git dependency consumed only by a workspace member — the exact bug class #13769 reports and this PR fixes for direct root deps.

    Impact and severity

    This is an incomplete fix, not a regression: before this PR the git arm had no should_update gate at all, so catalog git deps didn't update either. Merging leaves that narrow scenario (catalog + git spec + consumed only by a non-root workspace member) exactly where it was, while the primary reported case is fixed. REVIEW.md ("Fix the whole class in the same PR ... parallel switch arms") asks that sibling arms share the same guard shape, and the fix is a one-token addition mirroring the sibling the author copied from — so it's worth flagging, but not worth blocking merge.

    Fix

    Mirror the npm sibling:

    let should_update = this.to_update && dep.resolved.is_empty() && {
        let this_ptr: *mut PackageManager = this;
        (dependency.version.tag == dependency::version::Tag::Catalog
            || unsafe { &*(*this_ptr).lockfile }.is_root_dependency(unsafe { &mut *this_ptr }, id))
            && (this.update_requests.is_empty()
                || this.updating_packages.contains(
                    dependency.name.slice(this.lockfile.buffers.string_bytes.as_slice()),
                ))
    };

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

Good catch on the catalog case. Added the Tag::Catalog disjunct in cfa0ae8 so a git+... catalog entry consumed by a workspace member is also re-resolved, matching the npm sibling's guard shape.

On the related issues: the refspec half of this change covers #11548 and #18947 (stale refs in the bare cache). #8590 is the github: tarball path, which this PR does not touch.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/install/PackageManager/PackageManagerEnqueue.rs:1175-1186 — The Tag::Catalog disjunct added here in cfa0ae8 makes should_update permanently true for a catalog: dep whose catalog entry is git+..., so the post-checkout re-enqueue never reaches get_package_id and resolutions[id] is left at invalid_package_id. The PR description's termination invariant ("After the checkout writes resolved back into the dependency... the short-circuit applies again") does not hold for Catalog: runTasks.rs:1509-1518 writes into dependencies[id].version.value.git, but pass 3 re-reads version from lockfile.catalogs (line 728-742), which was never updated — so dep.resolved.is_empty() stays true across all three passes. Fix the write-back to branch on dependencies[id].version.tag and, for Catalog, write resolved into the catalog store (or set resolutions[id] directly there) so pass 3 can terminate.

    Extended reasoning...

    What the bug is

    For a workspace member consuming a git+... catalog entry via "lib": "catalog:", running bun update from the root now leaves that dependency's resolution slot at invalid_package_id with an orphan callback in task_queue. Before this PR the same dep resolved (stalely) to the lockfile pin; after this PR it does not resolve at all.

    Code path

    The git resolve loop takes three passes through enqueue_dependency for the same id: (1) enqueue clone, (2) after clone → find_commit → enqueue checkout, (3) after checkout → must terminate by binding resolutions[id].

    Termination relies on the write-back at runTasks.rs:1509-1518:

    let repo = unsafe { &mut *manager.lockfile.buffers.dependencies[id].version.value.git };
    repo.resolved = pkg.resolution.git().resolved;

    For a plain Tag::Git dep, pass 3 clones dependencies[id].version at line 749, so dep.resolved is now the SHA → dep.resolved.is_empty() at line 1177 is false → should_update = false → get_package_id matches the freshly-appended package → success_fn sets resolutions[id]. ✅

    For a Tag::Catalog dep this write-back is invisible on two counts:

    1. dependencies[id].version.tag == Catalog, so the active union arm is .catalog (a SemverString), not .git; the write lands in bytes that are never read back as .catalog.
    2. More importantly, on re-enqueue line 728-742 re-derives version from this.lockfile.catalogs.get(...) — the catalog store, not dependencies[id]. Nothing wrote resolved into the catalog store, so version.git().resolved is still empty.

    Why the guard never flips

    On pass 3 with a Catalog→git dep and bare bun update:

    • this.to_update == true
    • dep.resolved.is_empty() == true (catalog store was never touched)
    • is_direct_update_target(id, dependency): the dep.version.tag == Catalog disjunct at line 2760 short-circuits to true; update_requests.is_empty() is true → returns true

    So should_update == true again, get_package_id is skipped, control falls through to the git_repositories hit → find_commit returns the same SHA → same checkout_id. needs_ctx = (resolutions[id] == invalid_package_id) == true → ctx is re-pushed to task_queue[checkout_id] (which was mem::take'd at runTasks.rs:1492). has_created_network_task(checkout_id) finds the existing dedupe entry → returns true → return Ok(()). resolutions[id] is never assigned; success_fn never fires; an orphan callback sits in task_queue[checkout_id] with no task to drain it.

    process_extracted_tarball_package appends the package but does not set resolutions[dep_id] — that is delegated to the re-enqueue's success_fn, which is unreachable here.

    Step-by-step proof

    1. Root package.json: {"workspaces": {"packages": ["pkg/*"], "catalog": {"lib": "git+http://…/lib.git"}}}; pkg/app/package.json: {"dependencies": {"lib": "catalog:"}}.
    2. bun install → lockfile pins commit A.
    3. bun update from root → for the lib dep in pkg/app, dependency.version.tag == Catalog, to_update == true, update_requests empty.
    4. Pass 1: line 728 resolves version from catalog store (Tag::Git, resolved empty). should_update = true → skip get_package_id → git_repositories empty → push ctx to task_queue[clone_id], enqueue clone.
    5. Pass 2 (after clone): same version, should_update = true → skip get_package_id → repo present → find_commit → push ctx to task_queue[checkout_id], enqueue checkout.
    6. GitCheckout completes: package appended; runTasks.rs:1509-1518 writes resolved into dependencies[id].version.value.git (wrong arm); process_dependency_list_item re-enqueues.
    7. Pass 3: line 728 fires again, version re-read from lockfile.catalogs — resolved still empty. should_update = true → skip get_package_id → repo present → same checkout_id → ctx re-pushed → has_created_network_task true → return Ok(()). resolutions[id] never assigned.

    Why this is a regression from this PR

    Before this PR the Tag::Git arm ran get_package_id unconditionally. On pass 1 it matched the stale lockfile entry via Repository::eql's committish fallback and bound the slot — the "silently stale" bug this PR fixes. The Catalog disjunct in is_direct_update_target (added in cfa0ae8 specifically to handle this case per the earlier review note) now sends catalog→git down the clone/checkout path but removes its only termination mechanism. The scenario goes from "resolves to stale commit" to "resolution slot left invalid + orphan task-queue entry".

    Fix

    The write-back at runTasks.rs:1509-1518 already assumes the active union arm is .git, which is false for Catalog deps. Branch on dependencies[id].version.tag there: for Catalog, write resolved/package_name into lockfile.catalogs.get_mut(...).version.value.git instead (or in addition), so line 728-742's read on pass 3 sees a non-empty resolved and should_update flips to false. Alternatively, set resolutions[id] = package_id directly in the GitCheckout handler before re-enqueue and gate the needs_ctx push on it — but the write-back fix keeps the termination story symmetric with plain Tag::Git.

Comment thread src/install/repository.rs Resolved
@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

On the catalog termination concern: the should_update guard (and is_direct_update_target) was dropped entirely in b4486ea after tracing showed the get_package_id lookup already misses on the update path (realname() for git deps returns the empty package_name until a checkout has run). The only src change left is the git fetch refspec in repository.rs, so there is no new code on the catalog path.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🟡 test/regression/issue/13769.test.ts:178-201 — This test never reaches Repository::download(): after bun add, node_modules already matches the lockfile pin, so bun install short-circuits (installer.verify() → true, needs_install = false) and no GitClone task is scheduled — both assertions are reading state written by bun add. To make it a real guard, rm -rf node_modules and the @G@<sha> checkout-cache entry between publishCommit and bun install so the warm-cache fetch runs (moving refs/heads/main to B locally) and the checkout must still land on the lockfile's SHA A. Nit — tests 1 and 2 already fail-before/pass-after on the refspec fix; this is missing coverage, not incorrect behaviour.

    Extended reasoning...

    What the test claims vs. what it does

    Test 4's stated purpose (per the PR description and the inline comment "install must honour the lockfile pin, not float to HEAD") is to prove that the new explicit-refspec fetch does not cause bun install to float off the lockfile's pinned SHA when the remote has moved. But the code path this PR changes — Repository::download() in repository.rs:722-745 — is never reached by this test.

    The trace

    1. bun add git+http://…/repo.git populates node_modules/issue13769lib at COMMIT_A, writes bun.lock pinning SHA_A, writes .bun-tag = SHA_A into the installed dir, and warms both the bare-clone cache and the per-SHA @G@<sha> checkout cache.
    2. publishCommit(..., "COMMIT_B") mutates only the served bare repo. Nothing under app/ or cache/ changes.
    3. bun install loads the lockfile; package.json is unchanged since bun add, so Diff::generate maps every dep and summary.add/update stay 0 — enqueue_dependency_with_main is not called for the git dep, and no GitClone task is scheduled from the resolve phase.
    4. In the link phase, PackageInstaller.rs:1591 computes needs_install = !installer.verify(...). verify() for git deps (PackageInstall.rs:833-836 → verify_git_resolution) reads node_modules/issue13769lib/.bun-tag and compares it to the lockfile's resolved SHA — both are SHA_A → returns true. So needs_install = false, the if needs_install block at line 1598 (which contains the only enqueue_git_for_checkout call) is skipped, and summary.skipped is incremented.

    Net: Repository::download() — the only function this PR changes — never runs. bun install does zero work.

    Why the assertions are vacuous

    Both assertInstalled(app, "COMMIT_A") and lockAfter === lockBefore are asserting on bytes that bun add wrote and bun install never touched. If the refspec change floated the pin to HEAD, panicked, or was reverted entirely, test 4 would still pass — the changed code is unreachable from it. This is exactly what REVIEW.md's "Prove the test fails for the RIGHT reason" section warns against: "check that … fast paths can't satisfy the test without running your code" and "confirm deleting each load-bearing clause of your fix breaks at least one test — a test that passes both ways is worse than no test".

    How to make it a real regression guard

    Force the changed code path to run and then choose between the fetched HEAD and the lockfile pin:

    await publishCommit(env, upstream, serve, "COMMIT_B");
    // Force re-checkout from the (now-refreshed) bare cache using the lockfile SHA.
    await Bun.$`rm -rf ${join(app, "node_modules")}`;
    for await (const d of new Bun.Glob("@G@*").scan({ cwd: cache, onlyFiles: false })) {
      await Bun.$`rm -rf ${join(cache, d)}`;
    }
    const lockBefore = await Bun.file(join(app, "bun.lock")).text();
    const { stderr, exitCode } = await runBun(app, cache, env, "install", "--no-progress");

    With node_modules gone and the per-SHA checkout cache cleared (but the bare clone left in place), needs_install and package_missing_from_cache are both true, enqueue_git_for_checkout schedules a GitClone (fresh process ⇒ git_repositories empty), download() takes the cache-hit branch and runs the new refspec fetch — moving refs/heads/main in the local bare clone to B — and the subsequent checkout must still land on SHA_A because the lockfile's resolved (not find_commit) drives it. Now the assertions can actually fail if the fix regresses.

    Severity

    Nit. The refspec fix itself is correct and adequately covered by tests 1 and 2, which do exercise download() and fail-before/pass-after. Test 4 as written just doesn't provide the additional "install still honours the pin" guard it claims to; merging as-is causes no concrete failure.

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

Strengthened the install-pin guard in cb62ca8: it now drops node_modules and the @G@<sha> checkout cache (keeping the bare clone) before bun install, so Repository::download() runs, refreshes the bare cache to commit B, and the checkout must still land on the lockfile's SHA A.

Comment thread test/regression/issue/13769.test.ts Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the runtime change is now just three extra args to the warm-cache git fetch (origin + the two refspecs), which is the minimal correct fix for a bare clone with no configured fetch refspec.

What was reviewed:

  • Verified clone --bare sets remote.origin.url but no fetch = refspec, so origin resolves and the pre-PR default-remote behavior is unchanged; only the refspecs are new.
  • Confirmed the fifth test's tag-only push (serve's refs/heads/main stays at commit A) means tag-auto-follow on the heads refspec cannot see v2, so deleting the tags refspec breaks it — both load-bearing clauses are now covered.
  • Tests are hermetic (local dumb-HTTP Bun.serve, per-test tempDir/cache, GIT_CONFIG_NOSYSTEM), use test.concurrent, and release resources via using/await using.
  • All four earlier review threads (enqueue guard, catalog disjunct, url→origin, tag coverage) are addressed in the final diff.
Extended reasoning...

Overview

After several review iterations the PR reduced to a single-site runtime change in src/install/repository.rs: the warm-cache git fetch --quiet now passes origin +refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/*. The earlier PackageManagerEnqueue.rs guard was dropped entirely once tracing showed get_package_id already misses on the update path (git deps' realname() is empty pre-checkout), so the stale SHA came solely from find_commit reading un-refreshed refs in the bare cache. I locally confirmed git clone --bare writes remote.origin.url but no remote.origin.fetch, so a refspec-less fetch only touches FETCH_HEAD — the mechanism in the PR description is accurate. Using origin rather than the passed url keeps the pre-PR remote-selection behavior (the argless fetch already defaulted to origin), so no new https-then-ssh retry noise on SSH-only hosts.

The new test/regression/issue/13769.test.ts adds five test.concurrent cases over a local dumb-HTTP git server: no-committish bun update, named bun update <pkg> on #branch, no-op update leaves lockfile byte-identical, bun install honours the lockfile pin after dropping node_modules and the @G@<sha> checkout cache, and a tag-only push case for #11548/#18947 where serve's branch heads stay at commit A so only the explicit tags refspec can surface v2.

Security risks

None. The added git args are static byte literals; no user input flows into argv here. is_safe_resolved_tag and the --end-of-options guard in find_commit are unchanged. The + force prefix is appropriate for a cache mirror and does not touch any user working tree.

Level of scrutiny

Low-to-moderate. The package manager is production-critical, but the change is three constant argv entries on an existing git invocation whose remote-selection semantics are unchanged from before the PR. The failure mode of getting this wrong is "fetch fails and we log an error" (same error path as today), not silent corruption.

Other factors

The bug-hunting system found nothing this run. All four of my prior inline threads are resolved in the final diff — in particular the tag-coverage gap was closed with a test the author verified fails when the tags refspec is deleted. Tests follow harness conventions: tempDir, bunEnv spread, port: 0, using/await using, stderr asserted before exit code, no external network, isolated GIT_CONFIG_GLOBAL/GIT_CONFIG_NOSYSTEM/BUN_INSTALL_CACHE_DIR per test. The PR body's evidence block shows fail-before/pass-after on both ASAN-debug and release.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the final diff is the minimal fix (three static argv entries on the warm-cache git fetch) and all four rounds of feedback have been addressed.

What was reviewed:

  • origin + explicit heads/tags refspecs — clone --bare always writes remote.origin.url, so the remote exists; static literals, no user input reaches argv.
  • Fifth test's tag-only push leaves served refs/heads/main at A, so tag auto-follow can't satisfy it — the +refs/tags/* line is now load-bearing.
  • Confirmed this is the only git fetch site in src/install/; no sibling to update.
  • Tests are hermetic (Bun.serve({port:0}) dumb-HTTP git, tempDir, per-test cache dir), pipes drained concurrently, resources released via using.
Extended reasoning...

Overview

The PR now touches two files: a 14-line change to src/install/repository.rs that adds origin plus +refs/heads/*:refs/heads/* and +refs/tags/*:refs/tags/* to the existing warm-cache git fetch --quiet, and a 234-line regression test at test/regression/issue/13769.test.ts with five concurrent cases. Earlier revisions also touched PackageManagerEnqueue.rs with a should_update guard; that was dropped in b4486ea after tracing showed get_package_id already misses on the update path (git deps' realname() is empty pre-checkout), leaving only the refspec fix.

Security risks

None. The three new argv entries are compile-time byte literals; no user-controlled data reaches them. The pre-existing url argument is no longer passed to the warm-cache fetch (it now uses the fixed string origin), which is strictly less surface than before. find_commit already guards the committish with --end-of-options, and checkout already gates resolved on is_safe_resolved_tag — neither path is touched here.

Level of scrutiny

Medium. src/install/ is a load-bearing subsystem, but the actual code change is a well-understood git-CLI argv adjustment with no new control flow, no memory or lifetime changes, and no platform-gated code. The root cause (clone --bare writes remote.origin.url but no remote.origin.fetch, so a bare git fetch only updates FETCH_HEAD) is documented git behaviour, and the fix is the canonical remedy. I grepped src/install/ for other b"fetch" sites — this is the only one.

Other factors

This PR has been through four review rounds from me, each addressed:

  1. updating_packages never holds git deps → the enqueue guard was dropped entirely once tracing showed it was a no-op.
  2. Passing url to the warm-cache fetch would re-try the try_https rewrite on SSH-only hosts and dump git stderr on every update → switched to origin (75bab7e).
  3. The install-pin guard test wasn't forcing the fetch path → strengthened to drop node_modules and the @G@<sha> checkout cache while keeping the bare clone (cb62ca8).
  4. No test covered the +refs/tags/* refspec → added a fifth case (de70167) with a tag-only push so heads auto-follow can't satisfy it; author verified delete-the-tags-line turns it red.

The tests follow harness conventions: tempDir, bunEnv/bunExe, port: 0, test.concurrent, await using for the server, Promise.all on stdout/stderr/exited, per-test isolated BUN_INSTALL_CACHE_DIR, and GIT_CONFIG_NOSYSTEM/GIT_CONFIG_GLOBAL pointing at an empty file so CI git config can't interfere. The gate evidence shows fail-before on the debug build. No CODEOWNERS entry covers src/install/. The bug-hunting system found nothing this run.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right fix, but as written the refresh can start failing on caches that work today.

  • needs --prune, otherwise a branch or tag renamed into a nested name upstream makes every later warm-cache install of that repo fail until bun pm cache rm
  • the fetch hardcodes origin but the clone does not pin the remote name, so clone.defaultRemoteName in ~/.gitconfig breaks it
  • these bugs never worked, so the tests belong in test/cli/install/bun-install-git-deps.test.ts, not test/regression
    This is the same change as #35566. Whichever of the two lands, the other should be closed, and the same three points apply to that one.

Comment thread src/install/repository.rs Resolved
Comment thread src/install/repository.rs Resolved
Comment thread test/regression/issue/13769.test.ts Resolved
robobun and others added 7 commits August 13, 2026 00:44
… remote

`bun update` would report "no changes" for a `git+...` dependency after
new commits were pushed upstream, leaving the lockfile pinned to the
original SHA.

Two independent problems combined here:

- `enqueue_dependency` looks up the dependency's spec in
  `Lockfile::get_package_id` before scheduling a clone. The spec's
  `resolved` is empty, so `Repository::eql` falls back to comparing the
  committish and matches the package loaded from the lockfile. The clone
  was never scheduled and the stale pin was reused.

- The cached bare clone is refreshed with `git fetch --quiet`, but
  `git clone --bare` configures `remote.origin.url` without a fetch
  refspec, so that fetch only updates `FETCH_HEAD` and `find_commit`
  still reads the old refs.

For an update target whose `resolved` is not yet known, skip the
`get_package_id` short-circuit so the clone/fetch/find-commit path runs
and the committish is resolved against the remote. After the checkout
writes `resolved` back into the dependency, the short-circuit applies
again and binds the slot to the freshly appended package.

Pass explicit `+refs/heads/*` and `+refs/tags/*` refspecs (against the
dependency URL) to the cache refresh so `find_commit` sees the current
remote state, including caches populated by older builds. The
fetch-error log is now guarded on `attempt > 1` like the clone branch,
so the https-then-ssh fallback stays quiet when only the second
transport works.

The regression test serves a bare repo over git's dumb HTTP protocol
via `Bun.serve({port: 0})` and covers `bun update`, `bun update <name>`,
an unchanged HEAD (lockfile unchanged), and `bun install` (lockfile pin
preserved).

Fixes #13769
Replaces the raw-pointer split borrow with a disjoint field borrow in
`PackageManager::is_direct_update_target`, and drops the explanatory
paragraph at the call site.
Tracing showed `get_package_id` was already a miss on the update path:
`realname()` for a git dependency returns `version.git().package_name`,
which is empty until the checkout has run, so `name_hash` is 0 and the
lockfile lookup cannot match the stale entry. The clone was always
scheduled; the stale SHA came from `find_commit` reading the un-refreshed
bare cache. All four regression tests pass with the repository.rs change
alone.
`remote.origin.url` records the transport that the initial clone
succeeded with, so fetching from `origin` avoids re-trying the
`try_https` rewrite on the warm-cache path for SSH-only hosts. The
error log is restored to its original unconditional form since the fetch
no longer depends on which rewrite the caller passed.
Remove node_modules and the @g@<sha> checkout cache before `bun install`
so the install has to run `Repository::download()` (refreshing the bare
cache to commit B) and then check out the lockfile's pinned SHA.
The new tag is published without moving any branch head, so git's tag
auto-follow on the heads refspec cannot see it and only the explicit
`+refs/tags/*:refs/tags/*` makes `find_commit` resolve `#v2`. Fails with
"no commit matching" on main and if the tags refspec is removed.
Review follow-ups for the bare-clone refresh:

- `git fetch --prune`: a branch or tag renamed upstream into a nested
  name (`release` -> `release/1.0`) collides with the stale loose ref in
  the bare clone and, without pruning, fails every later refresh of that
  repository until `bun pm cache rm`. Lockfile-pinned installs take this
  path too whenever the pinned checkout is missing from the cache.

- `git clone --bare -o origin`: the refresh fetches from `origin` by
  name, so the clone must not let `clone.defaultRemoteName` in the
  user's gitconfig pick something else.

- The tests move from test/regression to
  test/cli/install/bun-install-git-deps.test.ts, reusing its helpers.
  None of these scenarios ever worked, so they are not regressions.
  The six cases: `bun update` on a HEAD-tracking dep (with
  `clone.defaultRemoteName` set in the global gitconfig), `bun update
  <name>` on a branch dep, an unchanged upstream leaving the lockfile
  byte-identical, a pinned `bun install` after upstream moved, an
  upstream branch renamed into a nested name, and a tag-only push of a
  new tag. Removing any one of `--prune`, `-o origin`, the heads refspec
  or the tags refspec fails at least one of them.
@robobun
robobun force-pushed the farm/6ee2ddd9/bun-update-git-deps branch from e3c60fa to 9014c04 Compare August 13, 2026 01:47
Comment thread src/install/repository.rs Resolved
Comment thread src/install/repository.rs Resolved

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the new head against the three points from the last round: the refresh now prunes, the clone pins the remote name, and the cases moved into bun-install-git-deps.test.ts and cover both. Ran the new fetch and clone command lines against local bare repos (nested branch and tag renames, moved tag, clone.defaultRemoteName, pinned checkout after prune, dumb http) and they behave as the PR describes. One non-blocking note inline; looks ready to merge from this side. Neither touched file has moved on main since the merge base.

Comment thread src/install/repository.rs Resolved
Comment thread src/install/repository.rs Resolved
Comment thread src/install/repository.rs Resolved
robobun added a commit that referenced this pull request Aug 13, 2026
…sh test

Folds the cases from #36689 into git-cache-refresh.test.ts: bun update moves
a committish-less dependency to the new HEAD and a #branch dependency to the
branch tip, bun install keeps the lockfile pin after the cache has been
refreshed past it, and bun update against an unchanged remote leaves the
lockfile unchanged.
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Consolidated into #35566, which makes the same git fetch refspec change and now also carries the bun update and lockfile-pin cases from this PR (in test/cli/install/git-cache-refresh.test.ts). Closing.

@robobun robobun closed this Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

3 participants