Skip to content

Bun.Archive: extract() with a glob rejects at a file or directory that a system call refuses; files(glob) matches normalized names - #44731

Closed
robobun wants to merge 2 commits into
mainfrom
robobun/a178f38c/archive-glob-entry-errors
Closed

robobun wants to merge 2 commits into
mainfrom
robobun/a178f38c/archive-glob-entry-errors

Conversation

@robobun

@robobun robobun commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Related to #43132

Draft, to be re-cut (second self-review, Notes). The files(glob) half is now #45456. The extract half waits for one answer: with a glob, does a refused mkdir or open reject like extract(dir), or skip? The branch has the first cut.

Problem

Fix

  • First cut, on the branch: with a glob, the promise rejects at a refused directory or file with that error (code, syscall, path). A file gets the owner-write bit.
  • Re-cut, not written yet: on main, only the directory mkdir, the parent mkdir and the open reject.
  • Verified for the first cut: test/js/bun/archive.test.ts, CI green at a0c0f971e4b. Self-reviewed twice (Notes).

Background

  • A glob call runs extract_to_disk_filtered. Other calls run Archiver::extract_to_dir, which bun install shares.
  • Weighed: open and write arms (the directory arm keeps the defect), an unlink before a refused open (it deletes foreign files), one filtered extractor (21 of 29 probed glob results change).

Downsides

  • extract(dir, { glob }) rejects where it resolved with a smaller count. For a directory entry, extract(dir) still resolves (Notes).
  • First cut: with a glob, a 0444 entry is written as 0644.
  • Cost of the first cut, with the files(glob) half: an extract() job 152 to 160 bytes, release .text +2,304 bytes.
Notes

State after the second self-review. The review ran on head a0c0f971e4b. Its ruling: right problem, wrong packaging.

The sections below are the account of the first cut.

Source of the report. An automated check found both defects. No user reported them. #43132 lists the glob skip in one row of its table: the destination holds a read-only a.txt, extract(dir) rejects, and the glob form resolves with count 0. With this PR both call forms reject, and the file keeps its content.

Three questions for a maintainer.

  1. The held part. extract(dir) without a glob resolves and counts a directory entry that it cannot create (src/libarchive/lib.rs:1672 to :1680). It also skips a symlink that it cannot create, as the glob form does. A fix is a policy in ExtractOptions that only Bun.Archive sets, because bun install, bun create and the --compile download share that extractor. It exists as commit 6aba634052 on branch robobun/a178f38c/archive-entry-errors, against an older main. It is not in this PR because it changes extract(dir) for callers that did not ask. Should it follow as its own PR?
  2. The interim. Until then the two call forms differ on a directory entry that cannot be created: extract(dir) resolves, extract(dir, { glob }) rejects (5 of 35 probed rows). If that is not acceptable, I move the directory arm of the glob form into the held part.
  3. The end state. Is it one extractor with a filter hook in Archiver::extract_to_dir, and extract_to_disk_filtered deleted? The design review set it aside for now: it changes 21 of 29 probed glob results (count, file mode, symlink rules). This question does not gate this PR.

Settlement per failure. Debug builds of main (5f5799ed16) and of this branch (4b17f325e5), Linux x64. 35 rows: 14 as uid 0, 21 as uid 65534 (the chmod cases need a user that is not root).

main this PR
rows where the glob form rejects 0 of 35 20 of 35
rows where extract(dir) rejects and the glob form resolves 15 0
rows where extract(dir) resolves and the glob form rejects 0 5 (directory entries)
rows where extract(dir) settles in another way than on main 0

The 15 rows where the glob form still resolves: 4 controls, 7 symlink rows (the symlink is skipped, as without a glob), 2 directory entries whose name is taken by a regular file (see "Not fixed here"), and 2 read-only entries that a later entry or a second extract now overwrites.

What the glob form rejects with:

failure rejection
destination not writable, top-level file EACCES, open, <dest>/a.txt
destination not writable, file sub/deep/x.txt EACCES, mkdir, <dest>/sub
directory entry in a directory that is not writable EACCES, mkdir, <dest>/sub/empty
read-only file of the same name in the destination EACCES, open, <dest>/r.txt (the file keeps its content)
a file has the name of the parent directory ENOTDIR, open, <dest>/blocker/x.txt
a file has the name of a directory two levels up ENOTDIR, mkdir, <dest>/blocker/sub
a directory has the name of a file entry EISDIR, open, <dest>/taken (EEXIST on Windows)
file name of 300 bytes ENAMETOOLONG, open, <dest>/<name>
file fresh/<300 bytes>/deep/x.txt ENAMETOOLONG, mkdir, <dest>/fresh/<300 bytes>
write past a file size limit EFBIG, write, <dest>/big.txt (the partial file is removed)

path is the destination as the caller passed it, joined with the name that the failed call was given. When the directory of an entry cannot be made, the rejection is that mkdir error with the directory that the system refused. It is not the ENOENT of the open after it. extract(dir) still rejects with Error("ReadError") and no code (#44509).

Measurements. Release builds of main (5f5799ed16) and of this branch (4b17f325e5), Linux x64. perf, valgrind and strace are not installed here, so the counts come from ptrace tools (a syscall counter and an int3 plus single-step instruction counter), objdump, nm and size. A count per entry is the difference between an archive of 400 entries and one of 200, or of 2,000 and 1,000 for syscalls.

  • Job allocation of extract(): mov $0x98,%edi before the malloc call on main, $0xa0 here (152 to 160 bytes, both in the 160-byte class). ExtractResult 8 to 16 bytes. A const assert keeps it at 16.
  • Output syscalls per 1,000 entries of a successful extract(dir, { glob: "**" }): files openat 1,000, pwrite64 1,000, close 1,000. Nested files add mkdirat 1,000. Directories mkdirat 1,000. Symlinks symlinkat 1,000. Identical on both builds, also for extract(dir).
  • Allocator calls per entry inside the job: with a glob 4 (file, nested file, directory) and 6 (symlink), without a glob 4 (file) and 5 (nested file), 6 for files(). Identical on both builds.
  • Instructions per entry inside the job. Two runs of one build differ by up to 6.
    • files("**"): 4,702 to 4,953 (+250, +5.3%) for 10-byte names. That is the one normalize pass. files() without a glob: 4,434 to 4,417.
    • extract(dir, { glob: "**" }): file 5,491 to 5,465, nested file 6,057 to 6,020, directory 5,059 to 5,006, symlink 5,489 to 5,421. The path buffer is now taken once per call and not once per entry.
    • extract(dir): file 4,835 to 4,836, nested file 5,429 to 5,433. Both are inside the spread.
  • Release .text: 80,758,501 to 80,760,805 bytes (+2,304). The file grows by 4,096 bytes. 12 functions change size. The larger ones: the ExtractContext job 3,808 to 4,431, the FilesContext job 2,156 to 2,624, extract_to_dir::<()> 6,030 to 6,314, ExtractTarball::run of bun install 10,274 to 10,430, read_data_into_fd 241 to 342. The streaming extractor of bun install does not change (tarball_stream::drain_callback, 10,407 bytes on both).

Behavior changes to know.

  • extract(dir, { glob }) stops at a refused directory or file, as extract(dir) does. The entries after it are not written. Main wrote them.
  • With a glob, a file is created with the owner-write bit. Main created a 0444 entry as 0444, and a later entry of the same name or a second extract then skipped it and kept the old content. extract(dir) adds 0o666 to every file mode, so it did not have the case. On Windows the glob form no longer creates such a file read-only.
  • A read-only file that an older Bun extracted with a glob makes a second extract reject with EACCES for a user that is not root. Main resolved and kept the old content.
  • files("./src/*") and extract(dir, { glob: "./src/*" }) now mean src/*. On main the first matched only stored ./ names, the second matched nothing.
  • files(glob) still lists an entry that extract() leaves out for its name (second byte :). A test pins it.
  • The docs recipe "validate before extraction" now checks with patterns. The old string checks rejected every entry of a tar -C dir . archive, and passed src/x\.env, which extract() writes as the hidden file src/x/.env.

Not fixed here.

  • extract(dir): a directory entry that cannot be created (question 1).
  • A symlink that cannot be made is skipped by both call forms. A symlink whose name is taken keeps what the name has (archive: replace an existing file on extract instead of writing into it #43144 replaces an existing node).
  • A directory entry whose name is taken by a regular file is counted by both call forms.
  • A failed or short read of entry data in the glob form is still a skip. That is the read side, Bun.Archive: reject when a header read fails, with one in-memory tar reader and a named policy #43242.
  • A name that does not fit the path buffer is skipped by both call forms.
  • Entries that the glob form leaves out by rule: a symlink whose target has a .. component, a name whose second byte is :, hard links, FIFOs and devices.
  • bun pm diff matches its patterns against stored names.
  • Two entry names that are not valid UTF-8 can share one key of files(). Another change tracks it.

Other open PRs on this function. #45307 rewrites the loop head and the write arm (see above). The others are mine, and none converts the mkdir and open arms.

Review history.

  • A self-review of the first shape (three commits on top of Bun.Archive: reject when a header read fails, with one in-memory tar reader and a named policy #43242) asked for this shape: a PR against main without the extract(dir) half. It raised 15 concerns. 9 are addressed. 1 is fixed on main since (the mkdir livelock, install: consolidate the open package manager pull requests #44968). 5 are for the held part: directory modes at mkdir time, the macOS errno of a trailing slash, the directory-over-a-file case, and two missing tests.
  • Two of its changes did not hold in review on this PR and are gone. An unlink of a file that open refuses with EACCES also deleted files that the call did not create. The owner-write bit replaces it. A rejection for a symlink that cannot be made differed from extract(dir) and failed on a file system without symlinks. The symlink arm is main's code again.

Tests run. Debug build (ASAN) of this head: test/js/bun/archive.test.ts 131 pass, 3 skip as root, 133 pass, 1 skip as uid 65534. With src/ of main (5f5799ed16): 10 of the 19 new tests fail as root, 13 as uid 65534. The 6 that pass on both pin what must not change (an existing directory, a symlink whose name is taken, a symlink that cannot be made, an entry that the glob leaves out, the : name, a long name). cargo clippy -p bun_libarchive -p bun_runtime: no warning from this diff. bun run rust:check-all: 12 targets ok. I cannot run Windows here. CI gave the Windows errno values of two tests (EEXIST, ENOENT).


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/archive.test.ts

@robobun

robobun commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 12:59 PM PT - Oct 15th, 2026

✅ @robobun, your commit a0c0f971e4b6b69a0932905ef0b636bd0bf00bb7 passed in Build #126062! 🎉


🧪   To try this PR locally:

bunx bun-pr 44731

That installs a local version of the PR into your bun-44731 executable, so you can run:

bun-44731 --bun

@robobun

robobun commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: closed. This draft was split in two: #46646 carries the extract(dir, { glob }) half, and #45456 carries the files(glob) half.

How I reproduced it. Linux x64, debug builds of main (5f5799ed16) and of this branch (4b17f325e5), as a user that is not root, because root ignores directory permissions.

  • extract(dir, { glob: "**" }) into a destination with mode 0555, into one whose sub directory has mode 0555, into one where a directory has the name of an entry, and under a file size limit (ulimit -f 8). Main resolves with 0, 2, 2 and 2, and files are missing. extract(dir) rejects in all four cases. This branch rejects with EACCES, EACCES, EISDIR and EFBIG.
  • The cases that do not depend on the user (a file where a directory must be, a name of 300 bytes, a write past a file size limit) are tests in test/js/bun/archive.test.ts.

@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: oven-sh/bun/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Essentials
  • Run ID: 4bccddc8-b68a-4beb-b126-17f77459bb8b
📥 Commits

Reviewing files that changed from the base of the PR and between 5749c31 and 158f2fa.

📒 Files selected for processing (5)
  • docs/runtime/archive.mdx
  • packages/bun-types/bun.d.ts
  • src/runtime/api/Archive.rs
  • src/runtime/error.rs
  • test/js/bun/archive.test.ts
💤 Files with no reviewable changes (1)
  • src/runtime/error.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

Archive glob matching now uses normalized extraction-style paths while files() retains raw UTF-8 entry names. Filtered extraction reports filesystem failures with syscall details and entry paths. Documentation and tests cover matching and extraction behavior.

Changes

Archive glob and extraction behavior

Layer / File(s) Summary
Normalize archive names for glob matching
src/runtime/api/Archive.rs, packages/bun-types/bun.d.ts, docs/runtime/archive.mdx, test/js/bun/archive.test.ts
Glob patterns ignore repeated leading ./. Filtered listing and extraction match normalized entry names, while files() map keys remain unnormalized. Tests cover path separators, exclusions, and long or unsafe names.
Propagate extraction filesystem failures
src/runtime/api/Archive.rs, src/runtime/error.rs, test/js/bun/archive.test.ts, docs/runtime/archive.mdx, packages/bun-types/bun.d.ts
Filtered extraction returns path-qualified system errors for filesystem failures. It handles file, directory, and symlink cases, removes partial files after write failures, and exposes system-error details to JavaScript. Tests cover failures, retries, and repeated extraction.

Suggested reviewers: jarred-sumner

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 158f2

The glob matching and extraction-error changes appear mergeable after normal checks; no actionable risk remains from this review.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly summarizes both primary changes: normalized glob matching and rejection of refused file or directory operations during glob extraction.
Description check Passed The description is detailed, relevant, and includes the problem, fix, limitations, test results, and verification details. It does not use the template headings exactly, but it provides the required i…
Full details: Description check

Explanation

The description is detailed, relevant, and includes the problem, fix, limitations, test results, and verification details. It does not use the template headings exactly, but it provides the required information in equivalent sections.

  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟣 src/runtime/api/Archive.rs — Pre-existing: users who extract a truncated or corrupt archive with a glob still get a resolved promise with a smaller count, where extract(dir) rejects. At Archive.rs:1488-1491 a failed archive.read_data only sets write_success = false, so the partial file is unlinked and the loop moves on with no rejection. At Archive.rs:1397 a Fatal read_next_header just ends the while loop and Success(count) is returned. The docs and bun.d.ts added here say the glob form rejects when an entry cannot be written. Fix: map a negative read_data result and a Failed/Fatal read_next_header in the glob loop to ExtractResult::Err(ExtractError::ReadError), the same outcome extract_to_dir gives at src/libarchive/lib.rs:1439-1441 and :1887.

    Why this was flagged

    An archive whose body is cut short is passed to archive.extract(dest, { glob: '**' }). For an entry whose data is missing, archive.read_data at Archive.rs:1487 returns a negative libarchive status; Archive.rs:1488-1491 sets write_success = false and breaks, Archive.rs:1524 unlinks the partial file, rejection is still None so no return happens, and the loop continues to the next header. When the next header is itself unreadable, read_next_header at Archive.rs:1397 returns Fatal, succeeded() is false, the loop ends and Archive.rs:1573 returns ExtractResult::Success(count). The caller gets a resolved promise with a count lower than the archive's entry count and no error. The base branch behaves the same way at these two sites, so this is pre-existing, but the non-glob path extract_to_dir returns Err(crate::Error::Fail) for both a Fatal header at src/libarchive/lib.rs:1439-1441 and a failed data read at src/libarchive/lib.rs:1887, so extract(dest) rejects for the same input. The new rejection variable at Archive.rs:1481 only covers bun_sys::write outcomes, not the libarchive read result.

    Verification: Pre-existing: the base already continued on a failed read_data and returned Ok(count). Trigger: archive.extract(dest, { glob }) on a tarball truncated mid-entry. src/runtime/api/Archive.rs:1488-1491 sets write_success = false and no rejection, so the loop continues; line 1397 ends on Fatal and line 1573 returns Success(count). src/libarchive/lib.rs:1439-1441 and 1887 return Err(crate::Error::Fail).

Comment thread src/runtime/api/Archive.rs Resolved
Comment thread src/runtime/api/Archive.rs Resolved
Comment thread src/runtime/api/Archive.rs Resolved
Comment thread src/runtime/api/Archive.rs Resolved
Comment thread src/runtime/api/Archive.rs Resolved
Comment thread src/runtime/api/Archive.rs Resolved
…t a system call refuses

With a glob, extract() resolved although entries were not written. The
filtered extractor did `continue` after a failed mkdir, open or write.
It now stops at the first directory or file that a system call refuses,
and the promise rejects with that error (code, syscall, path). The error
of a mkdir names the directory that the system refused.
read_data_into_fd returns the error of a refused write, so that the
caller can report it. A symlink that cannot be made is still skipped,
as without a glob.

With a glob, a file is created with the owner-write bit, as without a
glob. A later entry of the same name or a second extract can then open
a read-only entry.

files(glob) matched the stored entry name, and extract() with a glob
the normalized one. Both now match the normalized name, through the
NormalizedName type. A leading "./" of a pattern is dropped.
@robobun
robobun force-pushed the robobun/a178f38c/archive-glob-entry-errors branch from 158f2fa to 4b17f32 Compare October 15, 2026 14:16
@robobun robobun changed the title Bun.Archive: extract() with a glob rejects at an entry that a system call refuses; files(glob) matches normalized names Bun.Archive: extract() with a glob rejects at a file or directory that a system call refuses; files(glob) matches normalized names Oct 15, 2026
@robobun

robobun commented Oct 15, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed 4b17f325e5. It is rebased on main, fixes the CI failure, and takes the review.

CI. test/js/bun/archive.test.ts failed on Windows in one test. Windows reports EEXIST for the open of a name that is a directory, and the test expected EISDIR. The test now pins EEXIST on Windows.

Rebase. #44968 replaced the copy loop of the glob form with read_data_into_fd, which returned Failed for a refused write. It now returns that error, and the glob form rejects with it. extract(dir) and bun install handle the result as before: in a probe of 35 failure cases, extract(dir) settles the same way on main and on this branch.

Review.

  • Read-only file replaced (Archive.rs:1466): fixed. The unlink is gone. With a glob, a file is now created with the owner-write bit, as extract(dir) does. A later entry of the same name and a second extract still work. A read-only file that the destination already holds is kept, and both call forms reject. A new test covers it.
  • Refused symlink (Archive.rs:1571): taken. The symlink arm is main's code again. A symlink that cannot be made is skipped, with and without a glob. A test pins it.
  • Path of a mkdir error (Archive.rs:1474): taken. The rejection names the directory that the system refused. sub/deep/inside.txt into a destination that is not writable gives EACCES, mkdir, <dest>/sub. Two tests cover it.
  • Failed read of entry data, failed header: not changed here. Bun.Archive: reject when a header read fails, with one in-memory tar reader and a named policy #43242 owns the read side of this loop.
  • Name that does not fit the path buffer: not changed. It is a skip by name, and extract(dir) skips the same entry.
  • Directory entry whose name is a regular file: not changed. Both call forms count it today. A check costs one more fstatat for each directory entry that exists.
  • Symlink whose name is taken: not changed. That arm is main's code again. archive: replace an existing file on extract instead of writing into it #43144 is the PR that replaces an existing node.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

Still open from earlier reviews (3):

  • Unresolved: 3 minor or pre-existing.

@robobun

robobun commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

Closing this draft. It was split in two: #46646 carries the extract(dir, { glob }) half and #45456 carries the files(glob) half. Three test cases exist only on this branch and are not in #46646 yet (test/js/bun/archive.test.ts at a0c0f971e4): a name that is too long for the file system, a symlink entry below a regular file, and the content of a read-only file after the rejection. Reopen this PR if a maintainer wants anything else from it.

@robobun robobun closed this Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants