Repository navigation
Conversation
|
Updated 12:59 PM PT - Oct 15th, 2026
✅ @robobun, your commit a0c0f971e4b6b69a0932905ef0b636bd0bf00bb7 passed in 🧪 To try this PR locally: bunx bun-pr 44731That installs a local version of the PR into your bun-44731 --bun |
|
Status: closed. This draft was split in two: #46646 carries the How I reproduced it. Linux x64, debug builds of main (
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
WalkthroughArchive glob matching now uses normalized extraction-style paths while ChangesArchive glob and extraction behavior
Suggested reviewers: Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to
|
There was a problem hiding this comment.
Findings marked 🟡 are optional suggestions and need no follow-up push.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟣
src/runtime/api/Archive.rs— Pre-existing: users who extract a truncated or corrupt archive with a glob still get a resolved promise with a smaller count, where extract(dir) rejects. At Archive.rs:1488-1491 a failed archive.read_data only sets write_success = false, so the partial file is unlinked and the loop moves on with no rejection. At Archive.rs:1397 a Fatal read_next_header just ends the while loop and Success(count) is returned. The docs and bun.d.ts added here say the glob form rejects when an entry cannot be written. Fix: map a negative read_data result and a Failed/Fatal read_next_header in the glob loop to ExtractResult::Err(ExtractError::ReadError), the same outcome extract_to_dir gives at src/libarchive/lib.rs:1439-1441 and :1887.Why this was flagged
An archive whose body is cut short is passed to archive.extract(dest, { glob: '**' }). For an entry whose data is missing, archive.read_data at Archive.rs:1487 returns a negative libarchive status; Archive.rs:1488-1491 sets write_success = false and breaks, Archive.rs:1524 unlinks the partial file, rejection is still None so no return happens, and the loop continues to the next header. When the next header is itself unreadable, read_next_header at Archive.rs:1397 returns Fatal, succeeded() is false, the loop ends and Archive.rs:1573 returns ExtractResult::Success(count). The caller gets a resolved promise with a count lower than the archive's entry count and no error. The base branch behaves the same way at these two sites, so this is pre-existing, but the non-glob path extract_to_dir returns Err(crate::Error::Fail) for both a Fatal header at src/libarchive/lib.rs:1439-1441 and a failed data read at src/libarchive/lib.rs:1887, so extract(dest) rejects for the same input. The new rejection variable at Archive.rs:1481 only covers bun_sys::write outcomes, not the libarchive read result.
Verification: Pre-existing: the base already continued on a failed read_data and returned Ok(count). Trigger: archive.extract(dest, { glob }) on a tarball truncated mid-entry. src/runtime/api/Archive.rs:1488-1491 sets write_success = false and no rejection, so the loop continues; line 1397 ends on Fatal and line 1573 returns Success(count). src/libarchive/lib.rs:1439-1441 and 1887 return Err(crate::Error::Fail).
…t a system call refuses With a glob, extract() resolved although entries were not written. The filtered extractor did `continue` after a failed mkdir, open or write. It now stops at the first directory or file that a system call refuses, and the promise rejects with that error (code, syscall, path). The error of a mkdir names the directory that the system refused. read_data_into_fd returns the error of a refused write, so that the caller can report it. A symlink that cannot be made is still skipped, as without a glob. With a glob, a file is created with the owner-write bit, as without a glob. A later entry of the same name or a second extract can then open a read-only entry. files(glob) matched the stored entry name, and extract() with a glob the normalized one. Both now match the normalized name, through the NormalizedName type. A leading "./" of a pattern is dropped.
158f2fa to
4b17f32
Compare
|
Pushed CI. Rebase. #44968 replaced the copy loop of the glob form with Review.
|
|
Closing this draft. It was split in two: #46646 carries the |
Related to #43132
Draft, to be re-cut (second self-review, Notes). The
files(glob)half is now #45456. The extract half waits for one answer: with a glob, does a refused mkdir or open reject likeextract(dir), or skip? The branch has the first cut.Problem
archive.extract(dir, { glob })resolves although entries were not written.extract_to_disk_filtered(src/runtime/api/Archive.rs:1385,:1421,:1424) doescontinueafter a failed mkdir, open or write.extract(dir)rejects.ReadError. Its mkdir and open arms stillcontinue.Fix
code,syscall,path). A file gets the owner-write bit.test/js/bun/archive.test.ts, CI green ata0c0f971e4b. Self-reviewed twice (Notes).Background
extract_to_disk_filtered. Other calls runArchiver::extract_to_dir, whichbun installshares.Downsides
extract(dir, { glob })rejects where it resolved with a smaller count. For a directory entry,extract(dir)still resolves (Notes).files(glob)half: anextract()job 152 to 160 bytes, release.text+2,304 bytes.Notes
State after the second self-review. The review ran on head
a0c0f971e4b. Its ruling: right problem, wrong packaging.extract_to_disk_filtered. It rejects a failed header, a failed data read and a refused write with a bareReadError, and it leavesread_data_into_fdas it is. This branch conflicts with it in the write arm and inarchive.test.ts, and it takes another policy in that arm (a system error for the glob form only, a changedread_data_into_fd, a failed data read still skipped).files(glob)half does not depend on any of that. It is Bun.Archive: match files(glob) against the normalized entry name, as extract() does #45456 now, with the fixes that the review asked for.ReadErrorstay, and only the three arms that it still skips are converted (directorymake_path, parentmake_path,openat).ReadErroror an error withcode,syscallandpath. Whetherread_data_into_fdchanges. The owner-write bit and a name that is already taken, which belong to the replace decision of archive: replace an existing file on extract instead of writing into it #43144.extract(dir)does, or 10 of 10 probed symlink-then-entry orders reject with a glob and resolve without one. The truncate and seek paths of a refused write need tests. On Windows the mkdir arms reportopenand a path that the call was not given, and three ordinary refusals (a locked file, a read-only attribute, an invalid name) are not tested.src/xto.envand a file entrysrc/x, it writessrc/.env. The recipe of main stays. Bun.Archive: match files(glob) against the normalized entry name, as extract() does #45456 does not touch it.The sections below are the account of the first cut.
Source of the report. An automated check found both defects. No user reported them. #43132 lists the glob skip in one row of its table: the destination holds a read-only
a.txt,extract(dir)rejects, and the glob form resolves with count 0. With this PR both call forms reject, and the file keeps its content.Three questions for a maintainer.
extract(dir)without a glob resolves and counts a directory entry that it cannot create (src/libarchive/lib.rs:1672to:1680). It also skips a symlink that it cannot create, as the glob form does. A fix is a policy inExtractOptionsthat onlyBun.Archivesets, becausebun install,bun createand the--compiledownload share that extractor. It exists as commit6aba634052on branchrobobun/a178f38c/archive-entry-errors, against an older main. It is not in this PR because it changesextract(dir)for callers that did not ask. Should it follow as its own PR?extract(dir)resolves,extract(dir, { glob })rejects (5 of 35 probed rows). If that is not acceptable, I move the directory arm of the glob form into the held part.Archiver::extract_to_dir, andextract_to_disk_filtereddeleted? The design review set it aside for now: it changes 21 of 29 probed glob results (count, file mode, symlink rules). This question does not gate this PR.Settlement per failure. Debug builds of main (
5f5799ed16) and of this branch (4b17f325e5), Linux x64. 35 rows: 14 as uid 0, 21 as uid 65534 (thechmodcases need a user that is not root).extract(dir)rejects and the glob form resolvesextract(dir)resolves and the glob form rejectsextract(dir)settles in another way than on mainThe 15 rows where the glob form still resolves: 4 controls, 7 symlink rows (the symlink is skipped, as without a glob), 2 directory entries whose name is taken by a regular file (see "Not fixed here"), and 2 read-only entries that a later entry or a second extract now overwrites.
What the glob form rejects with:
EACCES,open,<dest>/a.txtsub/deep/x.txtEACCES,mkdir,<dest>/subEACCES,mkdir,<dest>/sub/emptyEACCES,open,<dest>/r.txt(the file keeps its content)ENOTDIR,open,<dest>/blocker/x.txtENOTDIR,mkdir,<dest>/blocker/subEISDIR,open,<dest>/taken(EEXISTon Windows)ENAMETOOLONG,open,<dest>/<name>fresh/<300 bytes>/deep/x.txtENAMETOOLONG,mkdir,<dest>/fresh/<300 bytes>EFBIG,write,<dest>/big.txt(the partial file is removed)pathis the destination as the caller passed it, joined with the name that the failed call was given. When the directory of an entry cannot be made, the rejection is thatmkdirerror with the directory that the system refused. It is not theENOENTof theopenafter it.extract(dir)still rejects withError("ReadError")and nocode(#44509).Measurements. Release builds of main (
5f5799ed16) and of this branch (4b17f325e5), Linux x64.perf,valgrindandstraceare not installed here, so the counts come from ptrace tools (a syscall counter and anint3plus single-step instruction counter),objdump,nmandsize. A count per entry is the difference between an archive of 400 entries and one of 200, or of 2,000 and 1,000 for syscalls.extract():mov $0x98,%edibefore themalloccall on main,$0xa0here (152 to 160 bytes, both in the 160-byte class).ExtractResult8 to 16 bytes. A const assert keeps it at 16.extract(dir, { glob: "**" }): filesopenat1,000,pwrite641,000,close1,000. Nested files addmkdirat1,000. Directoriesmkdirat1,000. Symlinkssymlinkat1,000. Identical on both builds, also forextract(dir).files(). Identical on both builds.files("**"): 4,702 to 4,953 (+250, +5.3%) for 10-byte names. That is the one normalize pass.files()without a glob: 4,434 to 4,417.extract(dir, { glob: "**" }): file 5,491 to 5,465, nested file 6,057 to 6,020, directory 5,059 to 5,006, symlink 5,489 to 5,421. The path buffer is now taken once per call and not once per entry.extract(dir): file 4,835 to 4,836, nested file 5,429 to 5,433. Both are inside the spread..text: 80,758,501 to 80,760,805 bytes (+2,304). The file grows by 4,096 bytes. 12 functions change size. The larger ones: theExtractContextjob 3,808 to 4,431, theFilesContextjob 2,156 to 2,624,extract_to_dir::<()>6,030 to 6,314,ExtractTarball::runofbun install10,274 to 10,430,read_data_into_fd241 to 342. The streaming extractor ofbun installdoes not change (tarball_stream::drain_callback, 10,407 bytes on both).Behavior changes to know.
extract(dir, { glob })stops at a refused directory or file, asextract(dir)does. The entries after it are not written. Main wrote them.extract(dir)adds0o666to every file mode, so it did not have the case. On Windows the glob form no longer creates such a file read-only.EACCESfor a user that is not root. Main resolved and kept the old content.files("./src/*")andextract(dir, { glob: "./src/*" })now meansrc/*. On main the first matched only stored./names, the second matched nothing.files(glob)still lists an entry thatextract()leaves out for its name (second byte:). A test pins it.tar -C dir .archive, and passedsrc/x\.env, whichextract()writes as the hidden filesrc/x/.env.Not fixed here.
extract(dir): a directory entry that cannot be created (question 1)...component, a name whose second byte is:, hard links, FIFOs and devices.bun pm diffmatches its patterns against stored names.files(). Another change tracks it.Other open PRs on this function. #45307 rewrites the loop head and the write arm (see above). The others are mine, and none converts the mkdir and open arms.
extract(dir)) makes the same change toread_data_into_fd. It addsExtractResult::SysErrunboxed, so the const assert of this PR then needs its bound changed or the box kept.archive.test.ts. Whichever lands second changes those expectations to a rejection.Review history.
extract(dir)half. It raised 15 concerns. 9 are addressed. 1 is fixed on main since (the mkdir livelock, install: consolidate the open package manager pull requests #44968). 5 are for the held part: directory modes at mkdir time, the macOS errno of a trailing slash, the directory-over-a-file case, and two missing tests.openrefuses withEACCESalso deleted files that the call did not create. The owner-write bit replaces it. A rejection for a symlink that cannot be made differed fromextract(dir)and failed on a file system without symlinks. The symlink arm is main's code again.Tests run. Debug build (ASAN) of this head:
test/js/bun/archive.test.ts131 pass, 3 skip as root, 133 pass, 1 skip as uid 65534. Withsrc/of main (5f5799ed16): 10 of the 19 new tests fail as root, 13 as uid 65534. The 6 that pass on both pin what must not change (an existing directory, a symlink whose name is taken, a symlink that cannot be made, an entry that the glob leaves out, the:name, a long name).cargo clippy -p bun_libarchive -p bun_runtime: no warning from this diff.bun run rust:check-all: 12 targets ok. I cannot run Windows here. CI gave the Windows errno values of two tests (EEXIST,ENOENT).no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/bun/archive.test.ts