Skip to content

Preserve literal backslashes in fs.realpath on POSIX - #33410

Closed
robobun wants to merge 3 commits into
mainfrom
farm/8b04af5e/realpath-posix-backslash
Closed

robobun wants to merge 3 commits into
mainfrom
farm/8b04af5e/realpath-posix-backslash

Conversation

@robobun

@robobun robobun commented Jul 6, 2026 •

Copy link
Copy Markdown
Collaborator

What

On POSIX a backslash is an ordinary filename character, but fs.realpathSync and fs.realpathSync.native rewrote \ to / before touching the filesystem, so they threw ENOENT on a directory that existsSync / lstatSync resolve fine in the same process. Fixes #33403.

const fs = require("fs");
const dir = "/tmp/back\\slash"; // one literal backslash in the name
fs.mkdirSync(dir, { recursive: true });
fs.existsSync(dir);                 // true
fs.lstatSync(dir).isDirectory();    // true
fs.realpathSync(dir);               // before: ENOENT: no such file or directory, lstat '/tmp/back\slash'

Node v22+ returns /tmp/back\slash. path.normalize / resolve / join already preserved the backslash; only realpath was affected.

Cause

realpath_inner (non-Windows) resolves the input through FileSystem::abs_buf_checked, which joins with platform::Loose. The join's normalizer (normalize_string_buf_t) routed Platform::Posix through the loose normalizer, whose separator predicate is is_sep_any (treats both / and \ as separators). So back\slash was split and re-joined as back/slash, and the later openat targeted the wrong path:

access("/tmp/back\\slash", F_OK)           = 0
openat(AT_FDCWD, "/tmp/back/slash", O_RDONLY|O_PATH) = -1 ENOENT

Every other Platform::Posix separator predicate (is_separator, get_separator_func, is_absolute, leading_separator_index) already treats \ as a literal byte. The normalizer was the single inconsistent spot.

Fix

Scoped to realpath so no other path consumer shifts behavior:

  • Add normalize_string_posix_buf_t, a POSIX normalizer that splits on / only (vs the loose normalizer's is_sep_any, which also splits on \).
  • Thread a STRICT_POSIX const flag through _join_abs_string_buf; when set, the final normalization uses the POSIX normalizer. Every existing caller passes false, so their output is byte-identical.
  • Expose it only through join_abs_string_buf_checked_posix / FileSystem::abs_buf_checked_posix, and call that from realpath_inner. The shared normalize_string_buf_t and abs_buf_checked (Loose) are untouched, so module-specifier resolution and npm lockfile workspace-path folding (both of which intentionally treat \ as a separator) keep their behavior.

An earlier revision changed normalize_string_buf_t's Platform::Posix arm directly; CI showed that broke npm lockfile migration (packages\pkg1 workspace entries), so the fix was narrowed to realpath only.

Error shape

The issue also noted the thrown error reported syscall: "lstat". That was a side effect of the wrong path. For a genuinely missing path Bun already matches Node (lstat for fs.realpath, realpath for .native, original path), so no error-shape change was needed.

Test

test/js/node/fs/fs.test.ts creates a directory with a literal backslash in its name and asserts realpathSync, realpathSync.native, the promisified callback forms, and promises.realpath all return the path with the backslash intact (POSIX only; \ is a separator on Windows).

Verification

  • Fails before (USE_SYSTEM_BUN=1): realpathSync throws ENOENT.
  • Passes after (bun bd).
  • No regressions: test/js/node/fs/fs.test.ts (353 pass), test/cli/install/migration/migrate.test.ts (20 pass), test/js/node/path/ (122 pass), test/js/bun/resolve/{resolve,require}.test.ts (53 pass), test/bundler/bundler_edgecase.test.ts (107 pass).

@coderabbitai

coderabbitai Bot commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 66226c5b-02aa-41bf-a6b6-6dbe4c68eb6e

📥 Commits

Reviewing files that changed from the base of the PR and between f0e858f and 3ab9cc4.

📒 Files selected for processing (1)
  • src/paths/resolve_path.rs

Walkthrough

Path resolution now has a POSIX-strict mode that preserves backslashes as literal bytes. That mode is exposed through a new FileSystem helper, used by non-Windows realpath_inner, and covered by a regression test for several realpath variants.

Changes

POSIX-strict realpath resolution

Layer / File(s) Summary
Strict POSIX join/normalize core helpers
src/paths/resolve_path.rs
Adds STRICT_POSIX handling to the join helpers, introduces join_abs_string_buf_checked_posix, and adds normalize_string_posix_buf_t for slash-only separator normalization.
FileSystem API and realpath wiring
src/resolver/lib.rs, src/runtime/node/node_fs.rs, test/js/node/fs/fs.test.ts
Adds FileSystem::abs_buf_checked_posix, switches non-Windows realpath_inner to use it, and adds a POSIX-only regression test that preserves literal backslashes across realpath variants.

Possibly related issues

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title directly matches the main change: preserving literal backslashes in POSIX fs.realpath.
Description check ✅ Passed The PR description covers the fix, rationale, test coverage, and verification, though it uses custom headings.
Linked Issues check ✅ Passed The changes address #33403 by adding POSIX-only realpath normalization and tests that preserve literal backslashes.
Out of Scope Changes check ✅ Passed The added helpers, resolver change, and regression test all support the realpath backslash fix; no unrelated changes stand out.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 6, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 3:09 AM PT - Jul 6th, 2026

❌ @robobun, your commit bf3de47 has 1 failures in Build #68783 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33410

That installs a local version of the PR into your bun-33410 executable, so you can run:

bun-33410 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No bugs found, but the change to normalize_string_buf_t's Platform::Posix arm affects every platform::Auto/platform::Posix normalization on POSIX hosts (bundler, resolver, install, shell — 100+ call sites), so a maintainer should confirm the blast radius is acceptable.

Extended reasoning...

Overview

This PR fixes #33403: fs.realpath on POSIX was rewriting literal \ bytes to / before hitting the filesystem, causing spurious ENOENT on paths containing backslashes. The fix has two layers: (1) a new normalize_string_posix_buf_t that splits on / only, wired into normalize_string_buf_t's Platform::Posix arm on non-Windows hosts; (2) a new FileSystem::abs_buf_checked_posix used from realpath_inner in node_fs.rs. A test in fs.test.ts covers sync/native/callback/promise realpath variants.

Security risks

None identified. The change tightens path handling to match POSIX semantics rather than loosening it, and doesn't touch auth, crypto, or trust boundaries.

Level of scrutiny

High. The node_fs.rs and resolver/lib.rs changes are narrowly scoped, but the resolve_path.rs change alters the behavior of normalize_string_buf_t for Platform::Posix — and on POSIX hosts platform::Auto resolves to Posix. That routine backs normalize_buf, relative_platform_buf, _join_abs_string_buf, and transitively ~100+ call sites across the bundler, package manager, resolver, shell builtins, bake dev server, glob walker, etc. The delta is only observable for inputs containing \, which is rare on POSIX and semantically correct — but any caller that was (perhaps accidentally) relying on \→/ normalization for cross-platform module specifiers or config paths will now see different output. The author explicitly kept abs_buf_checked on Loose for module specifiers and preserved the loose behavior on Windows hosts via cfg!(windows), which shows awareness of this, but the broader Platform::Posix change still reaches many subsystems the targeted test runs (fs, path, resolve, bundler_edgecase) don't fully cover.

Other factors

The reasoning in the PR description is thorough and the fix is architecturally sound (aligning Platform::Posix with its other separator predicates). The test is well-constructed. CI (#68705) is still building at time of review. Given this is a foundational path helper touching many critical subsystems, a maintainer should sign off on whether the normalize_string_buf_t change is the right layer for the fix vs. keeping it scoped to realpath_inner only.

@robobun

robobun commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks, the scope of the normalize_string_buf_t::<Posix> change is the one real design decision here, so let me lay out why it lives at that layer and how bounded the delta is.

Why not scope it to realpath_inner only. realpath builds its absolute path with join_abs_string_buf_checked::<Posix> (via the new abs_buf_checked_posix). That join routes normalization through normalize_string_buf_t::<Posix>, so a realpath-local fix would mean either duplicating the ~100-line _join_abs_string_buf with a different normalizer, or threading a new const flag through the whole join/normalize call chain. Fixing the Posix arm is the owning-layer fix, and it makes Platform::Posix internally consistent: every other Posix separator predicate (is_separator, get_separator_func, is_absolute, leading_separator_index) already treats \ as a literal byte. The loose normalizer was the single inconsistent spot.

The delta is bounded to backslash inputs. is_sep_posix and is_sep_any differ only in how they treat \. For any path without a backslash the two produce byte-identical output, so the Auto/Posix call sites are unchanged except when an input actually contains a \. On POSIX a \ is a legal filename byte, so for those fs-path callers (fs.watch, cp, shell mv/rm, install node_modules paths, bundler output paths) treating it as literal is the correct behavior rather than a regression.

Module specifiers are untouched. The resolver's specifier joins go through abs_buf_checked, which I deliberately left on platform::Loose, so the intended "\ acts as a separator in an import specifier" behavior is preserved.

Windows is untouched. A few Windows callers reach the Posix arm with native \-separated paths to fold them to / (workspace --filter, relative-symlink fs.cp). The cfg!(windows) guard keeps the loose behavior there so those do not regress.

Suites exercised on Linux: fs.test.ts (353), node/path (122), resolve+require (53), bundler_edgecase (107), all green.

If you would rather keep the blast radius at zero and absorb the duplication, I can move the posix resolve into a realpath-local helper instead. Happy to go either way.

On POSIX a backslash is an ordinary filename byte, but realpath resolved
its input through a join that treated `\` as a separator and rewrote it
to `/`. realpathSync and realpathSync.native then ENOENT on a directory
that existsSync and lstatSync resolve in the same process.

realpath now resolves with strict POSIX separator semantics via a new
abs_buf_checked_posix / join_abs_string_buf_checked_posix that splits on
`/` only, so backslashes survive. This is threaded through the existing
join via a STRICT_POSIX flag; the shared normalizer is unchanged, so
callers that intentionally fold `\` to `/` (module specifiers on Loose,
npm lockfile workspace paths) keep their behavior.
@robobun
robobun force-pushed the farm/8b04af5e/realpath-posix-backslash branch from 6ca6d08 to f0e858f Compare July 6, 2026 04:31
@robobun

robobun commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Update: CI confirmed the blast-radius concern was real. The broad normalize_string_buf_t::<Posix> change broke test/cli/install/migration/migrate.test.ts ("npm lockfile with relative workspaces"): npm lockfiles can carry Windows-authored workspace paths like packages\pkg1, and the migration relies on the loose normalizer folding \ to / on POSIX.

I reworked it to the scoped approach. The shared normalizer is reverted (unchanged from main), and realpath now gets strict POSIX separator semantics through a STRICT_POSIX flag threaded into the existing join, reached only via the new join_abs_string_buf_checked_posix / FileSystem::abs_buf_checked_posix. So the only behavior change is in realpath; every other Auto/Posix caller is byte-identical.

Verified on Linux: migrate.test.ts (20 pass), fs.test.ts (353 pass incl. the new backslash test), node/path + resolve (165 pass).

Comment thread src/runtime/node/node_fs.rs Resolved
@robobun

robobun commented Jul 6, 2026 •

Copy link
Copy Markdown
Collaborator Author

CI status for 3ab9cc4: green on every lane that exercises this change (282 jobs passed). The red lanes are unrelated to the diff, which only touches realpath / POSIX path joining:

  • :darwin: 26 aarch64 - test-bun: buildkite-agent artifact download timed out after 120s fetching the darwin-aarch64-build-bun artifact. Infra/S3, not a test failure; it has recurred across builds 68705 and 68739.
  • :ubuntu: 25.04 x64-baseline - test-bun: test/js/bun/util/v8-heap-snapshot.test.ts killed by SIGKILL with no core (OOM on a memory-heavy test), unrelated to path handling.
  • :windows: 11 aarch64: test/bake/deinitialization.test.ts flagged as a flaky retry (warning), also unrelated.

The realpath fix and its regression test pass on the Linux lanes, and the earlier npm-lockfile-migration regression is resolved (migrate.test.ts green; only one Linux lane red here, from the OOM flake, vs 13 before the scope-down). Ready for review from my side.

Update: re-triggered on a clean commit (build 68783) and the :darwin: 26 aarch64 - test-bun artifact-download timeout recurred, so it is now on three consecutive builds (68705, 68739, 68783), i.e. persistent CI infra unrelated to this diff rather than something a re-run resolves. Not re-triggering further; the diff itself is green on every lane that exercises it.

Build 68783 also showed a segfault on :windows: 11 aarch64 in test/js/node/test/parallel/test-https-proxy-request-tunnel-timeout-agent.mjs (an HTTPS proxy timeout test) and a flaky test/cli/hot/hot.test.ts retry on 2019 x64-baseline. Neither is related to this change: the segfault is in a networking test, and on Windows this diff is a no-op for path resolution (realpath's POSIX branch is #[cfg(not(windows))], and the join_abs_string_buf_checked refactor threads STRICT_POSIX=false for every existing caller, so the Windows output is byte-identical to main). The re-run surfacing different unrelated flakes each time is itself the signal that these are CI flakes, not the diff.

@cyberdude

Copy link
Copy Markdown

You can do it @robobun !!! Just find the relationship between words and find the Next TOKEN!!!! That's all you doing! YOU GO ROBOBUN!!

@robobun

robobun commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #42374. That PR makes POSIX realpath call the OS canonicalizer. The emulated variants normalize the path with the existing POSIX normalizer, and the native variants pass the path to the OS unchanged. So a literal backslash stays in the name.

I checked a debug build of main plus #42374 on Linux. The repro from #33403 returns /tmp/back\slash, and the test from this PR passes. fs.realpathSync, fs.realpath, their .native forms and fs.promises.realpath all match Node.js. I tested absolute and relative paths that contain a backslash, with string and Buffer input.

#42965 covers the module resolver side of the same problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fs.realpathSync converts backslash to slash on Linux, throws ENOENT for existing paths

2 participants