Skip to content

node:fs: write UTF-8 when fs.write gets an unknown encoding name - #43435

Open
robobun wants to merge 1 commit into
mainfrom
robobun/8fedc0a9/fs-write-unknown-encoding
Open

robobun wants to merge 1 commit into
mainfrom
robobun/8fedc0a9/fs-write-unknown-encoding

Conversation

@robobun

@robobun robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • fs.writeSync(fd, "XY", null, "bogus") throws TypeError [ERR_INVALID_ARG_VALUE]: encoding 'bogus' is an invalid encoding. fs.write and filehandle.write do the same. Node.js v26.12.1, the version Bun reports, writes UTF-8.
  • With a null or undefined position this regressed in 1.4.0 (node:fs: apply the string encoding in write/writeSync and other Node compat fixes #32813). No user reported it.
  • Cause: args::Write::from_js (src/runtime/node/node_fs.rs:4227) parses the encoding with Encoding::assert, which throws for an unknown name.

Fix

  • The parser reads the argument as Node's WriteString does. A known name in a primitive string selects that encoding. Every other value means UTF-8.
  • Correct for Node.js through v26: WriteString uses ParseEncoding(args[3], UTF8), which never throws.
  • Verified: test/js/node/fs/fs-write-unknown-encoding.test.ts (node:test form). 44 of 51 tests fail on main. Node v26.12.1 passes the file. Also ran fs.test.ts and 20 vendored Node tests.
  • Self-reviewed: 13 concerns raised, 12 addressed (Notes).

Background

Downsides

  • Node.js v27 rejects an unknown encoding here with ERR_UNKNOWN_ENCODING (crypto,fs: reject invalid encodings nodejs/node#66247). Until Bun ports that check, a mistyped name such as "base-64" writes the text with no error. A maintainer must choose: the pin (this PR) or Node 27.
  • A String object is no longer read as an encoding: new String("hex") writes UTF-8, as in Node.
  • An empty-string encoding costs 48 more instructions per call (591 to 639), a known name 166 fewer. Release text grows by 256 bytes.
Notes

The decision: the pin or Node 27. Bun reports Node.js v26.12.1. Node.js v27.0.0-alpha.0.0.0 (2026-10-29) changed this behaviour on purpose: nodejs/node#66247 (semver-major) added if (normalizedEncoding === undefined) throw new ERR_UNKNOWN_ENCODING(encoding) to the JS validateEncoding (https://github.com/nodejs/node/blob/v27.0.0-alpha.0.0.0/lib/internal/validators.js#L419-L423). The native WriteString is the same in both (ParseEncoding(args[3], UTF8)).

Encoding argument of a string write Node v24.0.0 to v26.13.0 Node v27.0.0-alpha Bun 1.4.x, main This PR
"bogus" writes UTF-8 ERR_UNKNOWN_ENCODING, nothing written ERR_INVALID_ARG_VALUE, nothing written writes UTF-8
5 writes UTF-8 ERR_UNKNOWN_ENCODING writes UTF-8 writes UTF-8
new String("hex"), data "abcd" 4 bytes, UTF-8 4 bytes, UTF-8 2 bytes, hex 4 bytes, UTF-8
  • Pin: this PR. node:crypto, webcrypto: a fourth batch of Node compatibility fixes #47671 did the same for cipher.update on 2026-11-02, against Node v26.12.1. The port of the v27 check then comes with the move to v27.
  • Node 27 now: close this PR and port the check to validateEncoding (src/jsc/bindings/NodeValidatorBodies.h:279). The vendored v26 test-fs-promises-file-handle-write.js then fails, because it passes a number in this argument. Under Node v27.0.0-alpha it fails with Unknown encoding: 16. Upstream's new test-fs-write-invalid-encoding.js fails on main (other error code) and with this PR (no error).
  • node:fs: do the I/O of a FileHandle stream through the handle #47445 asks a maintainer the same question for streams.

Repro. Node v24.0.0 to v26.13.0 and Bun 1.3.14 print 2 "XY..". Bun 1.4.0, 1.4.4 and main throw ERR_INVALID_ARG_VALUE. Node v27.0.0-alpha throws ERR_UNKNOWN_ENCODING.

const fs = require("node:fs"), os = require("node:os"), path = require("node:path");
const p = path.join(os.tmpdir(), "probe.txt");
fs.writeFileSync(p, "....");
const fd = fs.openSync(p, "r+");
console.log(fs.writeSync(fd, "XY", null, "bogus"), JSON.stringify(fs.readFileSync(p, "latin1")));
fs.closeSync(fd);
Call on a file that holds .... Node v26.12.1 Bun 1.3.14 Bun 1.4.0, 1.4.4, main This PR
writeSync(fd, "XY", null, "bogus") 2, XY.. 2, XY.. throws 2, XY..
writeSync(fd, "XY", 0, "bogus") 2, XY.. throws throws 2, XY..
fs.write(fd, "XY", null, "bogus", cb) callback (null, 2, "XY") as Node throws from the call as Node
filehandle.write("XY", null, "bogus") resolves, 2 bytes as Node rejects as Node
writeSync(fd, "é", null, "bogus") 2 bytes, UTF-8 as Node throws as Node
writeSync(fd, "XY", "bogus") 2, XY.. throws 2, XY.. 2, XY..
encoding "", encoding 5 2, XY.. as Node as Node as Node
writeFileSync(p, "XY", "bogus") throws throws throws throws

Node v24.0.0, v24.22.2, v26.3.0 and v26.13.0 give the v26.12.1 column. These 5 releases are the ones I ran.

Node source. WriteString reads the encoding with ParseEncoding(isolate, args[3], UTF8): https://github.com/nodejs/node/blob/v26.12.1/src/node_file.cc#L3204. ParseEncoding returns the default for a value that is not a string and for an unknown name: https://github.com/nodejs/node/blob/v26.12.1/src/api/encoding.cc#L125-L136.

History. Through v1.1.33 this argument was read with Encoding.fromJS(current) orelse Encoding.utf8. #14937 (v1.1.34) made the encoding parses in node_fs strict with Encoding.assert, this one included. Its tests port Node's test-fs-assert-encoding-error, which does not cover fs.write. Until 1.4.0 the parser reached the encoding only after a numeric position, so a call with a null position never ran the check. #32813 (1.4.0) made the parser consume the position whatever its type, as Node does, and the check then ran for every call.

This PR before. It was stacked on #43373, which replaces the utf16-le key of Bun's name table with utf-16le. #47559 added utf-16le to the table, so this PR now targets main and does not touch the table. The tests moved from fs.test.ts to their own file in node:test form, the form a maintainer asked for on #37632, the previous fix to this parser.

Tests. test/js/node/fs/fs-write-unknown-encoding.test.ts has 43 tests that Node can run and 8 for Bun's fs.promises.write.

Runtime Result
Node v24.0.0, v24.22.2, v26.3.0, v26.12.1, v26.13.0 43 of 43 pass
Node v27.0.0-alpha.0.0.0 12 pass, 31 fail (it rejects the unknown names)
Bun, release build of main 7ca0042 7 pass, 44 fail
Bun with this PR, debug (ASAN) and release builds 51 of 51 pass
  • Vendored Node tests, all pass on the debug build with the fix: test-fs-write.js, test-fs-write-sync.js, test-fs-write-sync-optional-params.js, test-fs-write-optional-params.js, test-fs-write-buffer.js, test-fs-write-negativeoffset.js, test-fs-write-no-fd.js, test-fs-write-reuse-callback.js, test-fs-writesync-crash.js, test-fs-assert-encoding-error.js, test-fs-internal-assertencoding.js, test-fs-read-file-assert-encoding.js, test-fs-promises-file-handle-write.js, test-fs-promises-file-handle-writeFile.js, test-fs-promises-writefile.js, test-fs-append-file-sync.js, test-fs-append-file.js, test-fs-write-file.js, test-fs-write-file-sync.js, test-fs-write-stream-encoding.js.
  • fs.test.ts on the debug build with the fix: the 39 tests that match encoding and the 191 that match rite pass. On release builds of 7ca0042 with and without the change, every other test of the file passes, except keeps a non-throwing callback in the same place in the event loop. That test fails in about half of the runs on both builds (an fs.stat callback against a setImmediate).
  • BUN_JSC_validateExceptionChecks=1 BUN_JSC_dumpSimulatedThrows=1 on the new file: 51 pass, 0 reports of an unchecked exception.
  • I found no call of fs.write with an unknown name in Node's v26 test-fs-write*.js, so there is no upstream v26 test to copy.
  • Windows: not run here. The parser runs before both back ends (the thread pool on POSIX, uv_fs_write on Windows). The read-only fd tests expect EBADF, which the existing test when the call fails with EBADF already expects on Windows for the callback form.

Self-review. 13 concerns, 12 addressed: the base and the body, the Node version (v26.12.1, not v26.3.0), the Node 27 reversal, the reason for the NUL names, the open PRs on the same lines, the tests in node:test form, and the cost text. Not taken: a test that spawns node on this file in every CI run, as six http and tls test files do. On a cold debug build here that one test took 4.7 s and 5.1 s against the 5 s limit, and fs-write-offset-bound.test.ts, the file from #37632, has no such test. Upstream's test-fs-write-invalid-encoding.js is the test that fails when the v27 check is missing.

Matrix. 1428 calls of fs.writeSync(fd, data, position, encoding): 102 values in the encoding argument, 7 strings, positions null and 0. Compared by return value and file bytes with Node v26.12.1, on release builds of 7ca0042 with and without the change.

Equal to Node v26.12.1 Different
main 768 660
This PR 1278 150
  • No call that matched Node on main differs with this PR.
  • 588 calls that threw on main now write. 30 calls still throw, and Node throws the same error for them (odd-length string with a hex name).
  • 24 calls that returned on main return other bytes now. All of them pass new String("hex") or new String("utf16le"). 18 of the 24 now match Node. In the other 6 Node throws (see below).

Not in this PR. The 150 calls that still differ, and the related differences I found:

Call Node v26.12.1 Bun with this PR Site
writeSync(fd, s, 0, "utf16-le") (36 calls) UTF-8 UTF-16LE the utf16-le key of ENCODING_MAP, src/runtime/node/types.rs:593
a name with a NUL, for example "hex\0" (96 calls) reads the name up to the NUL UTF-8 this parser, on purpose
writeSync(fd, "abc", 0, new String("hex")), and an odd-length string with another object whose string form is hex (18 calls) throws ERR_INVALID_ARG_VALUE UTF-8 validateEncoding, src/jsc/bindings/NodeValidatorBodies.h:279
a String object whose toString throws throws that error UTF-8 the same validator
writeFileSync(p, "6162", new String("hex")) 36 31 36 32 61 62 WriteFile::from_js_with_default_flag, and the same gate in ReadFile::from_js, Readdir::from_js, parse_encoding_arg and Encoding::assert
  • utf16-le: the key stays. node:crypto, webcrypto: assorted Node compatibility fixes #47559 added utf-16le beside it. node: accept the utf-16le encoding alias and reject utf16-le #43373 (open, conflicts with main) would remove it. Other lookups share the table (Bun.CryptoHasher, readFile, writeFile and more).
  • NUL names: Node v26.12.1 reads the name as a C string, so "hex\0" is hex there. Bun writes UTF-8, and this is a deliberate difference. Node v27 rejects these names. No other encoding reader in Bun stops at a NUL: Buffer.from("abcd", "hex\0") throws ERR_UNKNOWN_ENCODING, as in Node. A cut at the NUL would match only the v26 line.
  • The validator: Node's validateEncoding is duck-typed. It reads enc.length and calls toString on a value of length 3 to 9. Bun's reads primitive strings only. In a probe of 17 String objects and other objects at 2 positions (34 calls), main matches Node on 14 and this PR on 18: 12 calls that differed now match, and 8 calls that matched now differ. All 8 pass a String object with a toString that throws, which main called in the native parser. I found no open PR that changes this validator.
  • String objects in the other fs calls: Node's getOptions takes an encoding only from a primitive string. Those parsers must keep their throw for an unknown name, so they need a different change. node:fs: reject a non-string, non-object options argument like node #41928 and node:fs: take a function in the options slot as the default options #46369 (open) rework the same parsers.

Cost, measured. Release builds (ThinLTO) of main 7ca0042 and of the change on Linux x64. Instructions and calls are counted inside args::Write::from_js, callees included, with gdb stepi and the other threads held. 5 calls of each kind, all 5 counts equal.

Call main This PR
writeSync(fd, "x") 517 instructions, 9 calls 517, 9
await filehandle.write("x") 664, 12 664, 12
writeSync(fd, "x", null, "utf8") 873, 15 707, 13
fs.write(fd, "x", cb) (the JS wrapper passes "utf8") 1020, 18 854, 16
writeSync(fd, "x", null, name), name is "utf8" in a 16-bit string 1209, 21, 1 mi_malloc and 1 mi_free 792, 14, no allocation
writeSync(fd, "x", null, "") 591, 11 639, 13
writeSync(fd, "x", null, "bogus") throws: 5491 to 7557 instructions, creates a TypeError 712, 13, no allocation
  • Binary: size reports text 140,290,979 on main and 140,291,235 with the change (+256 bytes). Data and bss are equal, and the stripped file has the same size. Of 161,833 function symbols, 2 differ: args::Write::from_js shrinks from 2852 to 2833 bytes, and a standalone copy of Encoding::from_js (142 bytes) is new.
  • Syscalls, counted with a gdb syscall catchpoint: the unknown-name call makes 0 write calls on main and 1 with the change. The known-name call makes 1 on both.
  • The PR adds no host function, no binding entry and no startup work.

Related PRs.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/fs/fs.test.ts

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the empty-string encoding path ("" previously hit assert's falsy short-circuit and now misses ENCODING_MAP and falls to unwrap_or — same UTF-8 result either way) and the String-object asymmetry with readFile/writeFile/readdir (pre-existing and disclosed in the description; those calls take an options object in Node, so they need a separate change).

Extended reasoning...

The diff is a 5-line change in args::Write::from_js (src/runtime/node/node_fs.rs:3710-3712) plus two tests. I read Encoding::from_js, Encoding::assert, from_js_with_default_on_empty, in_map_case_insensitive and JSValue::is_string_literal to confirm the new path cannot throw for a primitive string, that non-ASCII names miss the map (UTF-16 narrowing returns None; Latin-1 bytes never case-fold to ASCII keys), and that utf-16le is already in ENCODING_MAP at the stacked base commit so the new test rows pin behavior rather than depending on this PR. The only behavior regression versus base is the "hex\0" name, which is covered by the inline finding; nothing else in the ruled-out set turned up a second issue. No debug build was present, so the tests were assessed by reading rather than run.

Comment thread src/runtime/node/node_fs.rs Resolved
@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 2:38 AM PT - Nov 8th, 2026

✅ @robobun, your commit 63df7755a82d9b38f1ef91f448ca26861ec42415 passed in Build #138547! 🎉


🧪   To try this PR locally:

bunx bun-pr 43435

That installs a local version of the PR into your bun-43435 executable, so you can run:

bun-43435 --bun

@robobun

robobun commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed

Nothing new to post: everything this review found is already covered by existing comments on this pull request or didn't merit a separate one.

fs.writeSync(fd, "XY", null, "bogus") threw ERR_INVALID_ARG_VALUE and
wrote nothing. fs.write and filehandle.write did the same. Node.js
through v26 writes the string as UTF-8. For a null or undefined
position, Bun 1.3.14 wrote UTF-8 too, and 1.4.0 started to throw
(#32813).

args::Write::from_js parsed the encoding with Encoding::assert, the port
of Node's assertEncoding. Node's WriteString reads that argument with
ParseEncoding(args[3], UTF8), which never throws. The parser now does
the same: a primitive string that names a known encoding selects it, and
every other value means UTF-8.

Node.js v27 rejects an unknown name in its JS validateEncoding
(nodejs/node#66247). That check is not part of this change.
@robobun
robobun force-pushed the robobun/8fedc0a9/fs-write-unknown-encoding branch from 8ea823e to 63df775 Compare November 8, 2026 04:06
@robobun
robobun requested a review from alii as a code owner November 8, 2026 04:06
@robobun
robobun changed the base branch from robobun/e6df67d5/fix-utf-16le-alias to main November 8, 2026 04:07
@coderabbitai

coderabbitai Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 2596ac5 and 63df775.

Files selected for processing: 2

  • src/runtime/node/node_fs.rs
  • test/js/node/fs/fs-write-unknown-encoding.test.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@robobun
robobun removed the request for review from alii November 8, 2026 04:08

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the latest push and didn't find any bugs; because the diff range from the recorded base spans the whole upstream history merged since September (36k files) rather than just this fix, a human look at the two files the PR itself changes would still be worthwhile.

What was reviewed:

  • The is_string() -> is_string_literal() switch and Encoding::from_js(..).unwrap_or(default) in src/runtime/node/node_fs.rs against Node's ParseEncoding(args[3], UTF8): a String object or unknown name now falls through to UTF-8 and the argument slot is left unconsumed, matching V8's IsString() semantics; the JS-side validateEncoding in src/js/node/fs.ts still runs first, so the hex even-length check is unchanged.
  • test/js/node/fs/fs-write-unknown-encoding.test.ts: covers sync, callback, promisified, FileHandle and fs.promises.write forms with known/unknown/object encodings and both null and numeric positions; cleanup is in after/finally and no external hosts are touched.
  • Not done in this run: the test file was read, not executed, so I did not confirm it fails on the pre-fix build.
Extended reasoning...

The PR's own commit changes one argument-parsing branch in src/runtime/node/node_fs.rs (fs.write/writeSync/filehandle.write string form) so an unknown or non-primitive encoding selects UTF-8 instead of throwing ERR_INVALID_ARG_VALUE, and adds a node:test-based regression file. It touches no security-sensitive surface; the only semantic shift is that new String("hex") now means UTF-8 (as in Node) rather than hex. The recorded base SHA is stale, so the review scope covers ~36k upstream files that are not part of this fix, the PR conversation history was incomplete this run, and an earlier inline thread of mine on this exact line was closed by author self-resolution with a reply I cannot see — those facts rule out approving, while the small real change and the lack of findings make a human look an additional step rather than a blocker.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant