Repository navigation
Conversation
The text arm of expr_from_blob stored the body as a JSON-escaped string, and the printer escapes an 8-bit string again. Since v1.1.35 a body with a newline, a quote, a backslash or a non-ASCII character was inlined as its escaped form: "a\nb" became "a\\nb". Store the string that blob.text() returns for the bytes. A UTF-8 byte-order mark is dropped, a UTF-16LE one selects that encoding, and non-ASCII text is stored as UTF-16, like a literal from the lexer.
|
Status: ready for review. How to reproduce. Use a build without this change. // m.ts
export function text() {
return new Response("a\nb \"q\" \\ é", { headers: { "content-type": "text/plain" } });
}// index.ts
import { text } from "./m.ts" with { type: "macro" };
console.log(JSON.stringify(text()));bun index.ts
|
|
Updated 11:26 PM PT - Oct 23rd, 2026
✅ @robobun, your commit f7cc50a40b710c0f78d4e8d88dab2b576ffeab40 passed in 🧪 To try this PR locally: bunx bun-pr 46680That installs a local version of the PR into your bun-46680 --bun |
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Decode text-like Blob content src/js_parser_jsc/Macro.rs, test/bundler/transpiler/macro-test.test.ts |
expr_from_blob uses blob_text to create macro strings. The new test checks text from Blobs, Responses, and fetch(), including BOM handling and malformed UTF-8. It also checks empty stderr and a zero exit code. |
Suggested reviewers: jarred-sumner, dylan-conway
Priority:
Merge Risk: 
Minimal · up to f7cc5
-
The text-decoding change has no identified issue requiring a fix before merge.
Pre-merge checks: 
4
-
Full details: Description check
-
Explanation
The description explains the problem, fix, scope, verification results, regression context, tests, performance impact, and related changes. It does not use the template headings exactly, but it provides the required information in equivalent sections.
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I also checked: the UTF-16LE odd-trailing-byte drop and the lone-BOM edge (BOM::detect needs 3 bytes) match Blob::to_string_with_bytes in src/runtime/webcore/Blob.rs, which uses the same BOM::detect_and_split and len & !1 truncation, so blob_text is consistent with what .text() returns; an is_utf16 E::String from a macro is not a new shape, the T::String arm in Macro.rs already builds one via init_utf16; and all three branches copy into the bump arena, so nothing borrows the Blob's store past the call.
Extended reasoning...
The change replaces the double-escaping text path in expr_from_blob with a blob_text helper mirroring Blob.text() (BOM handling, ASCII as 8-bit, otherwise UTF-16 via init_re_encode_utf8) and adds one subprocess test covering Blob, Response, local fetch, both BOMs and malformed UTF-8. It touches no security-sensitive surface. The parity with Blob.rs and the existing UTF-16 macro-string shape rule out the encoding-edge concerns listed; the one confirmed finding is posted inline.
Problem
ResponseorBlobthat a macro returns is inlined as its JSON-escaped form: a newline becomes the two characters\n, andébecomes\u00E9. The documentedreturn fetch(url)of atext/*body is affected.expr_from_blob(src/js_parser_jsc/Macro.rs:1171) stores thequote_for_jsonform of the bytes as an 8-bit string, and the printer escapes an 8-bit string again.Fix
blob.text()returns for the bytes: ASCII as 8-bit, other text as UTF-16, like a literal from the lexer.Blob::to_string_with_bytes.test/bundler/transpiler/macro-test.test.ts, which fails on main.Background
Run::coerceturns the value a macro returns into AST nodes. For aBlob,expr_from_blobpicks JSON, text or a base64 data URL by MIME type.E::String) holds 8-bit bytes or UTF-16. The printer escapes both.init_re_encode_utf8over the raw bytes): it keeps a byte-order mark thatblob.text()drops.Downsides
Notes
Repro
bun index.tsprints["a\nb \"q\" \\ é","a\nb \"q\" \\ é"]["a\\nb \\\"q\\\" \\\\ \\u00E9","a\\nb \\\"q\\\" \\\\ \\u00E9"]I ran the two old releases to find the version. I did not bisect the commit. #15009 is in that range and changed how the printer writes strings.
What the inlined string is now. The test compares six macro results with the string
blob.text()gives:éand an emoji, as aBlob, as aResponse, and fetched from a local serverMeasurements (release, main 6292c65 against this PR, linux x64):
llvm-size: text 107,688,070 to 107,688,326.expr_from_blob2,493 to 2,602 bytes (blob_textis inlined into it).expr_from_blobper macro-returned textBlob, counted with gdb breakpoints on the mimalloc entry points over 101 call sites: ASCII 6 to 2, non-ASCII 6 to 3, empty 4 to 2.é: 61 to 72, because the text is UTF-16 now and was the escaped bytes before.Suites run on the debug build:
test/bundler/transpiler/macro-test.test.ts(35 pass).Related
quote_for_jsonlines withinit_re_encode_utf8(bump.alloc_slice_copy(bytes), bump). This PR does that and the byte-order marks, so that hunk of Store non-ASCII define, env, macro and inlined path strings as UTF-16 so they fold like literals #42019 can be dropped when it rebases.Str::check_lencalls around thequote_for_jsonlines that this PR deletes. After this PR one check ofbytes.len()at the top of the text arm covers both, because the literal is never longer than the body.Bun.file(), which main inlines as empty. It needs this PR: a text file almost always has a newline.