Skip to content

bundler: report the emitted size of each chunk as metafile outputs[..].bytes - #42760

Merged
Jarred-Sumner merged 1 commit into
mainfrom
robobun/9e0bed10/metafile-output-bytes
Sep 15, 2026
Merged

Jarred-Sumner merged 1 commit into
mainfrom
robobun/9e0bed10/metafile-output-bytes

Conversation

@robobun

@robobun robobun commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • In the JSON metafile (bun build --metafile, Bun.build({ metafile: true })), outputs[path].bytes is smaller than the emitted file when the chunk references another output. On 1.4.3 m0.js is 130 bytes on disk, 98 in the metafile. bun.d.ts documents the field as "Size of the output file in bytes".
  • The cause is generate_chunk_json (src/bundler/linker_context/MetafileBuilder.rs:75). It ran before code() and wrote IntermediateOutput::get_size() (src/bundler/Chunk.rs:610), which adds only the literal pieces. It leaves out every path and HTML manifest that code() writes, and the //# debugId and //# sourceMappingURL comments.

Fix

  • Chunk::final_output_size records the length of the emitted buffer. Each place that makes the final buffer sets it: two in generateChunksInParallel.rs (standalone HTML, in-memory) and one in writeOutputFilesToDisk.rs.
  • metafile_builder::generate builds each chunk fragment. Both callers run it after generate_chunks_in_parallel, so the size is known. esbuild defers this fragment in the same way.
  • Removed: get_size(), Chunk::metafile_chunk_json, and the unsafe detach_lifetime() in generate. The fragment was its only borrow that was not 'static.
  • Verified: test/bundler/metafile.test.ts, 15 new cases, all fail on 1.4.3. Also the esbuild/metafile, bundler_compile_splitting, standalone, bundler_html and html-import-manifest suites.

Background

  • The bundler prints a chunk before its final path exists, because the path holds a content hash. A reference to another output is a placeholder (a unique key) in the printed text.
  • IntermediateOutput::Pieces holds the printed text, split at each placeholder. code() joins the pieces and writes the final path at each split.
  • The bundler then appends the source map comment and writes the buffer to disk, or returns it as a BuildArtifact.
Notes

Repro (1.4.3 and main at 5fce36e):

mkdir -p src
printf 'import { v1 } from "./m1.ts";\nimport { v2 } from "./m2.ts";\nconsole.log(v1 + v2);\n' > src/m0.ts
printf 'import c from "./c3.cjs";\nexport const v1 = "V1" + c.v3;\nconsole.log(v1);\n' > src/m1.ts
printf 'import c from "./c3.cjs";\nexport const v2 = "V2" + c.v3;\nconsole.log(v2);\n' > src/m2.ts
printf 'module.exports = { v3: "V3" };\n' > src/c3.cjs
bun build --splitting --target=bun --outdir=out --metafile=meta.json src/m0.ts src/m1.ts src/m2.ts
output on disk bytes before bytes after
./m0.js (2 imports) 130 98 130
./m0-5begjve3.js (1 import) 195 179 195
./m1.js (1 import) 74 58 74
./m0-8x1hvvzq.js (no import) 1400 1400 1400

Each import costs 16 bytes, the length of ./m0-xxxxxxxx.js. Other cases measured on 1.4.3:

  • --sourcemap=linked: m0.js is 207 on disk, the metafile said 98.
  • --sourcemap=inline: 611 on disk, 98 in the metafile.
  • --sourcemap=external: 176 on disk (the //# debugId comment, which code() writes), 98 in the metafile.
  • import logo from "./logo.svg" (file loader): 98 on disk, 79 in the metafile.
  • An HTML entry point: index.html is 218 on disk, 160 in the metafile.
  • A server build that imports an HTML file (the manifest placeholder): server.js now matches, 889 bytes.

esbuild 0.25.1 on a split build with source maps reports the exact file size (a.js: 146 on disk, 146 in the metafile). Its linker passes len(outputContents) to jsonMetadataChunkCallback(finalOutputSize) after it appends the sourceMappingURL comment.

The new tests compare outputs[key].bytes with the size of each emitted file:

  • Code splitting with each sourcemap mode (none, linked, inline, external), with an outdir (the disk loop) and without (the in-memory loop).
  • An asset referenced from JS, and a 128 KB asset referenced from CSS. A stylesheet inlines a smaller file as a data: URL.
  • An HTML entry point.
  • A standalone HTML file (compile with target: "browser"). This is the only case that reaches the third assignment. The test compares the bytes of each inlined chunk with the text between its tags in the document.
  • The CLI with --metafile and --metafile-md. The test reads the [ENTRY: ... (N bytes)] lines of the report.

With any one of the three assignments removed, the chunks of that path report 0 and its cases fail.

More builds checked by hand against the fixed binary, all equal to the file on disk: --banner and --footer, a shebang entry with --target=bun, --minify with --sourcemap=linked, --format=cjs --bytecode, --public-path, --entry-naming '[dir]/[name]-[hash].[ext]' with entry points in subdirectories, and a CSS entry point with an @import and a 200 KB url() asset.

Why a field on the chunk and not OutputFile.size: a standalone HTML build gives each inlined chunk an empty placeholder OutputFile and then filters those out of the list. The index of a chunk and the index of its output file match only by a debug_assert, and not in that mode.

What changes for a user:

  • bytes grows by the paths, by the debugId comment, and by the sourceMappingURL comment. With sourcemap: "inline" that comment holds the whole map. The number now equals BuildArtifact.size.
  • The --metafile-md report uses the same numbers for "Total output size", "Bundle size", the chunk sizes and the "% of Total" column.
  • --compile --target=browser inlines the JS and CSS chunks into the document. index.html now reports its real size (403, was 203). An inlined chunk keeps its outputs entry, as before. Its bytes is the size of the inlined code.
  • --compile embeds each chunk in the executable. bytes is the size of the embedded module.
  • An error from generate_chunk_json now fails generate with the existing "Failed to generate metafile" warning. Before, unwrap_or_default() left an empty fragment and the JSON was not valid. The only fallible call is a write to a Vec.

Not changed here, tracked in #42758:

  • .map, .jsc and copied asset files have no outputs entry. This is a different defect (a missing key, not a wrong value) and it adds keys that users see.
  • inputs[..].bytesInOutput counts a placeholder at its own length. A fix needs a count per compile result after the substitution, as in esbuild's accurateFinalByteCount.
  • A standalone HTML build lists the inlined chunks as outputs.

Self-reviewed: 5 concerns raised, 5 addressed. A test for the standalone HTML assignment. The unsafe that this change made needless. Stricter test assertions (statSync, exact key lists, one combined CLI assertion). The debugId wording in the commit message. A tracker for the gaps above.

Open PRs that touch MetafileBuilder.rs but not this field: #38341, #38624, #36974, #38666. None reads state that is gone when generate runs. #38666 has a small text conflict in Chunk.rs where this PR deletes get_size().


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/bundler/metafile.test.ts

…].bytes

The metafile fragment of a chunk was built before code() ran, from
IntermediateOutput::get_size(). That sum leaves out every path and
HTML manifest code() writes over a placeholder, the debugId comment
code() adds, and the sourceMappingURL comment appended after it.

Record the length of the emitted buffer on the chunk where the output
is made, and build the fragment in metafile_builder::generate, which
runs after generate_chunks_in_parallel. get_size() and
Chunk::metafile_chunk_json have no other use and are removed. With
that borrow gone every borrowed node of the joiner in generate() is
'static, so the unsafe detach_lifetime() goes too.
@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

On-demand reviews are free for the next 6 days. After that, they cost $0.25 per reviewed file.

Or wait 6 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 5b65b76e-8f49-44a9-8883-3536e6ba27a5

📥 Commits

Reviewing files that changed from the base of the PR and between dad5177 and 93429ba.

📒 Files selected for processing (5)
  • src/bundler/Chunk.rs
  • src/bundler/linker_context/MetafileBuilder.rs
  • src/bundler/linker_context/generateChunksInParallel.rs
  • src/bundler/linker_context/writeOutputFilesToDisk.rs
  • test/bundler/metafile.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator Author

Status

  • Reproduced on 1.4.3: bun build --splitting --target=bun --outdir=out --metafile=meta.json src/m0.ts src/m1.ts src/m2.ts. out/m0.js is 130 bytes, outputs["./m0.js"].bytes is 98. The full script is in the Notes block of the description.
  • Fail before: USE_SYSTEM_BUN=1 bun test test/bundler/metafile.test.ts -t "bytes is the size" gives 0 pass, 15 fail.
  • Pass after: bun bd test test/bundler/metafile.test.ts gives 65 pass, 0 fail.
  • The gaps that this PR does not change are in Metafile: outputs omit source maps, bytecode and copied assets, and bytesInOutput counts placeholders #42758.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — a clean root-cause fix that records the emitted buffer length at each of the three emission sites and reads it after emission.

What was reviewed:

  • Verified metafile_builder::generate runs after generate_chunks_in_parallel at both call sites in bundle_v2.rs, and that write_output_files_to_disk is invoked from within it, so final_output_size is set before it is read.
  • Checked that generate_chunk_json no longer reads any state (intermediate_output) consumed between the old and new call sites; every field it reads survives to the generate call.
  • Confirmed the detach_lifetime() removal is sound: the only remaining push_static inputs are b"…" literals and ImportKind::label() -> &'static [u8]; the fragment now goes through push_owned.
  • Test file already imports bunExe/bunEnv (line 865), so the new CLI test resolves; subprocess pipes are drained concurrently and the matrix covers all sourcemap modes × on-disk/in-memory plus the standalone-HTML path for the third assignment.
Extended reasoning...

Overview

This PR fixes outputs[..].bytes in the bundler metafile so it reports the actual emitted file size rather than the pre-substitution intermediate size. It replaces Chunk::metafile_chunk_json: Box<[u8]> with Chunk::final_output_size: usize, sets that field at each of the three places a chunk's final buffer is produced (generateChunksInParallel.rs standalone-HTML scc loop, the in-memory insert_for_chunk path, and writeOutputFilesToDisk.rs), and defers generate_chunk_json until metafile_builder::generate, which both callers in bundle_v2.rs already invoke after generate_chunks_in_parallel. Dead code is removed in the same commit: IntermediateOutput::get_size(), the pre-loop that populated metafile_chunk_json, and the unsafe { j.detach_lifetime() } with its SAFETY comment.

Security risks

None. The change touches only metafile JSON assembly and a usize field on Chunk; no user input parsing, no filesystem path handling beyond what already existed, no new unsafe (one unsafe is removed). The removed detach_lifetime was the only lifetime escape hatch in the function, and I confirmed every remaining push_static argument is 'static (ImportKind::label() returns &'static [u8]), so the borrow checker now enforces what the old SAFETY comment claimed.

Level of scrutiny

Moderate. This is a self-contained bug fix in a diagnostic output path (the metafile), not runtime behavior. The main risk was a state-ordering regression from moving generate_chunk_json to run later — I traced every field it reads (final_rel_path, files_with_parts_in_chunk, cross_chunk_imports, entry_point, content.css_chunks, c.graph.meta) and none is consumed or mutated between the old and new call sites; the only consumed field was intermediate_output, which the new code no longer reads. Error handling improved slightly: unwrap_or_default() (which produced invalid JSON on error) is replaced with ? propagation into the existing "Failed to generate metafile" warning.

Other factors

Test coverage is thorough per REVIEW.md's variant-matrix rule: test.each over four sourcemap modes × on-disk/in-memory, copied assets from JS and CSS (with a 128 KB asset to defeat data-URL inlining), HTML entry points, standalone compile: true HTML (the only path exercising the scc assignment), and the CLI --metafile/--metafile-md flags. Tests use tempDir, bunExe/bunEnv (already imported mid-file at line 865), Buffer.alloc(n, fill), drain stdout/stderr/exited concurrently, and assert structured toEqual on { reported, actual } maps. No CODEOWNERS entry covers the changed paths. No prior review comments or outstanding objections on the PR.

@robobun

robobun commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 5:04 PM PT - Sep 14th, 2026

✅ @robobun, your commit 93429ba1977cb69458efd084da72b12ffedd6a49 passed in Build #115690! 🎉


🧪   To try this PR locally:

bunx bun-pr 42760

That installs a local version of the PR into your bun-42760 executable, so you can run:

bun-42760 --bun

@Jarred-Sumner
Jarred-Sumner merged commit 3ef1657 into main Sep 15, 2026
11 of 12 checks passed
@Jarred-Sumner
Jarred-Sumner deleted the robobun/9e0bed10/metafile-output-bytes branch September 15, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants