Skip to content

bundler: let DCE drop unreachable top-level await before rejecting CJS - #29246

Closed
robobun wants to merge 14 commits into
mainfrom
farm/38b7d2c8/dce-top-level-await-cjs
Closed

robobun wants to merge 14 commits into
mainfrom
farm/38b7d2c8/dce-top-level-await-cjs

Conversation

@robobun

@robobun robobun commented Apr 13, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

bun build --format=cjs rejected any top-level await at parse time, even when the await sat inside a branch that constant folding / dead code elimination would have removed:

// entry.js
if (typeof TEST === "undefined" ? false : TEST) {
    await import("node:fs");
    // Do something with the filesystem
}
foo();
$ bun build entry.js --minify --format=cjs --define TEST=false
2 |     await import("node:fs");
              ^
error: "await" can only be used inside an "async" function

esbuild handles this fine: the if (false) { ... } block is dropped by DCE and the unreachable await is never reported. That blocks users who use --define to gate test-only code paths behind an import("node:fs") that never runs in production.

Cause

At module scope, when the target output format didn't support top-level await (--format=cjs), the parser used allow_await = .allow_ident. That treats await as a plain identifier and relies on a lexer-level prev_token_was_await_keyword flag to emit the TLA error the moment a follow-up token can't be interpreted as an identifier continuation (e.g. the import keyword). The diagnostic fires at lex/parse time — well before constant folding has a chance to eliminate the dead branch.

Fix

Parse await EXPR as an await expression at module scope even when the output format is CJS, as long as the following token unambiguously starts an expression rather than continuing an identifier (so var await = 42 and console.log(await) still work as identifier uses). Track "live" top-level awaits separately in the visit pass (has_live_top_level_await), and only let them imply .esm after DCE has run. If a live top-level await survives DCE in a CJS target, emit the diagnostic after visit:

error: Top-level await is currently not supported with the "cjs" output format

This matches esbuild's behaviour.

Verification

test/regression/issue/29243.test.ts covers 21 cases. The core four:

  • CJS + dead await → builds to foo();, no error (the bug).
  • CJS + live await → rejected with the new diagnostic, not the old lex-level one.
  • var await = 42; console.log(await) in a CJS file → still parses as identifier use.
  • await inside a non-async function → still reports the existing "await" can only be used inside an "async" function error with the "Consider adding the async keyword here" note.

Review rounds added coverage for: dead/live for await loops and their diagnostic ranges, tagged templates, await {}, default parameters of non-async functions and arrows, top-level return plus a dead await, TS await! / await as T / await satisfies T, await using declarations, TS namespace and declare enum bodies, and dead switch/with statements. The last case also fixes the dead-control-flow filter (should_keep_stmt_in_dead_control_flow): switch/with husks were kept wholesale in dead branches, which would have leaked their await-bearing test expressions into non-async CJS output; they now recurse into their bodies like the other statement kinds.

The dead-await cases regress on the installed bun but pass on this build.

Fixes #29243


[review] gate passed · iteration 15 · 10 files touched

fails on main (without fix)
ASAN without fix: BUILD FAILED (no junit output)
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/regression/issue/29243.test.ts"
ninja: Entering directory `/workspace/bun/build/debug'
[1/166] gen generated_host_exports.rs
generated_host_exports.rs: 94 exports (host=3, lazy=10, generic=81, rust=0); 238 extern-C blocks audited
[2/166] gen cpp.rs (cppbind)
[3/166] gen JSSink.{cpp,h,lut.h,rs}
generated_jssink.rs: 8 sinks, 96 exported symbols
Generating /workspace/bun/build/debug/codegen/JSSink.lut.h from /workspace/bun/build/debug/codegen/JSSink.lut.txt
[4/166] gen ZigGeneratedClasses.{cpp,h,rs}
Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts
  - ResolveMessage (13 fields)
  - BuildMessage (10 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts
  - Archive (4 fields, 1 class fields)
Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts
  - ResourceUsage (8 fields)
  - Subprocess (20 fields)
Found 1 classes from /workspace/bun/src/runtime/api/cron.classes.ts
  - CronJob (5 fields)
Found 3 classes from /workspace/bun/src/runtime/api/filesystem_router.classes.ts
 
... (truncated)

release without fix: 1 FAILED
bun test v1.4.0-canary.1 (a070fd192)

test/regression/issue/29243.test.ts:
(pass) bun build --format=cjs drops an unreachable top-level await before reporting TLA [18.06ms]
(pass) bun build --format=cjs still rejects a live top-level await [16.23ms]
(pass) await can still be used as an identifier at module scope in CJS output [15.93ms]
(pass) await inside a non-async function nested in a CJS file still reports a useful error [15.64ms]
(pass) bun build --format=cjs drops a literal if (false) top-level await [15.91ms]
(pass) await as a tagged-template call identifier keeps working in CJS [15.36ms]
(pass) bun build --format=cjs drops an unreachable await of an object literal [14.81ms]
(pass) bun build --format=cjs still rejects a live top-level for await loop [13.56ms]
(pass) bun build --format=cjs rejects await in a default parameter of a non-async function [13.05ms]
(pass) bun build --format=cjs rejects await in a default parameter of a non-async arrow [12.47ms]
(pass) bun build --format=cjs drops an unreachable top-level for await loop [14.07ms]
(pass) bun build --format=cjs underlines the await token of a live for-await loop [11.97ms]
(pass) bun build --format=cjs 
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" "test/regression/issue/29243.test.ts"
bun test v1.4.0 (f38e79913)

test/regression/issue/29243.test.ts:
(pass) bun build --format=cjs still rejects a live top-level await [216.21ms]
(pass) await inside a non-async function nested in a CJS file still reports a useful error [208.69ms]
(pass) bun build --format=cjs drops an unreachable top-level await before reporting TLA [316.50ms]
(pass) await can still be used as an identifier at module scope in CJS output [283.57ms]
(pass) bun build --format=cjs drops a literal if (false) top-level await [281.19ms]
(pass) await as a tagged-template call identifier keeps working in CJS [294.18ms]
(pass) bun build --format=cjs still rejects a live top-level for await loop [234.81ms]
(pass) bun build --format=cjs drops an unreachable await of an object literal [309.44ms]
(pass) bun build --format=cjs rejects await in a default parameter of a non-async function [240.80ms]
(pass) bun build --format=cjs drops an unreachable top-level for await loop [327.93ms]
(pass) bun build --format=cjs rejects await in a
... (truncated)

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 684ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/125] gen generated_host_exports.rs
generated_host_exports.rs: 94 exports (host=3, lazy=10, generic=81, rust=0); 238 extern-C blocks audited
[2/125] gen ZigGeneratedClasses.{cpp,h,rs}
Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts
  - ResolveMessage (13 fields)
  - BuildMessage (10 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts
  - Archive (4 fields, 1 class fields)
Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts
  - ResourceUsage (8 fields)
  - Subprocess (20 fields)
Found 1 classes from /workspace/bun/src/runtime/api/cron.classes.ts
  - CronJob (5 fields)
Found 3 classes from /workspace/bun/src/runtime/api/filesystem_router.classes.ts
  - FileSystemRouter (5 fields)
  - FrameworkFileSystemRouter (2 fields)
  - MatchedRoute (8 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Glob.classes.ts
  - Glob (5 fields)
Found 1 classes from /workspace/bun/src/runtime/api/h2.classes.ts
  - H2FrameParser (31 fields)
Found 8 
... (truncated)
diff hotspot
src/ast/s.rs                            |   2 +
 src/js_parser/p.rs                      |  37 +-
 src/js_parser/parse/parse_entry.rs      |  29 ++
 src/js_parser/parse/parse_prefix.rs     | 128 +++++--
 src/js_parser/parse/parse_stmt.rs       |  16 +-
 src/js_parser/scan/scan_side_effects.rs |  38 +-
 src/js_parser/visit/visit_expr.rs       |  12 +
 src/js_parser/visit/visit_stmt.rs       |  23 +-
 src/react_compiler/codegen.rs           |   1 +
 test/regression/issue/29243.test.ts     | 640 ++++++++++++++++++++++++++++++++
 10 files changed, 879 insertions(+), 47 deletions(-)

gate history · 7 passed · 0 rejected · iteration 15

evidence per changed file
file                                     reads  edits  tests
src/ast/s.rs                                 2      3     51
src/js_parser/p.rs                          14     11     49
src/js_parser/parse/parse_entry.rs           4      6     49
src/js_parser/parse/parse_prefix.rs          9     11     49
src/js_parser/parse/parse_stmt.rs            2      2     49
src/js_parser/scan/scan_side_effects.rs      4      3     49
src/js_parser/visit/visit_expr.rs            1      1     49
src/js_parser/visit/visit_stmt.rs            3      3     49
src/react_compiler/codegen.rs                2      1     50
test/regression/issue/29243.test.ts         15     18     49

root cause · written by the author bot

The parser rejected top-level await at parse time, before dead code elimination ran, so an await inside a branch that would later be proven unreachable still produced an error in CJS builds. The fix parses await tolerantly at module scope and defers validation to after the visit pass, recording only awaits encountered in live control flow via a new has_live_top_level_await flag, so unreachable awaits are eliminated along with their enclosing dead branches and an error is reported only when a live top-level await survives in a format that cannot support it. It also corrects var relocation fo…

@robobun

robobun commented Apr 13, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 1:49 AM PT - Aug 4th, 2026

❌ Your commit f38e7991 has 1 failures in Build #88669 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 29246

That installs a local version of the PR into your bun-29246 executable, so you can run:

bun-29246 --bun

@coderabbitai

coderabbitai Bot commented Apr 13, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Distinguishes parse-time TLA tokens from live top-level await discovered during the visit pass, records live TLA, removes unreachable TLA prior to ESM classification and diagnostics, refines parsing disambiguation for await vs identifier at module scope, and adds regression tests for CJS/TLA behavior.

Changes

Cohort / File(s) Summary
Parser state & behavior
src/ast/P.zig, src/ast/Parser.zig
Added has_live_top_level_await: bool = false to parser state; compute tla_implies_esm from parse-time keyword + feature flag; use it for ESM/export classification and strict-mode inference; after visit, clear or report surviving TLA based on DCE and options.features.top_level_await.
Parsing disambiguation (await vs identifier)
src/ast/parsePrefix.zig, src/ast/parseStmt.zig
Added tokenStartsAwaitExpr and AwaitOrYield binding; conditionally reinterpret await identifier as an await expression at module top-level in non-ESM CJS contexts when next token permits, temporarily overriding fn_or_arrow_data_parse.allow_await; relaxed parse-time for await validation to allow tolerated module-scope for await when allow_await == .allow_ident and is_top_level.
Visit-time live-TLA recording
src/ast/visitExpr.zig, src/ast/visitStmt.zig
e_await and s_for_of now set p.top_level_await_keyword and p.has_live_top_level_await for the first live module-scope await encountered during reachable control-flow outside functions/arrows.
Tests (regression)
test/regression/issue/29243.test.ts
Added suite exercising bun build --format=cjs with minification across reachable vs unreachable TLA and for await scenarios, await as identifier/tagged-template cases, and asserts on build success/failure and diagnostics.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR directly addresses issue #29243: it defers top-level await validation until after DCE runs, allows await parsing at module scope when unambiguous, and emits diagnostics post-visit matching esbuild behavior.
Out of Scope Changes check ✅ Passed All changes scope to the DCE + top-level await fix: parser modifications, visit pass tracking, test coverage, and for-await handling are directly related to the linked issue objective.
Title check ✅ Passed The title clearly summarizes the main change: allowing DCE to remove unreachable top-level await before rejecting CJS output.
Description check ✅ Passed The description explains the problem, cause, fix, verification, regression coverage, and linked issue in sufficient detail.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/ast/visitExpr.zig`:
- Around line 988-990: When marking a live top-level await inside the visitor
(the branch that sets p.has_live_top_level_await via
p.fn_or_arrow_data_visit.is_outside_fn_or_arrow), also record the location/token
for that live await so downstream CJS diagnostics use the surviving await
location instead of the original parse-time one; update
p.top_level_await_keyword (or the parse-state field that holds the await
span/token) to the current await token/span when you set
p.has_live_top_level_await, ensuring you only overwrite it when the new location
corresponds to a live await discovered during this visit.

In `@test/regression/issue/29243.test.ts`:
- Line 41: Remove the flaky empty-stderr assertions in the test by deleting the
two occurrences of expect(stderr).toBe("") (the assertions at lines that
reference stderr in test/regression/issue/29243.test.ts); keep the existing
exit-code and stdout/content assertions intact (do not change assertions using
exitCode or stdout), and ensure the variable stderr is still collected if used
elsewhere but no longer asserted to be exactly "" to avoid ASAN-induced flakes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: dde02832-03b2-4a9d-8a33-8b3e382e96e5

📥 Commits

Reviewing files that changed from the base of the PR and between d7526e2 and 86d144e.

📒 Files selected for processing (5)
  • src/ast/P.zig
  • src/ast/Parser.zig
  • src/ast/parsePrefix.zig
  • src/ast/visitExpr.zig
  • test/regression/issue/29243.test.ts

Comment thread src/ast/parsePrefix.zig Resolved
Comment thread src/js_parser/visit/visit_expr.zig Resolved
Comment thread test/regression/issue/29243.test.ts Resolved
@dragoncoder047

Copy link
Copy Markdown

Copying my comment in the linked issue in case @robobun hasn't seen it:

This bug has nothing to do with --define since omitting the --defines and using the code if (false) { await import("foo"); } still triggers the bug!

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
test/regression/issue/29243.test.ts (1)

29-29: ⚠️ Potential issue | 🟠 Major

Remove empty-stderr assertions from success-path subprocess tests.

Line 29 and Line 70 make these tests flaky on ASAN CI, where benign warnings may appear on stderr even when behavior is correct.

Proposed patch
-  const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
+  const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]);

-  expect(stderr).toBe("");
   expect(stdout).toBe("foo();\n");
   expect(exitCode).toBe(0);
-  const [stdout, stderr, exitCode] = await Promise.all([proc.stdout.text(), proc.stderr.text(), proc.exited]);
+  const [stdout, exitCode] = await Promise.all([proc.stdout.text(), proc.exited]);

-  expect(stderr).toBe("");
   expect(stdout).toContain("var await = 42");
   expect(stdout).toContain("globalThis.output = await");
   expect(exitCode).toBe(0);

Based on learnings: in test/regression/issue/*.test.ts, avoid asserting spawned Bun subprocess stderr is empty because ASAN may emit benign warnings.

Also applies to: 70-70

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/regression/issue/29243.test.ts` at line 29, Remove the brittle
empty-stderr assertions in the regression test by deleting the
expect(stderr).toBe("") checks (e.g., the assertion at expect(stderr).toBe("")
in test/regression/issue/29243.test.ts and the similar one around line 70),
leaving the success-path subprocess tests to only assert on stdout/exit
behavior; locate the assertions by searching for expect(stderr).toBe("") in the
file and remove them so ASAN benign warnings no longer cause flakes.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@test/regression/issue/29243.test.ts`:
- Line 29: Remove the brittle empty-stderr assertions in the regression test by
deleting the expect(stderr).toBe("") checks (e.g., the assertion at
expect(stderr).toBe("") in test/regression/issue/29243.test.ts and the similar
one around line 70), leaving the success-path subprocess tests to only assert on
stdout/exit behavior; locate the assertions by searching for
expect(stderr).toBe("") in the file and remove them so ASAN benign warnings no
longer cause flakes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 309a8c18-d823-45aa-964f-06b5d39f07bd

📥 Commits

Reviewing files that changed from the base of the PR and between 86d144e and 43d383a.

📒 Files selected for processing (1)
  • test/regression/issue/29243.test.ts

Comment thread src/js_parser/ast/Parser.zig Resolved
Comment thread src/ast/parsePrefix.zig Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/ast/Parser.zig`:
- Around line 649-663: The parser currently clears top_level_await_keyword when
p.has_live_top_level_await is false because s_for_of never sets that flag for
top-level "for await" loops; update the s_for_of visitor (visitStmt.zig) so that
when encountering a for-of statement with data.is_await == true AND the
statement is at module scope (p.current_scope == p.module_scope) you set
p.has_live_top_level_await = true (and ensure p.top_level_await_keyword remains
pointing to the live await range), so the diagnostic in Parser.zig that checks
p.has_live_top_level_await will be preserved for top-level for-await loops.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 57506c01-8895-4880-b9db-3c8aec88640b

📥 Commits

Reviewing files that changed from the base of the PR and between 43d383a and 481e42c.

📒 Files selected for processing (4)
  • src/ast/Parser.zig
  • src/ast/parsePrefix.zig
  • src/ast/visitExpr.zig
  • test/regression/issue/29243.test.ts

Comment thread src/js_parser/ast/Parser.zig Resolved
Comment thread src/ast/parsePrefix.zig Resolved
Comment thread src/ast/parsePrefix.zig Resolved
Comment thread src/ast/P.zig Resolved
Comment thread src/ast/visitStmt.zig Resolved
Comment thread src/ast/parseFn.zig Resolved
Comment thread src/js_parser/parse/parse_prefix.zig Resolved
Comment thread src/js_parser/visit/visit_stmt.zig Resolved
Comment thread src/js_parser/ast/parsePrefix.zig Resolved
Comment thread src/ast/parsePrefix.zig Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is in good shape after extensive review — all identified bugs have been addressed — but the parser changes are complex enough to warrant a human look before merging.

Extended reasoning...

Overview

The PR defers top-level-await rejection in CJS targets from parse time to post-DCE visit time, touching seven parser source files (, , , , , , ) and adding a 452-line regression test.

Security risks

No direct security impact — this is pure parser/bundler logic with no auth, crypto, or user-data exposure.

Level of scrutiny

High. This modifies the JavaScript/TypeScript parser's module-scope disambiguation path, which runs on every CJS build. The change introduces a new scope-stack walk (), a token whitelist (), a new live-TLA tracking flag, and a post-visit error path. The number of edge cases that surfaced during review (TS non-null assertion, for-of/in contextual keywords, / operators, arrow default params, class field initializers, TS namespace scopes, for-await squiggle range) suggests the problem space is larger than it first appears.

Other factors

All reported issues were addressed across multiple fix commits (481e42c → 1cfe041). The final scope-stack approach is more robust than the earlier flag approach. Test coverage is comprehensive. The PR is a net correctness improvement (matching esbuild's behaviour) but given the volume of iterations and the criticality of the code path, a human review is the right call before landing.

@robobun

robobun commented Apr 13, 2026

Copy link
Copy Markdown
Collaborator Author

CI is still failing on debian-13-x64-baseline-test-bun on 1cfe041 and I'm unable to read the specific test output without Buildkite auth. I've iterated through 6 review rounds fixing edge cases (TS namespace scopes, class field initializers, default param values inside arrow args, await! in TS, for (await of …), as/satisfies infix, for-await diagnostic range drift), and the local regression suite of 18 tests passes cleanly on the debug+ASAN build. All review threads resolved.

The underlying code change — speculatively upgrading an await identifier to an await expression at module scope so DCE can drop dead branches — has more parser-context interactions than I originally scoped for, and each fix has surfaced another edge case. Claude's latest review (which acknowledged all fixes landed) flagged the PR as complex enough to warrant human review before merging. I concur. Flagging for maintainer triage rather than continuing to blind-patch against CI without visibility into the failing step.

The 18 regression tests cover the path the original issue is on (if (false) { await import(...) } with and without --define), plus all edge cases surfaced during review. If the remaining CI failure is an unrelated flake it'll go away on rebase; if it's a genuine regression from this PR's scope walk, it'd be most efficient for a maintainer with log access to point at the failing test so I can reproduce.

@robobun
robobun force-pushed the farm/38b7d2c8/dce-top-level-await-cjs branch from 9647a34 to 78152a0 Compare May 6, 2026 11:08
Comment thread src/js_parser/ast/Parser.zig Resolved
Comment thread test/regression/issue/29243.test.ts Resolved
Comment thread src/js_parser/ast/parseStmt.zig Resolved
Comment thread src/js_parser/ast/parsePrefix.zig Resolved
Comment thread src/js_parser/parse/parse_entry.zig Resolved
Comment thread test/regression/issue/29243.test.ts Resolved
Comment thread src/js_parser/p.zig Resolved
@robobun
robobun force-pushed the farm/38b7d2c8/dce-top-level-await-cjs branch from f20b3bb to 3bb72fe Compare May 16, 2026 19:32
Comment thread src/js_parser/parse/parse_prefix.zig Resolved
Comment thread src/js_parser/parse/parse_prefix.zig Resolved
@robobun
robobun force-pushed the farm/38b7d2c8/dce-top-level-await-cjs branch from 9eaa6db to 85c7417 Compare June 27, 2026 17:55
@robobun

robobun commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main. The conflict was non-trivial: main completed the Zig→Rust parser migration and deleted all the src/js_parser/**/*.zig files this PR originally modified, so every Zig-touching commit hit a modify/delete conflict.

Resolution: the fix was already ported to Rust in a prior commit, so I collapsed the branch to a single clean commit containing only the net Rust changes (src/js_parser/{p,parse/parse_entry,parse/parse_prefix,parse/parse_stmt,visit/visit_expr,visit/visit_stmt}.rs, src/ast/s.rs) plus the regression test, reapplied onto current main. Also added the new await_range field to the S::ForOf construction site in src/react_compiler/codegen.rs that main introduced.

All 18 regression tests pass on the debug+ASAN build; original issue #29243 reproduction is fixed.

@robobun

robobun commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator Author

CI status: the diff's own lanes are green. The parser changes compile and link on every platform (69/73 steps pass, including all build lanes and the --revision smoke test), and the test/regression/issue/29243.test.ts suite passes.

The remaining red is unrelated flake on heavy tests, reproduced identically across two consecutive runs on this same commit (builds 65604 and 65695):

  • test/js/bun/util/v8-heap-snapshot.test.ts — SIGKILL (OOM) on ubuntu 25.04 x64
  • test/bake/dev-and-prod.test.ts — SIGKILL on Windows (retried, flagged flaky)
  • test/js/bun/webview/webview-chrome.test.ts, test/napi/napi.test.ts, test/cli/inspect/inspect.test.ts, test/js/bun/spawn/spawn-pipe-leak.test.ts, test/cli/install/bun-install-security-provider.test.ts — code 1 / timeout on assorted lanes

Build 65695 has zero ERROR annotations (only the flaky WARNING group); the one hard-failed darwin-26-aarch64 step is flaky-retry exhaustion on those tests. None of them touch the parser (src/js_parser/**, src/ast/s.rs, src/react_compiler/codegen.rs) — they're heap-snapshot / dev-server / webview / napi / spawn OOM and timeout flakes on memory-constrained runners.

CI already re-ran on its own (65604 → 65695) and hit the same class of failure, so this needs a maintainer to merge rather than another retrigger.

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

CI status: the diff is green on all lanes that exercise it. Build 65604's failures are unrelated flake:

  • test/js/bun/util/v8-heap-snapshot.test.ts — SIGKILL / OOM-killed on the ubuntu-25.04 x64 runner (no core file), a known CI memory flake.
  • test/bake/dev-and-prod.test.ts, test/cli/install/bun-install.test.ts, test/napi/napi.test.ts — all flagged flaky (passed on retry).

None touch the parser (src/js_parser/**, src/ast/s.rs) or top-level-await handling. A ci: retrigger re-roll (531fe82) is already on the branch, so I'm not pushing another. The 18 regression tests pass locally on the debug+ASAN build and the original issue #29243 reproduction is fixed. This needs a maintainer to merge through the flaky lanes.

Comment thread src/runtime/cli/test/Timings.rs Resolved
Comment thread src/runtime/cli/test/Timings.rs Resolved
robobun and others added 9 commits August 4, 2026 01:31
`bun build --format=cjs` rejected any top-level `await` at parse time,
even when it sat inside a branch that dead-code elimination would
remove (e.g. behind a `--define`-controlled `if (false) { await
import(...) }`). esbuild parses the `await`, lets DCE drop the
unreachable branch, and only reports the CJS / TLA incompatibility if
a live `await` survives.

Match that behavior:

- `P::is_at_module_scope()` walks the scope stack (stops at the real
  module `Entry`, returns false on function/class/TS-namespace
  scopes) to decide when an `await` identifier at module scope in a
  non-ESM target should be upgraded to an `await` expression.
- `token_starts_await_expr()` gates the upgrade on the following
  token: keywords / literals / `{` / `~` / a non-contextual
  identifier force the keyword reading, while `of`/`in`/`as`/
  `satisfies` and template/operator continuations keep `await` as an
  identifier. `!` is left out so TS `await!.foo` non-null assertions
  parse.
- `has_live_top_level_await` is set by the `e_await` / `s_for_of`
  visitors only in reachable control flow at module scope, and
  records the surviving `await`'s range for the diagnostic.
- post-visit in `parse_entry`: in non-ESM targets, clear the pre-DCE
  range if nothing survived, or emit "Top-level await is currently
  not supported with the \"X\" output format" if it did. Gated on
  `!features.top_level_await` so ESM / bake-dev builds are untouched.
- `for await` and the `s_return` ESM-classification check get the
  same module-scope / feature gating.

Regression test covers dead/live TLA, `for await`, `await` as an
identifier (incl. tagged template / `as` / `satisfies`), default
params, TS namespaces, and the diagnostic range.

Fixes #29243
…located

A switch or with kept wholesale in dead control flow leaked its
await-bearing test expression into non-async CJS output. Recurse into
their bodies like the other statement kinds; hoisted declarations are
already relocated out by the visit pass. Also remove the unreachable
in exclusion (reserved word, never tokenizes as an identifier).
@robobun
robobun force-pushed the farm/38b7d2c8/dce-top-level-await-cjs branch from 50e2dff to 7a28526 Compare August 4, 2026 01:32

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/runtime/cli/test/Timings.rs:1-6 — This file appears to be an accidental inclusion — it implements bun test --timings=<path> shard-balancing, which is entirely unrelated to the CJS top-level-await parser fix. It has no mod Timings; declaration anywhere in the tree (so it isn't compiled into any crate), no git history on any branch, no test coverage, and is absent from this PR's own "diff hotspot" and "evidence per changed file" tables. Please drop it from this PR (git rm src/runtime/cli/test/Timings.rs) and land it separately with its own mod wiring and tests.

    Extended reasoning...

    What the issue is. src/runtime/cli/test/Timings.rs is a brand-new 242-line file implementing bun test --timings=<path> / --update-timings shard-balancing (loading per-test-file duration tables, partitioning files across shards by cost, sorting slowest-first, writing timing JSON). None of this has any relation to the PR's stated purpose — deferring the CJS top-level-await diagnostic until after DCE runs (#29243).

    Evidence that this is an accidental inclusion, verified against the current tree:

    1. Not compiled. grep -rn 'mod Timings\|Timings::' src/ returns zero matches. src/runtime/cli/test/ contains only ChangedFilesFilter.rs, ParallelRunner.rs, Scanner.rs, and parallel/ — there is no mod Timings; declaration wiring this file into any crate. Cargo will not compile it; it is dead source.
    2. No git history. git log --all -- src/runtime/cli/test/Timings.rs returns nothing. The file has never existed on any branch in this repository.
    3. The PR's own metadata omits it. The PR description's "diff hotspot" diffstat lists exactly 10 files changed — Timings.rs is not among them. The robobun "evidence per changed file" table (reads/edits/tests) also lists exactly 10 files and omits Timings.rs. Yet GitHub's changed-files list shows 11 files. The author-bot's own instrumentation never observed this file being read, edited, or tested during the 14 iterations that produced this PR.
    4. No test coverage. test/regression/issue/29243.test.ts contains 21 tests, all exercising bun build --format=cjs await behavior. Nothing touches --timings.

    Step-by-step proof that the file is orphaned dead code. Suppose this PR merges as-is:

    1. Timings.rs lands at src/runtime/cli/test/Timings.rs.
    2. The Rust module tree for src/runtime/cli/test/ is defined by whatever mod declarations exist in its parent — currently only ChangedFilesFilter, ParallelRunner, Scanner, and parallel are declared (verified: no mod Timings anywhere in src/).
    3. cargo build therefore never sees Timings.rs. It is not type-checked, not compiled, and cannot be called from anywhere.
    4. The types it references (super::parallel::file_range::FileRange, bun_options_types::context::Shard, etc.) may or may not exist / match — nothing verifies this because nothing compiles it.
    5. The result is 242 lines of untested, uncompiled, unreviewed feature code sitting in the tree with no history explaining where it came from, riding on a parser bugfix PR.

    Why this is flagged as blocking. REVIEW.md (the repo's merge-blocking checklist, "everything here has blocked merges") states: "Every line you add must be demonstrably live" — this file has no consumer and is not compiled. "Delete dead code in the same PR that makes it dead" — this file is born dead. "Don't ride file-wide standardization on a focused bugfix" and "New cross-cutting abstractions need maintainer agreement before appearing inside a feature PR" — a 242-line --timings shard-balancing subsystem is a new cross-cutting abstraction with zero relation to the CJS TLA fix, and it appears without maintainer discussion, tests, or even a mention in the PR description.

    Impact and fix. No runtime behavior is affected (the file isn't compiled), but merging it lands untested, unwired, unreviewed code into the tree under the cover of an unrelated bugfix — exactly what REVIEW.md's dead-code and scope rules exist to prevent. Fix: git rm src/runtime/cli/test/Timings.rs and force-push. If the --timings feature is intended, it should land in its own PR with a mod Timings; declaration, callers, and test coverage.

Comment thread test/regression/issue/29243.test.ts Resolved
@robobun

robobun commented Aug 4, 2026 •

Copy link
Copy Markdown
Collaborator Author

The diff is complete and all review threads are resolved. The only failures on the latest CI run (build 88669) are unrelated to this change: one pre-existing worker_threads stress-test SIGABRT that also fails on main (reported separately), and a handful of known-flaky tests that passed on retry (s3, hot reload, bun-install, napi builds on Windows aarch64). All 24 regression tests plus the transpiler, bundler edgecase, minify, and dead-code suites pass locally on this branch. Ready for a maintainer to review and merge.

Comment thread src/js_parser/scan/scan_side_effects.rs Resolved
…bodies

An async or generator function declaration surviving in a dead switch
case, with body, or loop body kept the whole statement, leaking its
await-bearing test expression into non-async CJS output. Statements
reached through the filter's body recursion can drop such declarations:
they are block-scoped there, and plain sloppy-mode functions were
already relocated. A dead module-level function still hoists and is
kept.
Comment thread src/js_parser/scan/scan_side_effects.rs Resolved
Comment thread src/js_parser/scan/scan_side_effects.rs Resolved
Comment thread src/js_parser/scan/scan_side_effects.rs Resolved
maybe_relocate_vars_to_top_level returns ok with no statement when every
decl was value-less and absorbed into the hoisted vars; s_for kept the
stale SLocal in the init slot, duplicating the declaration and keeping a
dead loop (and its await-bearing test) alive in non-ESM output. Assign
the relocation result whenever ok, matching s_for_in/s_for_of.
Comment thread src/js_parser/visit/visit_stmt.rs Resolved

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found this round; the three earlier leak classes (dead switch/with husks, block-scoped async/generator decls, and for (var x; await …; …) init retention) are all fixed and covered by regression tests. Deferring to a maintainer — this is a substantial parser-semantics change (module-scope await disambiguation, deferred TLA-implies-ESM detection, and DCE-filter behavior changes in should_keep_stmt_in_dead_control_flow / s_for var relocation that affect code beyond the await case) and should get a human look.

What was reviewed:

  • token_starts_await_expr disambiguation against identifier continuations (of/as/satisfies/using, tagged templates, TS !) — each has a locking test.
  • is_at_module_scope scope walk vs. TS namespace/declare-enum bodies and function/arrow arg scopes.
  • The should_keep_nested_stmt_in_dead_control_flow split and the s_for relocate.ok change for live-code impact — the latter also changes non-dead for (var x; …) output.
  • has_live_top_level_await set sites (E::Await visit, for await visit) vs. is_control_flow_dead and the post-visit clear/error in parse_entry.rs.
Extended reasoning...

Overview

This PR defers the CJS top-level-await rejection from parse time to after the visit pass so DCE can drop unreachable await before the error fires. It touches ten files: the await-prefix parse path (parse_prefix.rs), for await parsing (parse_stmt.rs), a new is_at_module_scope helper and tla_implies_esm gating on is_file_considered_to_have_esm_exports / has_es_module_syntax / implicit strict mode (p.rs), visit-pass liveness tracking (visit_expr.rs, visit_stmt.rs), the post-visit diagnostic (parse_entry.rs), DCE filter changes in should_keep_stmt_in_dead_control_flow including a new nested-body variant and new SSwitch/SWith arms (scan_side_effects.rs), a change to s_for var-init relocation, a new await_range field on S::ForOf (s.rs, react_compiler/codegen.rs), and a 640-line regression suite with 24 concurrent tests.

Security risks

None identified. This is parser/bundler output-shape logic with no auth, network, filesystem-path, or untrusted-size handling. The worst failure mode is emitting syntactically invalid CJS (which Node then rejects at load time) — three such cases were found in earlier review rounds and are now fixed with tests.

Level of scrutiny

High. The parser is production-critical and every JS/TS file flows through the changed pfx_t_identifier path. Several changes have blast radius beyond the await case: (1) the tla_implies_esm gating changes when a file is classified as ESM / when implicit strict mode is applied for non-TLA-capable formats; (2) should_keep_nested_stmt_in_dead_control_flow now drops block-scoped function declarations from dead bodies for all DCE, not just await-bearing ones; (3) the s_for relocation change (if relocate.ok { data.init = relocate.stmt }) alters live-code output — the author confirmed it also removes a redundant var duplication in live for (var y; …) loops. These are reasonable and match esbuild's intent, but a maintainer should confirm the DCE-filter and for-init changes are semantically what they want across the bundler.

Other factors

The PR went through 15 iterations. Earlier automated review rounds found three real correct-rejection → silent-invalid-output regressions (dead switch/with husks kept wholesale; async/generator fn decls in dead bodies keeping their enclosing statement; for (var x; await …; …) init not cleared after full relocation). Each was fixed at the layer that owns the invariant and each has a dedicated regression test. Two acknowledged follow-up limitations remain (parenthesized (await …) at module scope still errors at parse time due to speculative arrow-arg scopes; a same-line @await class decorator edge case) — both are non-regressions vs. main and documented in resolved threads. Test coverage is thorough (24 cases, all draining both pipes and asserting the quiet stream). CI on the latest build passed except for infrastructure flakes (Azure throttling, R2 ServiceUnavailable). All review threads are resolved.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@robobun

robobun commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator Author

Reopened as #47812 after the stale-bot closure: the bug still reproduces on bun 1.4.3 and the branch rebases cleanly onto current main. Same change; all review threads here were addressed before closure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bun doesn't dead-code eliminate the same way ESBuild does

2 participants