Skip to content

bun test --reporter=junit: fill <failure> for a test that fails with a build or resolve error - #44434

Closed
robobun wants to merge 1 commit into
mainfrom
robobun/3eab222d/junit-build-error-text
Closed

robobun wants to merge 1 commit into
mainfrom
robobun/3eab222d/junit-build-error-text

Conversation

@robobun

@robobun robobun commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • A test can fail with a syntax error or a missing module. The JUnit report then has <failure type="Error" /> with no type, message or location.
  • bun test captures a failure through the hook on_print_error_zig_exception, which print_error_instance_body calls (src/jsc/VirtualMachine.rs:6769). BuildMessage and ResolveMessage print from their Msg in print_error_from_maybe_private_data (:6036) and return before that call.

Fix

  • print_error_from_maybe_private_data calls the hook for both classes. It passes the class name, the message text and one frame from the location of the message.
  • The call is under allow_side_effects, like the existing call. It also runs when the message was logged before.
  • Verified: the new test in test/js/junit-reporter/junit.test.js fails on main. Also ran inspect-error.test.js, resolve-error.test.ts, build-error.test.ts, reportError.test.ts.
  • Self-reviewed: 1 concern raised, 1 addressed.

Background

  • BuildMessage and ResolveMessage are what bun throws for a transpile error and a failed resolution. Each holds a bun_ast::Msg: text, file, line, column.
  • The hook is a function pointer on the VM. bun test sets it while it prints the error of a failed test, and TestFailure::record_cb fills <failure>. No other code sets it.
  • Considered: read name and message from the JS value in the reporter. That can run user code, and bun already has the text.

Downsides

  • Each printed BuildMessage or ResolveMessage executes 7 more instructions when no hook is installed (89 to 96 in print_error_from_maybe_private_data).
  • Release binary: +1,125 bytes in the two functions (nm -S).
  • <failure> has one frame, without the notes of the message. The inspector and GitHub annotations still get nothing for these classes.
Notes
  • This is the first PR of a stack. bun test --reporter=junit: record a failure that is not a finished test #44435 records a failure that is not a finished test in the JUnit report. A file that fails to load with a syntax error gets <failure type="BuildMessage" message="Unexpected ;"> there because of this PR.
  • Before: <failure type="Error" />. After: <failure type="BuildMessage" message="Unexpected ;">BuildMessage: Unexpected ; and at broken.js:1:11.
  • print_error_instance_body does three things for an error under allow_side_effects: it reports to the inspector, it calls this hook, and it prints a GitHub annotation. This PR adds only the hook call for the two classes. bun test: emit one anchored GitHub Actions annotation per failed test #41876 is open for the annotations.
  • The instruction count is the number of guest instructions that qemu-user executes inside the function (-one-insn-per-tb -d exec,nochain -dfilter <range>) for bun run of a file with const x = ;, on release builds of the merge base and of this change. It is 90 to 97 for a missing import. The whole process executes about 2.5 million, with a spread of 3% between runs. perf, valgrind and bloaty are not installed on the machine I used.
  • The bytes: print_error_from_maybe_private_data 1,520 to 1,668, plus 977 for the cold helper that builds the exception.
  • ZigException::with_stack holds the field initialiser that Holder::zig_exception had. The hook call does not need a Holder.
  • The self-review concern: the hook is one of three observers. The Downsides bullet and the note above state that limit.

…a build or resolve error

BuildMessage and ResolveMessage print from their Msg, so
print_error_instance_body never ran for them and the hook that captures
an error for the JUnit report never fired. A test that rejected with a
syntax error or a missing module got <failure type="Error" /> with no
message.

print_error_from_maybe_private_data now calls the hook, when one is
installed, with the class name, the message text and one frame made from
the message's location.
@robobun

robobun commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. First PR of a stack: #44434 (this one), #44435, #44436 (draft, optional).

How I reproduced it: broken.js is const x = ;. A test does await import("./broken.js"). Run bun test --reporter=junit --reporter-outfile=junit.xml.

  • main: <failure type="Error" />
  • this PR: <failure type="BuildMessage" message="Unexpected ;">

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 7d677973-8130-4e29-8f19-b72989572b0a

📥 Commits

Reviewing files that changed from the base of the PR and between bc7a813 and b3ede46.

📒 Files selected for processing (3)
  • src/jsc/VirtualMachine.rs
  • src/jsc/ZigException.rs
  • test/js/junit-reporter/junit.test.js

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.


Walkthrough

When side effects are allowed, VirtualMachine reports build and resolve messages to the configured exception hook as ZigException instances. The change adds exception initialization with an existing stack and a JUnit test for dynamic-import build and resolution failures.

Changes

Message Exception Reporting

Layer / File(s) Summary
Initialize exceptions with an existing stack
src/jsc/ZigException.rs
ZigException::with_stack initializes exception defaults with a supplied stack. Holder::zig_exception uses this helper.
Report build and resolve messages
src/jsc/VirtualMachine.rs, test/js/junit-reporter/junit.test.js
VirtualMachine reports build and resolve messages to the configured exception hook when side effects are allowed. The test checks failure types, messages, source location, and process exit code.

Suggested reviewers: jarred-sumner

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to b3ede

The change appears mergeable after normal checks. The new JUnit test has not been executed in this review.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly describes the main change: populating JUnit details for build and resolve errors.
Description check ✅ Passed The description explains the problem, implementation, verification steps, scope, and trade-offs. It does not use the exact template headings, but it provides the required information and is substantia…
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline findings, I also checked the lifetime of the synthesized ZigException: name/message/source_url are borrowed from the Msg and a stack array, but the only hook consumer (TestFailure::record in src/runtime/cli/test_command.rs) copies name, message and each frame into owned Vecs synchronously, so nothing outlives the call. The with_stack refactor in src/jsc/ZigException.rs is field-for-field identical to the initializer it replaced.

Extended reasoning...

The change adds a hook call for BuildMessage/ResolveMessage in src/jsc/VirtualMachine.rs, extracts a ZigException constructor in src/jsc/ZigException.rs, and adds one junit reporter test; it touches no security-sensitive surface. Two inline findings (missing JS frames for synchronous throws, and the test discarding stdout/stderr) were already posted, so a human look is signaled by those; this note records only what else was examined and ruled out.

Comment thread src/jsc/VirtualMachine.rs Resolved
Comment thread test/js/junit-reporter/junit.test.js Resolved
@robobun

robobun commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: one PR in place of the stack. This change is the first commit of #44435, which now targets main.

@robobun robobun closed this Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants