Skip to content

bun test: flush .snap file before afterAll runs - #36679

Closed
robobun wants to merge 6 commits into
mainfrom
claude/farm/e250d8e7/snapshot-flush-before-afterall
Closed

robobun wants to merge 6 commits into
mainfrom
claude/farm/e250d8e7/snapshot-flush-before-afterall

Conversation

@robobun

@robobun robobun commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #12114.

Repro

const fs = require("node:fs");
test("make a dummy snapshot", () => {
  expect({ dummy: "snapshot" }).toMatchSnapshot();
});
afterAll(() => {
  console.log("SNAPSHOT-LEN:", fs.readFileSync("./__snapshots__/dummy.test.js.snap").toString().length);
});
$ bun test --update-snapshots dummy.test.js
(pass) make a dummy snapshot
SNAPSHOT-LEN: 0          # file populated only after process exit

Cause

Snapshots::get_snapshot_file opened the .snap file with O_TRUNC under --update-snapshots, and write_snapshot_file only ran when the next test file opened its own snapshot or at process exit. Between those two points (which includes every afterAll in the file) the on-disk file was zero bytes. The first-create path had the same shape: O_CREAT made an empty file that wasn't written until later.

Fix

src/runtime/test_runner/snapshot.rs

  • Drop O_TRUNC from the open flags; the old bytes stay on disk until we overwrite them.
  • Track needs_write (set when get_or_put appends a new entry, or under -u when the buffer is rebuilt from a clean header) and add flush_current_file, which does seek(0) + write_all + ftruncate(len) and leaves the fd open so a later toMatchSnapshot can keep appending.
  • write_snapshot_file now calls flush_current_file and then closes/clears. A passing run with no added snapshots no longer rewrites an unchanged file.

src/runtime/test_runner/Execution.rs

  • step_group flushes the snapshot file whenever a hook-only group (beforeAll/afterAll, any describe depth) is about to start. needs_write makes the beforeAll case and repeated flushes no-ops. The flush is best-effort here; a persistent write error still surfaces via the end-of-run write_snapshot_file()?.

Verification

$ bun test --update-snapshots dummy.test.js
(pass) make a dummy snapshot
SNAPSHOT-LEN: 125

New tests in test/js/bun/test/snapshot-tests/new-snapshot.test.ts: the #12114 repro on first create, -u with a shorter value than the existing snapshot (exercises the ftruncate path and checks the exact body seen by afterAll against the final file), a two-file -u run where each file's afterAll reads its own .snap, and a nested describe whose inner and outer afterAll both read the flushed file. All fail on main and pass with this change. bun_test.test.ts, ci-restrictions.test.ts, test-filter-lifecycle-snapshot.test.ts, and the rest of snapshot-tests/ pass unchanged.

#34042 also touches this area as part of obsolete-snapshot detection; this PR is the narrow fix for the afterAll visibility bug and should rebase cleanly onto it in either direction.


[review] gate passed · iteration 0 · 4 files touched

fails on main (without fix)
ASAN without fix: 3 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/test/snapshot-tests/new-snapshot.test.ts
bun test v1.4.0 (13576e4ee)

test/js/bun/test/snapshot-tests/new-snapshot.test.ts:
(pass) it will create a snapshot file and directory if they don't exist [1155.92ms]
67 |   {
68 |     const { stderr, exitCode, seen } = await run(longValue);
69 |     expect(stderr).toContain("1 pass");
70 |     const onDisk = fs.readFileSync(snapPath, "utf8");
71 |     expect(onDisk).toContain(longValue);
72 |     expect(seen).toEqual({ len: onDisk.length, body: onDisk });
                      ^
error: expect(received).toEqual(expected)

  {
-   "body": 
- "// Bun Snapshot v1, https://bun.sh/docs/test/snapshots
- 
- exports[`make a snapshot 1`] = `"long-initial-value-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"`;
- "
- ,
-   "len": 172,
+   "body": "",
+   "len": 0,
  }

- Expected  - 7
+ Received  + 2

      at <anonymous> (/workspace/bun/test/js/bun/test/snapshot-tests/new-snapshot.test.ts:72:18)
(fail) afterAll sees the written .snap file under --update-snapshots [1377.10ms]
142 
... (truncated)

release without fix: 3 FAILED
bun test v1.4.0-canary.1 (1498d7b77)

test/js/bun/test/snapshot-tests/new-snapshot.test.ts:
(pass) it will create a snapshot file and directory if they don't exist [366.57ms]
67 |   {
68 |     const { stderr, exitCode, seen } = await run(longValue);
69 |     expect(stderr).toContain("1 pass");
70 |     const onDisk = fs.readFileSync(snapPath, "utf8");
71 |     expect(onDisk).toContain(longValue);
72 |     expect(seen).toEqual({ len: onDisk.length, body: onDisk });
                      ^
error: expect(received).toEqual(expected)

  {
-   "body": 
- "// Bun Snapshot v1, https://bun.sh/docs/test/snapshots
- 
- exports[`make a snapshot 1`] = `"long-initial-value-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"`;
- "
- ,
-   "len": 172,
+   "body": "",
+   "len": 0,
  }

- Expected  - 7
+ Received  + 2

      at <anonymous> (/workspace/bun/test/js/bun/test/snapshot-tests/new-snapshot.test.ts:72:18)
(fail) afterAll sees the written .snap file under --update-snapshots [283.90ms]
110 |     env: { ...bunEnv, CI: "false" },
111 |     cwd: String(dir),
112 |     stdout: "pipe",
113 |     stderr: "pipe",
114 |   });
115 |   const [stdout, stderr, exitCode] = await
... (truncated)
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/bun/test/snapshot-tests/new-snapshot.test.ts
bun test v1.4.0 (13576e4ee)

test/js/bun/test/snapshot-tests/new-snapshot.test.ts:
(pass) it will create a snapshot file and directory if they don't exist [1385.09ms]
(pass) describe-level afterAll sees the written .snap file [1550.67ms]
(pass) afterAll sees the written .snap file across multiple test files [1940.46ms]
(pass) afterAll sees the written .snap file under --update-snapshots [2977.51ms]

 4 pass
 0 fail
 22 expect() calls
Ran 4 tests across 1 file. [10.67s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     13576e4eea
  features     baseline

22 deps, 108 codegen, 1171 objects in 4046ms

ninja: Entering directory `/workspace/bun/build/release'
[1/1234] gen ProcessBindingConstants.lut.h
Generating /workspace/bun/build/release/codegen/ProcessBindingConstants.lut.h from /workspace/bun/src/jsc/bindings/ProcessBindingConstants.cpp
[2/1234] fetch libjpeg-turbo
[libjpeg-turbo] up to date
[3/1234] fetch picohttpparser
[picohttpparser] up to date
[4/1234] gen JSBuffer.lut.h
Generating /workspace/bun/build/release/codegen/JSBuffer.lut.h from /workspace/bun/src/jsc/bindings/JSBuffer.cpp
[5/1234] gen ProcessBindingBuffer.lut.h
Generating /workspace/bun/build/release/codegen/ProcessBindingBuffer.lut.h from /workspace/bun/src/jsc/bindings/ProcessBindingBuffer.cpp
[6/1234] fetch zlib
[zlib] up to date
[7/1234] gen .bind.ts → GeneratedBindings.cpp
[8/1234] subst deps/libjpeg-turbo/jconfigint.h
[9/1234] fetch tinycc
[tinycc] up to date
[10/1234] gen ErrorCode+*.h
[11/1234] fetch nodejs (prebuilt)
[nodejs
... (truncated)
diff hotspot
src/runtime/cli/test_command.rs                    |   1 +
 src/runtime/test_runner/Execution.rs               |   7 ++
 src/runtime/test_runner/snapshot.rs                |  31 ++++--
 .../bun/test/snapshot-tests/new-snapshot.test.ts   | 122 ++++++++++++++++++++-
 4 files changed, 153 insertions(+), 8 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                                  reads  edits  tests
src/runtime/cli/test_command.rs                           3      1      0
src/runtime/test_runner/Execution.rs                      8     12      0
src/runtime/test_runner/snapshot.rs                       5      7      0
test/js/bun/test/snapshot-tests/new-snapshot.test.ts      4      4      0

The snapshot file was opened with O_TRUNC under --update-snapshots and only
written when the next test file opened its own snapshot (or at process exit),
so an afterAll() that read the .snap file saw zero bytes.

Drop the eager truncate, track a needs_write dirty bit, and flush (seek + write
+ ftruncate) when execution reaches the root describe's afterAll groups. The
end-of-run write_snapshot_file path now reuses the same flush, and a passing
run with no new snapshots no longer rewrites an unchanged file.

Fixes #12114
@coderabbitai

coderabbitai Bot commented Aug 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

The test runner now tracks pending snapshot writes and flushes them when execution reaches the current scope’s afterAll groups. Order generation supplies this group boundary. Regression tests cover initial runs, snapshot updates, and multiple test files.

Snapshot flush coordination

Layer / File(s) Summary
Track and flush pending snapshot writes
src/runtime/test_runner/snapshot.rs, src/runtime/cli/test_command.rs
Snapshots track pending changes, flush buffered content explicitly, and initialize needs_write to false.
Configure the afterAll flush boundary
src/runtime/test_runner/Order.rs, src/runtime/test_runner/bun_test.rs, src/runtime/test_runner/Execution.rs
Order generation returns the afterAll group index. Execution stores the index and flushes the current snapshot file when that group is reached.
Validate snapshot visibility during afterAll
test/js/bun/test/snapshot-tests/new-snapshot.test.ts
Tests verify snapshot contents during initial runs, update runs, and multi-file execution.

Suggested reviewers: jarred-sumner, cirospaciari

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes directly fix issue #12114 by flushing generated snapshots before afterAll runs and adding regression tests.
Out of Scope Changes check ✅ Passed The code and tests remain focused on snapshot visibility and flushing behavior described by issue #12114.
Title check ✅ Passed The title clearly summarizes the main change: flushing snapshot files before afterAll hooks run.
Description check ✅ Passed The description explains the cause, fix, verification steps, regression tests, and linked issue in sufficient detail.

Comment @coderabbitai help to get the list of available commands.

Comment thread src/runtime/test_runner/Execution.rs Resolved
Comment thread src/runtime/test_runner/Order.rs Resolved
Comment thread src/runtime/test_runner/snapshot.rs Resolved
Comment thread src/runtime/test_runner/snapshot.rs Resolved
Comment thread src/runtime/test_runner/snapshot.rs Resolved
@robobun

robobun commented Aug 1, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:05 AM PT - Aug 1st, 2026

❌ @robobun, your commit 13576e4 has 2 failures in Build #87037 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 36679

That installs a local version of the PR into your bun-36679 executable, so you can run:

bun-36679 --bun

Comment thread src/runtime/test_runner/Execution.rs Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/runtime/test_runner/Execution.rs`:
- Around line 874-879: Update the snapshot flush branch in step_group to
propagate the Result from runner.snapshots.flush_current_file() instead of
discarding it. Only set snapshot_flush_group to usize::MAX after the flush
succeeds; return the flush error immediately when it fails.

In `@test/js/bun/test/snapshot-tests/new-snapshot.test.ts`:
- Around line 75-85: The snapshot test currently uses identical output and
substring assertions, so it does not exercise truncation or detect stale
trailing bytes. Update the fixture and first run to create a longer snapshot,
then change it before the --update-snapshots run to produce a shorter one; in
the afterAll output assertions, require the exact framed length or full body,
and verify the final file exactly matches the shorter snapshot with no suffix
from the original.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a3083844-fb8c-446b-a681-65e6f7ab0f38

📥 Commits

Reviewing files that changed from the base of the PR and between a7838c5 and 0a2069c.

📒 Files selected for processing (6)
  • src/runtime/cli/test_command.rs
  • src/runtime/test_runner/Execution.rs
  • src/runtime/test_runner/Order.rs
  • src/runtime/test_runner/bun_test.rs
  • src/runtime/test_runner/snapshot.rs
  • test/js/bun/test/snapshot-tests/new-snapshot.test.ts

Comment thread src/runtime/test_runner/Execution.rs Resolved
Comment thread src/runtime/test_runner/bun_test.rs Resolved
Comment thread src/runtime/test_runner/Execution.rs Resolved
… too

Replaces the root-afterAll group index tracking with a simpler check in
step_group: when a beforeAll/afterAll group (all sequences have test_entry ==
None) is about to start, flush pending snapshot writes. needs_write makes the
beforeAll case a no-op. Order.rs and bun_test.rs are back to their original
shape.
@robobun robobun changed the title bun test: flush .snap file before file-level afterAll runs bun test: flush .snap file before afterAll runs Aug 1, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both prior concerns are addressed and the reworked approach is simpler than the original.

What was reviewed:

  • flush_current_file's seek(0)+write_all+ftruncate against the O_TRUNC removal; needs_write only clears on full success so the end-of-run write_snapshot_file()? still surfaces I/O errors.
  • .all() on an empty/beforeAll group is a no-op via needs_write; Jest::runner() here matches the existing reset_sequence pattern in the same file.
  • Single Snapshots { .. } construction site initializes needs_write; parallel workers spawn subprocesses that go through the same step_group path.
  • get_end_pos is fstat-based (no cursor move), so the non--u read-then-rewrite path is unchanged aside from now skipping the write when nothing was added.
Extended reasoning...

Overview

Fixes #12114: .snap files are now flushed to disk before beforeAll/afterAll hooks run, at any describe depth. Changes: snapshot.rs gains a needs_write flag and flush_current_file() (seek(0) + write_all + ftruncate, fd stays open), write_snapshot_file delegates to it, and O_TRUNC is dropped from the open flags so the on-disk file is never emptied mid-run. Execution.rs::step_group calls flush_current_file() before starting any group whose sequences all have test_entry == None. test_command.rs initializes the new field. Three new subprocess tests cover first-create, -u with a long→short value (exercising ftruncate), multi-file, and nested describes.

Prior review

My previous run flagged (1) the original root-scope-only snapshot_flush_group missing describe-level afterAll, and (2) the discarded I/O result needing a comment. Commit 13576e4 addressed both: the Order.rs/bun_test.rs index-tracking was dropped entirely in favor of a hook-only-group check directly in step_group (idempotent via needs_write), the best-effort comment was added, and a nested-describe test was added. CodeRabbit's two comments (error propagation, ftruncate test coverage) were also resolved.

Security risks

None. Local file I/O to __snapshots__/*.snap only; no user-controlled paths, network, or auth.

Level of scrutiny

Moderate — touches the test runner's step_group loop and snapshot-file write path. I traced: needs_write is only set in get_or_put and the -u branch of get_snapshot_file, and only cleared as the last line of flush_current_file after all three I/O ops succeed, so a mid-run flush failure leaves the retry armed for the end-of-run write_snapshot_file()?. bun_sys::ftruncate is cross-platform (libc / NtSetInformationFile) and already used by write_inline_snapshots in the same file. get_end_pos uses get_file_size (not lseek), so cursor semantics on the non--u read path are unchanged. The Jest::runner() call in step_group matches the existing pattern at reset_sequence in the same file. The parallel runner forwards --update-snapshots to worker subprocesses that run the same step_group, so no separate fix is needed there.

Other factors

The one intentional behavioral side effect — a passing run with no new snapshots no longer rewrites an unchanged .snap — is called out in the PR description. Tests follow harness conventions (tempDir, bunEnv spread, concurrent subprocess drain, exitCode asserted last) and the ftruncate case asserts exact-body equality between what afterAll observed and the final on-disk file. PR description reports the surrounding suites (bun_test.test.ts, snapshot-tests/, etc.) pass unchanged.

@robobun

robobun commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator Author

CI on 13576e4: the diff's own test file (test/js/bun/test/snapshot-tests/new-snapshot.test.ts) and the surrounding snapshot/test-runner suites are green on every lane.

The red lanes are unrelated to this change:

  • test/cli/install/bun-upgrade.test.ts on windows-aarch64: "Canary builds are not available for this platform yet"; also red on main builds 87044 and 86911.
  • test/regression/issue/36577.test.ts on windows-x64: pre-existing install cache ENOENT.
  • The remaining eight (complex-workspace, sourcetextmodule-leak, request-clone-leak, inspect-error-leak, child_process, 23865, 22978, tty-reopen-after-stdin-eof) all passed when rerun alone.

Ready for review.

@github-actions

Copy link
Copy Markdown
Contributor

Closing this PR because it has been inactive for more than 90 days.

@github-actions github-actions Bot closed this Oct 31, 2026
@robobun

robobun commented Oct 31, 2026

Copy link
Copy Markdown
Collaborator Author

This PR stays closed. main already has the part of it that the repro in #12114 needs.

The other part of this PR wrote the new contents before the afterAll hooks. Jest does not do that, and main does not do that. If a maintainer wants that behavior, it needs a new PR on the current write path. #38974 is open and writes each .snap when its test file ends.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

On --update-snapshots, snapshot file remains empty in afterAll()

2 participants