Repository navigation
test(spawn-maxbuf): widen 'yes is killed' wall-clock budget on debug, unquarantine - #36782
Conversation
…builds The two 'yes is killed' cases time from before spawn() to after exited, which includes the child process's startup. The child is bunExe() itself (for cross-platform `yes` via `bun exec`), and a debug+ASAN bun takes ~150-250ms to reach main(), so the 100ms ceiling is missed before the child has written a byte. Release and release-asan start in <15ms and pass. Widen the window to 1000ms under isDebug (release stays at 100ms). The byte-bound 'caps the buffer' tests already prove maxBuffer stops reading within one 64KB read of the limit, so this assertion is only the coarse 'kill wasn't deferred' check. Drop the whole-file [FLAKY] quarantine: the unmodified file already passes every release and release-asan CI lane (build 87834), and with this change `bun bd test` passes locally too.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
WalkthroughThe spawn max-buffer test now uses a longer termination window for debug builds and a shorter window for release builds. The test is also removed from the quarantine list. ChangesSpawn max-buffer test
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
There was a problem hiding this comment.
LGTM — test-only de-flake that branches the timing budget on isDebug and restores 16 quarantined cases to CI.
What was reviewed:
isDebugis exported fromharness.tsand the branch-per-build-type pattern matches existing usage; release keeps the original 100ms bound.- The property under test (kill isn't deferred by a full tick) is preserved; byte-level promptness is separately covered by the "caps the buffer" tests, so widening the coarse wall-clock check on debug loses no precision.
- Checked the other timing assertions in the file (
timeout killsat <200ms,timeout Infinityat <1500ms) — those are parent-side timers not gated on child startup writing bytes, and the PR'sbun bdrun confirms all 16 pass on debug.
Extended reasoning...
Overview
Two-file test-only change: (1) test/js/bun/spawn/spawn-maxbuf.test.ts widens the yes is killed wall-clock assertion from a hardcoded 100ms to isDebug ? 1000 : 100, replacing a // TODO with a comment explaining why; (2) test/expectations.txt drops the whole-file [ FLAKY ] quarantine that was hiding all 16 maxBuffer/timeout cases on every CI lane.
Security risks
None. No runtime code is touched; this only adjusts a test timing threshold and removes a quarantine entry.
Level of scrutiny
Low. This is the canonical de-flake pattern the repo's review guidance calls for: "When de-flaking, keep asserting the property the original assertion protected — branch per-platform rather than dropping precision." The release bound is unchanged at 100ms, and the debug-only widening is well-justified (the measured window includes a debug+ASAN child bun reaching main(), ~150-250ms on its own). The added comment names the mechanism and points to the byte-precise sibling tests that guard actual promptness.
Other factors
- The PR description shows a probe build (87834) passing cleanly on release and release-asan, five consecutive
bun bdruns at 128-157ms (well under 1000), and release at 2-11ms (well under 100), so the new bounds have headroom on both sides. - Unquarantining is a net coverage gain — the file had been entirely skipped on all lanes.
- I spot-checked the remaining timing assertions in the file for the same debug-startup hazard: the
timeout: 100/< 200cases fire a parent-side timer and send the signal regardless of child readiness (default SIGHUP disposition terminates during startup), and thebun bd16-pass verification confirms they hold. - No prior reviews or comments on the timeline.
|
Follow-up: #36792 fixes the remaining |
…wnSync timeout (#36792) Follow-up to #36782, which unquarantined `spawn-maxbuf.test.ts` and exposed a latent boundary flake in a different subtest. ## Failure ``` 240 | expect(timeEnd - timeStart).toBeGreaterThan(100); // make sure it actually waits error: expect(received).toBeGreaterThan(expected) Expected: > 100 Received: 100 ✗ timeout kills the process > Bun.spawnSync [101.64ms] ``` Seen on main [build 87859](https://buildkite.com/bun/bun/builds/87859) (alpine 3.23 x64, retry passed) and [build 87890](https://buildkite.com/bun/bun/builds/87890) (debian 13 aarch64, red on every retry). The assertion has been in the file since #18293; it was hidden by the whole-file `[ FLAKY ]` quarantine until #36782 removed it. ## Cause The `spawnSync` timeout deadline is computed as `CLOCK_MONOTONIC + 100ms` at nanosecond precision (`js_bun_spawn_bindings.rs`: `now.add_ms(timeout_ms)`), and the timeout fires when `CLOCK_MONOTONIC >= deadline` (`SpawnSyncEventLoop::tick_with_timeout`). So by the time `spawnSync` returns, at least 100ms of monotonic time has elapsed. The test measures with `Date.now()`, which truncates the wall clock to whole milliseconds. For a real elapsed interval of 100.x ms, `floor(end) - floor(start)` can be exactly 100 (e.g. start at T+0.1ms → `Date.now()=T`, end at T+100.2ms → `Date.now()=T+100`). The test duration `[101.64ms]` in the CI output confirms the wait itself was not short. Local 500-iteration probe on release: ``` {"min":100,"hit100":7,"under100":0,"iterations":500} ``` The delta lands on exactly 100 in ~1.4% of runs and never goes below 100, so `>=` is the tight bound. ## Fix Change `toBeGreaterThan(100)` to `toBeGreaterThanOrEqual(100)`. The lower bound is kept because it is the only thing in this test that would catch `timeout: 100` being misread as `timeout: 0` (the `exitedDueToTimeout`/`signalCode` checks would still pass if the kill fired immediately). Node's own `test-child-process-spawnsync-timeout.js` only asserts the upper bound and relies on `error.code === 'ETIMEDOUT'`; Bun's `exitedDueToTimeout` is equivalent, but the `>= 100` here is cheap extra coverage. The sibling `toBeGreaterThan(1000)` assertions in the `timeout Infinity` block measure a child `sleep 1` plus `bunExe()` startup, which is always comfortably above 1000ms, so they are left as-is. ## Verification ``` $ bun bd test test/js/bun/spawn/spawn-maxbuf.test.ts 16 pass 0 fail ``` Five consecutive full-file debug runs pass. Test-only change; no runtime behaviour is touched. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 0 · Platform-specific test-only change; deferring to CI. <!-- robobun:evidence:end -->
Empirically re-derived which `test/expectations.txt` entries are still needed by removing all 29 and running the full CI matrix ([build 87834](https://buildkite.com/bun/bun/builds/87834)). ## Result: 29 entries → 3 ### Kept (3): still fail on the named lane | entry | lane | observed failure (build 87834) | |---|---|---| | `test/bundler/native-plugin.test.ts` | WINDOWS | MSB8020: ClangCL build tools not found (agent image gap) | | `test/js/node/test/parallel/test-net-pingpong.js` | WINDOWS | named-pipe half-close: count 1000 !== 1001 | | `test/js/node/test/sequential/test-net-listen-shared-ports.js` | LINUX | SO_REUSEPORT shared-listener semantics; passes on macOS/Windows | ### Deleted (6): vendored Node tests that fail deterministically on every lane These can never pass as vendored; removing the files instead of re-quarantining. | file | reason | |---|---| | `test-stream-wrap.js`, `test-stream-wrap-drain.js`, `test-stream-wrap-encoding.js` | require `internal/js_stream_socket` which Bun does not implement | | `test-net-connect-keepalive.js`, `test-net-server-keepalive.js` | assert `_handle.setKeepAlive` receives seconds (libuv convention); Bun's `_handle` is Bun.Socket (ms). End-to-end TCP_KEEPIDLE coverage is in `test/js/bun/net/socket.test.ts` | | `test-set-http-max-http-headers.js` | spawns `test-http-max-http-headers.js` which is not vendored | ### Moved to `no-validate-exceptions.txt` (7): fail only via unchecked-exception assertions These now **run** on ASAN with `validateExceptionChecks` off, instead of being removed from the run entirely. | file | unchecked exception scope | |---|---| | `test/integration/next-pages/test/dev-server-ssr-100.test.ts` | `JSOrderedHashTable::getImpl` → `executeBoundCall` | | `test/integration/next-pages/test/dev-server.test.ts` | same | | `test/integration/next-pages/test/next-build.test.ts` | same | | `test/js/third_party/next-auth/next-auth.test.ts` | same | | `test/napi/napi.test.ts` | `Process_functionDlopen` (BunProcess.cpp:397) | | `test/cli/run/require-cache.test.ts` | `NapiClass::finishCreation` (NapiClass.cpp:120) | | `test/cli/inspect/inspect.test.ts` | `getOwnNonIndexPropertyNames` → `JSObjectInlines::get` (inspector Runtime.evaluate) | Also bumped `esm-fixture-leak-small.mjs` ASAN threshold 400→500 MB (build 87834 measured 407 MB; ASAN quarantine overhead) so `require-cache.test.ts` passes end to end on ASAN. ### Removed (13): now pass on their named lane | entry | was scoped to | now passes on | |---|---|---| | `test/js/node/test/parallel/test-repl-close.js` | WINDOWS-AARCH64 | windows 11 aarch64 | | `test/js/node/test/parallel/test-tls-connect-memleak.js` | LINUX-X64-MUSL | alpine 3.23 x64 + aarch64 | | `test/js/bun/spawn/spawn-maxbuf.test.ts` | (all) | every lane (also fixed for debug in #36782) | | `test/js/bun/spawn/spawn.test.ts` | ASAN | every lane (heap-use-after-free fixed in #36783) | | `test/js/sql/tls-sql.test.ts` | ASAN | debian 13 x64-asan | | `test/js/node/url/pathToFileURL.test.ts` | ASAN | debian 13 x64-asan | | `test/js/node/fs/abort-signal-leak-read-write-file.test.ts` | ASAN | debian 13 x64-asan | | `test/js/web/streams/streams-leak.test.ts` | ASAN | debian 13 x64-asan | | `test/js/node/test/parallel/test-net-server-listen-path.js` | WINDOWS | windows 2019 x64 + 11 aarch64 | | `test/js/node/test/parallel/test-net-pipe-connect-errors.js` | WINDOWS | fixed in #36786 | | `test/js/node/test/parallel/test-net-client-bind-twice.js` | WINDOWS | fixed in #36786 | | `test/js/node/test/parallel/test-net-server-reset.js` | WINDOWS | fixed in #36786 | | `test/js/bun/io/fetch/fetch-abort-slow-connect.test.ts` | DARWIN | darwin 26 aarch64 + 14 x64 | ### Caveats - `test-tls-connect-memleak.js` and `fetch-abort-slow-connect.test.ts` were `FLAKY` quarantines; both passed in probe build 87834 and confirmation builds 87845 / 87860 / 87868. <!-- robobun:evidence:begin --> --- **no test proof** · iteration 4 · docs-only change; test-proof not applicable <!-- robobun:evidence:end -->
test/js/bun/spawn/spawn-maxbuf.test.tshas been whole-file[ FLAKY ]intest/expectations.txt, hiding all 16 maxBuffer/timeout cases on every CI lane. Probed in build 87834 it passes every release and release-asan lane cleanly, so the quarantine is stale for CI. What it was hiding is a debug-only false positive: underbun bdthe twoyes is killedcases fail deterministically.Repro
Cause
Both tests assert
Date.now()from beforespawn()to afterexitedis under 100ms, with the comment "make sure it's not waiting a full tick". The child is[bunExe(), "exec", "yes"], so the measured window includes the childbunprocess reachingmain()beforeyescan write its first byte. On a debug+ASAN build that startup alone is ~150-250ms:so the 100ms budget is exhausted before maxBuffer has anything to react to. Release and release-asan start in <15ms and stay well inside the bound.
Fix
Widen the wall-clock window to
isDebug ? 1000 : 100, matching the pattern used elsewhere for timing budgets that include a debug-build spawn (e.g.fetch-tls-abortsignal-timeout.test.ts,buffer-indexof-worstcase.test.ts). Release keeps the original 100ms bound unchanged. Promptness of the kill itself is already asserted byte-precisely by the "caps the buffer while the child is still writing" tests (stdout.length <= maxBuffer + 64*1024against an 8MB firehose), so this assertion is only the coarse "not deferred by seconds" sanity check.Drop the whole-file quarantine line.
Verification
Five consecutive
bun bdruns:yes is killedat 128-157ms (< 1000). Release: 2-11ms (< 100).Test-only change; no runtime behaviour is touched.
no test proof · iteration 0 · test-only change (timing threshold + unquarantine), no src/ change to stash