Skip to content

test: pin iat in the jsonwebtoken async vs sync sign comparison - #41304

Closed
robobun wants to merge 4 commits into
mainfrom
robobun/74605bbe/jsonwebtoken-async-sign-iat
Closed

robobun wants to merge 4 commits into
mainfrom
robobun/74605bbe/jsonwebtoken-async-sign-iat

Conversation

@robobun

@robobun robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • test/js/third_party/jsonwebtoken/async_sign.test.js fails on main with this test timed out after 90000ms, before its done callback was called in "should return the same result as singing synchronously". Seen in builds 108836, 109097, 109276, 109807 and 109861, on five different lanes.
  • The test signs { foo: "bar" } once asynchronously and once synchronously and compares the tokens. Each jwt.sign() reads Date.now() for the iat claim (jsonwebtoken/sign.js:185). When a second boundary falls between the two calls, the tokens differ and expect() throws.
  • The throw is lost. The callback runs inside SignStream.prototype.sign (jws/lib/sign-stream.js), whose try/catch emits 'error'. jsonwebtoken attached the same once()-wrapped callback as the error listener, and it already ran. So done() never runs and the test waits out its ceiling.

Fix

  • Freeze the clock with setSystemTime(60000) from bun:test in beforeEach and reset it with setSystemTime() in afterEach. It only fixes Date.now() and new Date(), so process.nextTick and the jws async path run as before. The payload stays the upstream { foo: "bar" }.
  • Wrap the assertions in try/catch and pass a failure to done(err). A future mismatch reports the diff at once instead of a silent 90 s timeout.
  • Verified: with Date.now patched to cross a second boundary, the old test reproduces the exact timeout with the release binary. With setSystemTime(60000), Date.now() reads 60000 before and after a nextTick, and the signed iat is 60. A forced expect failure now reports Expected/Received in 3 ms. bun bd test test/js/third_party/jsonwebtoken/async_sign.test.js passes, as does all of test/js/third_party/jsonwebtoken/.

Background

  • The window between the two Date.now() reads is about 1.5 ms on a release build and about 107 ms on a debug build (most of it is the first util.format call). That gives roughly a 0.15% to 10% chance per run, depending on the build and the load on the runner.
  • No bun code is at fault. The test itself is racy and has been since it was added in 2024. The recent sightings are this race, not a stream or nextTick regression.
Notes

The first version of this PR pinned iat: 1700000000 in the payload instead. Review asked for fake timers like the upstream tests, then for the bun:test ones. The later commits switch to setSystemTime and restore the upstream payload.

Repro used to confirm the mechanism (release bun, USE_SYSTEM_BUN=1):

const realNow = Date.now; let calls = 0;
Date.now = () => { calls++; const b = Math.floor(realNow() / 1000) * 1000; return calls === 1 ? b + 999 : b + 1000; };

With the original test body this prints two tokens that differ only in iat and then times out.

Window measurement (time between the two Date.now() calls, first test in a fresh process): debug build 106 to 107 ms over 3 runs, release 1.56 to 1.57 ms. Of the debug cost, util.format first call was 149 ms in a separate probe, process.nextTick 32 ms, first HMAC 6 ms.

CI on the first revision (build 109876): one red lane, pnpm-migration.test.ts on debian 13 x64-asan, where Verdaccio exited with code 2 before the test ran. Reported separately. Two other lanes passed on retry.


[auto-merge] gate passed · iteration 2 · 1 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/third_party/jsonwebtoken/async_sign.test.js'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test test/js/third_party/jsonwebtoken/async_sign.test.js
bun test v1.4.1 (e0a2b82fd)

test/js/third_party/jsonwebtoken/async_sign.test.js:
(pass) signing a token asynchronously > when signing a token > should return the same result as singing synchronously [187.25ms]
(pass) signing a token asynchronously > when signing a token > should work with empty options [10.42ms]
(pass) signing a token asynchronously > when signing a token > should work without options object at all [9.25ms]
(pass) signing a token asynchronously > when signing a token > should work with none algorithm where secret is set [11.10ms]
(skip) signing a token asynchronously > when signing a token > should work with none algorithm where secret is falsy
(pass) signing a token asynchronously > when signing a token > should return error when secret is not a cert for RS256 [4.75ms]
(pass) signing a token asynchronously > when signing a token > should not work for RS algorithms when modulus length is less than 2048 when allowInsecureKeySizes is false or not set [52.90ms]
(pass) signing a token asynchronously > when signing a token > should work for RS algorithms when modulus length is less than 2048 when allowInsecureKeySizes is true [49.62ms]
(pass) signing a token asynchronously > when signing a token > should return error when secret is not a cert for PS256 [4.90ms]
(pass) signing a token asynchronously > when signing a token > should return error on wrong arguments [9.48ms]
(pass) signing a token asynchronously > when signing a token > should return error on wrong arguments (2) [5.43ms]
(pass) signing a token asynchronously > when signing a token > should not stringify the payload [20.13ms]
(pass) signing a token asynchronously > when signing a token > when mutatePayload is not set > should not apply claims to the original payload object (mutatePayload defaults to false) [17.52ms]
(pass) signing a token asynchronousl
... (truncated)
Exit: 0
diff hotspot
.../js/third_party/jsonwebtoken/async_sign.test.js | 27 ++++++++++++++++++----
 1 file changed, 22 insertions(+), 5 deletions(-)

gate history · 3 passed · 0 rejected · iteration 2

evidence per changed file
file                                                 reads  edits  tests
test/js/third_party/jsonwebtoken/async_sign.test.js      1      3      7

The test signs the same payload once asynchronously and once
synchronously and compares the tokens. Each sign() reads Date.now()
for iat. When a second boundary falls between the two calls the
tokens differ and expect() throws inside the jws callback. jws
catches the throw and emits 'error', but jsonwebtoken's once()
wrapped callback already ran, so the error is lost and the test
times out after 90s with no message.

Pin iat in the payload so both tokens are deterministic. Wrap the
assertions in try/catch so a failure reaches done(err).
@robobun

robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: closed. #45873 landed a fix for this test on main, so this PR is not needed.

Reproduction: patch Date.now so that a second boundary falls between the async and the sync sign. With the release binary, the test from before #45873 then times out, as in build 109861. The file on main passes under the same patch.

@github-actions github-actions Bot added the claude label Sep 3, 2026
@robobun

robobun commented Sep 3, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 5:27 PM PT - Sep 3rd, 2026

❌ @robobun, your commit bc61e7b has 2 failures in Build #109894 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 41304

That installs a local version of the PR into your bun-41304 executable, so you can run:

bun-41304 --bun

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: f5336a86-b5a4-4e63-892c-4da4bff85251

📥 Commits

Reviewing files that changed from the base of the PR and between b958f3c and 86ba1c3.

📒 Files selected for processing (1)
  • test/js/third_party/jsonwebtoken/async_sign.test.js

Included review availability: Your plan provides up to 10 included reviews per hour; 4 remain after this review.


Walkthrough

The async JWT signing test freezes and restores system time so asynchronous and synchronous tokens receive matching iat claims. The test catches signing and assertion failures and forwards them to done().

Changes

JWT signing test

Layer / File(s) Summary
Deterministic signing and callback validation
test/js/third_party/jsonwebtoken/async_sign.test.js
The test controls system time during signing, restores it after each case, and reports synchronous signing or assertion failures through done().

Merge Risk: ⚪ Minimal · up to bc61e

The JWT signing test now uses a stable timestamp and reports callback failures directly, removing the intermittent timeout without introducing an identified merge risk.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the test change: pinning the iat claim in the jsonwebtoken asynchronous-versus-synchronous signing comparison.
Description check ✅ Passed The description explains the race, the failure mode, the implementation, and verification results. It does not use the exact template headings, but it provides the required information and is sufficie…
Full details: Description check

Explanation

The description explains the race, the failure mode, the implementation, and verification results. It does not use the exact template headings, but it provides the required information and is sufficiently complete.


Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Review noted, nothing to change. Buildkite build 109876 is in progress. The claude-find-issues action failed inside the action itself before it ran anything, which is not related to this diff.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — a clean de-flake that keeps asserting the original property (async and sync sign produce identical tokens) while removing the wall-clock race.

What was reviewed:

  • Pinning iat still exercises the same HS256 sign path; jsonwebtoken keeps a caller-provided iat, so both tokens are now deterministic.
  • Sharing one payload object between the two sign() calls is safe — mutatePayload defaults to false (the sibling test at L123-135 asserts exactly this), so the async call cannot alter it before the sync call.
  • The try/catch → done(e) wrapper matches the "wire failures to reject, never throw inside event callbacks" convention and turns the silent 90s timeout into an immediate diff.
Extended reasoning...

Overview

This PR touches a single file, test/js/third_party/jsonwebtoken/async_sign.test.js, and only one test case within it. It pins the iat claim in the JWT payload to a fixed timestamp so the async and sync jwt.sign() calls produce byte-identical tokens regardless of when a wall-clock second boundary falls, and it wraps the callback's assertions in try/catch → done(e) so a failed expect() surfaces as a test failure instead of being swallowed by jws's internal try/catch and hanging until the 90s timeout.

Security risks

None. This is a test-only change to a vendored third-party compatibility test. No production code, no crypto logic, no user-facing surface is touched. The pinned timestamp is an arbitrary constant used only to make two HMAC inputs identical.

Level of scrutiny

Low. The diff is ~15 lines in a single test case. The fix addresses a well-understood race (two Date.now() reads straddling a second boundary) and the PR description documents a reproduction that confirms both the mechanism and the fix. The de-flake preserves the original assertion's intent — it still compares the async token against the sync token for equality, just with deterministic inputs — which satisfies the REVIEW.md rule that de-flakes must "keep asserting the property the original assertion protected."

Other factors

I checked whether reusing the same payload object across both jwt.sign() calls could be a problem (if the async call mutated it before the sync call ran). It isn't: mutatePayload defaults to false and jsonwebtoken shallow-copies the payload before adding claims — the same file's test at L123-135 asserts this contract directly. The two added comments are concise and explain genuinely non-obvious behavior (the iat race and the jws error-swallowing path), passing the CLAUDE.md comment-hygiene bar. No CODEOWNERS entry covers this path, there are no prior reviews or outstanding objections on the PR, and the bug hunt exited on dry_streak with no findings.

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks. The shared payload object is safe for the reason you give: mutatePayload defaults to false, so the async sign works on a copy. No change needed. Build 109876 is still running.

@dylan-conway

Copy link
Copy Markdown
Member

pretty sure upstream tests solve this with fake timers. can we do the same?

The sibling claim tests in this directory already use
sinon.useFakeTimers({ now: 60000 }). Do the same here so the async
and the sync sign read the same Date.now() and the payload stays
the upstream { foo: "bar" }.
@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Done in b35dd70. The test now uses sinon.useFakeTimers({ now: 60000 }) in beforeEach and uninstalls it in afterEach, the same pattern as claim-iat and the other claim tests in this directory. The payload is back to the upstream { foo: "bar" }. sinon leaves process.nextTick alone by default, so the jws async path still runs. Checked that Date.now() reads 60000 on both sides of the nextTick and the signed iat is 60. The try/catch around the assertions stays, since jws swallows a throw from that callback.

@dylan-conway

Copy link
Copy Markdown
Member

we can't use bun:test fake timers?

@robobun

robobun commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

Yes. 86ba1c3 uses setSystemTime(60000) from bun:test in beforeEach and setSystemTime() in afterEach, and drops the sinon import. setSystemTime only fixes Date.now() and new Date(), so process.nextTick and the jws async path run as before. Checked that Date.now() reads 60000 on both sides of a nextTick, the signed iat is 60, and the clock is real again after the reset. Passes on the debug and the release build.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the setSystemTime approach is the right fix and the done(e) forwarding turns the silent timeout into an actual failure report.

What was reviewed:

  • Confirmed setSystemTime only touches Date, so the jws nextTick-driven async path still runs and afterEach restores the real clock even on failure.
  • Checked all sibling tests in the when signing a token block — none depend on wall-clock time, so the frozen epoch of 60 s is harmless to them.
  • Traced the swallowed-throw claim: jws SignStream.sign wraps the callback in try/catch and re-emits as error, and jsonwebtoken's listener is once()-wrapped, so try { ... } catch (e) { done(e) } is necessary here.
Extended reasoning...

Overview

This PR touches a single file, test/js/third_party/jsonwebtoken/async_sign.test.js, to deflake a comparison between an async-signed and sync-signed JWT. The root cause is well-established: each jwt.sign() reads Date.now() independently for the iat claim, so a second boundary between the two calls yields different tokens, and jws's internal try/catch swallows the failed assertion so the test times out at 90 s instead of reporting the diff. The fix freezes the clock with setSystemTime(60000) in beforeEach, restores it in afterEach, and forwards assertion failures to done(e) so future mismatches surface immediately.

Security risks

None. This is a test-only change to a third-party compatibility test. No runtime, native, or user-facing code is touched. setSystemTime is scoped to the test process and reset in afterEach.

Level of scrutiny

Low. The diff is 22 lines added / 5 removed in one test file, with no CODEOWNER coverage. The mechanism is documented in the PR body with a working repro, and the approach matches REVIEW.md guidance verbatim: freeze the clock rather than sleep or retry, and wire every failure path to the done callback rather than throwing inside an event callback. I read the full file to confirm the beforeEach/afterEach hooks apply to the whole when signing a token block and verified none of the sibling tests (empty options, RS/PS error paths, mutatePayload, falsy-secret loop) depend on real time — they only assert on error presence, token shape, or property mutation, all unaffected by an iat of 60.

Other factors

The PR went through two revisions after maintainer comments (pinned iat → sinon fake timers → setSystemTime from bun:test), and the current diff reflects the final requested approach. The two comments added survive the "would the next reader spend tool calls discovering this" test — the jws swallowing behavior in particular is genuinely non-obvious. Bug hunt exited on dry_streak with no findings.

@robobun

robobun commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the review of 86ba1c3. It needs no change.

@robobun

robobun commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Closing. #45873 landed a fix for this test.

On main, both jwt.sign calls now use the payload { foo: "bar", iat: 1437018582 }, so the two tokens no longer depend on the clock. I checked this with a Date.now that advances by one second on every call. The file from before #45873 times out. The file on main passes.

One difference remains. This PR also wrapped the assertions in try/catch so that a failed expect() reaches done(e). Main does not have that, so a future mismatch in this test still shows as a timeout. It is small, and it can be a separate change if a maintainer wants it.

@robobun robobun closed this Oct 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants