Skip to content

commitment: build the ModeParallel prefix trie off the execution thread - #24252

Open
awskii wants to merge 19 commits into
mainfrom
awskii/commitment-background-trie-build
Open

awskii wants to merge 19 commits into
mainfrom
awskii/commitment-background-trie-build

Conversation

@awskii

@awskii awskii commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

In ModeParallel every touched key hit the prefix trie inline, on the execution thread. Touches now buffer into 16384-key chunks; each full chunk is inserted by its own goroutine while the next fills, in arrival order.

Changes

  • parallelUpdate.Collect hands a full chunk to a goroutine once the previous one is inserted; two buffers alternate. Process calls Build() to wait and insert the tail.
  • No goroutine waits for work, so a dropped collection leaks nothing.
  • TouchHashedKey passes the dedup string's bytes, which a pending chunk now outlives.
bench main this PR
TouchDirect/parallel/n=100000 17.77 ms 14.27 ms (−19.7%)
TouchAndProcess/100K/w4 59.46 ms 56.81 ms (−4.5%)
same, B/op 71.0 MiB 76.3 MiB

M5 Max, -benchtime 5x, 6 alternating rounds, p≤0.004. B/op counts a fresh Updates per op.

…unk threshold and the unreachable nil-trie guard
…kground-trie-build

# Conflicts:
#	execution/commitment/parallel_streaming_bench_test.go
@awskii
awskii marked this pull request as ready for review October 12, 2026 08:43
@awskii
awskii requested a review from AskAlexSharov October 13, 2026 06:33
@AskAlexSharov

Copy link
Copy Markdown
Collaborator

No correctness issues found.

  • execution/commitment/commitment.go:1383-1385 (SetMode): drop the added t.parallel.Reset(). When the new mode is ModeParallel, the t.Reset() at the end of SetMode already waits for the in-flight chunk. In other modes nothing reads the trie or the key arena, and Close waits.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Process-only parallel benchmarks now include residual asynchronous collection work, making comparisons inaccurate.

1 open finding
What changed in this PR

Moves ModeParallel prefix-trie construction off the execution thread using ordered, asynchronous chunks.

Changes:

  • Adds double-buffered 16,384-entry collection and asynchronous trie insertion.
  • Finalizes pending entries before processing and safely resets lifecycle state.
  • Adds correctness tests and end-to-end benchmarks.
File Description
execution/​commitment/​parallel_update.go Implements chunked asynchronous trie construction.
execution/​commitment/​commitment.go Routes parallel touches through buffered collection.
execution/​commitment/​parallel_patricia_hashed.go Builds pending entries before processing.
execution/​commitment/​parallel_collect_test.go Tests ordering, reuse, lifetimes, and goroutine completion.
execution/​commitment/​prefix_trie_test.go Adapts trie and lifecycle tests.
execution/​commitment/​commitment_test.go Adapts parallel update tests.
execution/​commitment/​fork_walk_test.go Finalizes collections before direct trie access.
execution/​commitment/​touch_collect_bench_test.go Includes deferred build work in touch benchmarks.
execution/​commitment/​parallel_streaming_bench_test.go Adds touch-and-process benchmarks.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread execution/commitment/parallel_streaming_bench_test.go
@awskii
awskii requested review from AskAlexSharov and a balanced review from Copilot October 13, 2026 08:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants