Skip to content

test: enable txpool insertion batching with 5ms timeout#5578

Draft
mattsse wants to merge 2 commits into
mainfrom
mattsse/test-txpool-batch-timeout
Draft

test: enable txpool insertion batching with 5ms timeout#5578
mattsse wants to merge 2 commits into
mainfrom
mattsse/test-txpool-batch-timeout

Conversation

@mattsse

@mattsse mattsse commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Pins reth dependencies to mattsse/reth@676d996756paradigmxyz/reth#20661 plus the tempo_tx_batch_metrics timing logs from #5542 and a new sampled info log of every dispatched insertion batch with its size and flush trigger (immediate/full/timeout).

Enables the new batch-and-timeout mode: 5ms batch timeout, max batch size lowered from 50000 to 150. Also exposes a transaction_pool_batch_size histogram.

For testing under tempo load; not meant to be merged until the reth PR lands and the deps point back at upstream.

Patches reth deps to paradigmxyz/reth#20661 and configures the txpool
batcher with a 5ms batch timeout and a max batch size of 150.
@mattsse

mattsse commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

derek bench run-pairs=1

@decofe

decofe commented Jun 10, 2026

Copy link
Copy Markdown
Member

cc @mattsse

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs mattsse/test-txpool-batch-timeout
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20 duration=90 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=500 baseline=main feature=mattsse/test-txpool-batch-timeout baseline-hardfork="" feature-hardfork="" gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false
  • Bloat: 100000 MiB
  • Preset: tip20
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 1
  • Baseline blocks: 159
  • Feature blocks: 164

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 21635 19606 -9.38% ⚪
Gas Throughput [Mgas/s] 957.1 867.4 -9.37% ⚪
Block Time Mean [ms] 560.1 551.3 -1.57% ⚪
Block Time P50 [ms] 556.0 557.0 +0.18% ⚪
Block Time P90 [ms] 610.0 610.0 +0.00% ⚪
Block Time P99 [ms] 654.0 640.0 -2.14% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.1 251.1 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2317.3 2105.6 -9.14% ⚪
P50 [ms] 229.9 229.4 -0.22% ⚪
P90 [ms] 244.9 460.7 +88.12% ⚪
P99 [ms] 281.6 474.1 +68.36% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 27.5 22.9 -16.73%
Finish P90 [ms] 37.6 35.9 -4.52%
Finish P99 [ms] 47.3 46.6 -1.48%
Pool Fetch P50 [ms] 9.4 8.8 -6.38%
Pool Fetch P90 [ms] 19.6 18.6 -5.10%
Pool Fetch P99 [ms] 26.5 23.2 -12.45%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 2950.2 3078.8 +4.36%
Serialized Block Size P90 [KiB] 3343.1 3506.6 +4.89%
Serialized Block Size P99 [KiB] 3776.2 3972.8 +5.21%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 23.0 434.0 +1786.96%
Fill Idle P99 [ms] 102.0 449.0 +340.20%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2223.2 1947.5 -12.40% ⚪
P50 [ms] 240.9 240.7 -0.08% ⚪
P90 [ms] 276.2 283.2 +2.53% ⚪
P99 [ms] 312.7 304.6 -2.59% ⚪

Observability

Points at mattsse/reth@676d996756, which is the reth PR plus the
tempo_tx_batch_metrics timing logs from #5542 and a new sampled log of
dispatched batch size and flush trigger (immediate/full/timeout).
@mattsse

mattsse commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

derek bench run-pairs=1

@decofe

decofe commented Jun 10, 2026

Copy link
Copy Markdown
Member

cc @mattsse

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs mattsse/test-txpool-batch-timeout
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20 duration=90 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=500 baseline=main feature=mattsse/test-txpool-batch-timeout baseline-hardfork="" feature-hardfork="" gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false
  • Bloat: 100000 MiB
  • Preset: tip20
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 1
  • Baseline blocks: 160
  • Feature blocks: 163

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 22300 22489 +0.85% ⚪
Gas Throughput [Mgas/s] 986.6 994.9 +0.84% ⚪
Block Time Mean [ms] 555.3 548.4 -1.24% ⚪
Block Time P50 [ms] 558.0 548.0 -1.79% ⚪
Block Time P90 [ms] 584.0 583.0 -0.17% ⚪
Block Time P99 [ms] 611.0 610.0 -0.16% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.1 251.1 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2373.8 2383.7 +0.41% ⚪
P50 [ms] 229.6 226.2 -1.48% ⚪
P90 [ms] 239.8 239.4 -0.17% ⚪
P99 [ms] 277.2 259.1 -6.53% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 25.5 24.9 -2.35%
Finish P90 [ms] 36.1 38.4 +6.37%
Finish P99 [ms] 42.6 48.4 +13.62%
Pool Fetch P50 [ms] 10.2 12.2 +19.61%
Pool Fetch P90 [ms] 24.7 25.1 +1.62%
Pool Fetch P99 [ms] 31.9 30.8 -3.45%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 3057.0 3055.6 -0.05%
Serialized Block Size P90 [KiB] 3426.7 3441.4 +0.43%
Serialized Block Size P99 [KiB] 3996.3 4203.7 +5.19%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 103.0 0.0 -100.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2288.4 2315.8 +1.20% ⚪
P50 [ms] 240.4 236.9 -1.46% ⚪
P90 [ms] 261.9 260.5 -0.53% ⚪
P99 [ms] 280.8 280.8 +0.00% ⚪

Observability

@mattsse

mattsse commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

derek bench samply=true run-pairs=1

@decofe

decofe commented Jun 10, 2026

Copy link
Copy Markdown
Member

cc @mattsse

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs mattsse/test-txpool-batch-timeout
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20 duration=90 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=500 baseline=main feature=mattsse/test-txpool-batch-timeout baseline-hardfork="" feature-hardfork="" gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false samply
  • Bloat: 100000 MiB
  • Preset: tip20
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 1
  • Baseline blocks: 163
  • Feature blocks: 163

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 21069 21476 +1.93% ⚪
Gas Throughput [Mgas/s] 932.1 950.1 +1.93% ⚪
Block Time Mean [ms] 552.7 547.4 -0.96% ⚪
Block Time P50 [ms] 552.0 552.0 +0.00% ⚪
Block Time P90 [ms] 589.0 584.0 -0.85% ⚪
Block Time P99 [ms] 620.0 615.0 -0.81% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.1 251.1 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2257.5 2293.2 +1.58% ⚪
P50 [ms] 228.0 225.4 -1.14% ⚪
P90 [ms] 247.0 238.6 -3.40% ⚪
P99 [ms] 309.5 252.9 -18.29% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 27.9 30.1 +7.89%
Finish P90 [ms] 44.5 42.6 -4.27%
Finish P99 [ms] 57.6 54.3 -5.73%
Pool Fetch P50 [ms] 9.7 11.4 +17.53%
Pool Fetch P90 [ms] 22.6 24.2 +7.08%
Pool Fetch P99 [ms] 31.8 31.9 +0.31%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 2909.5 2884.7 -0.85%
Serialized Block Size P90 [KiB] 3259.7 3302.9 +1.32%
Serialized Block Size P99 [KiB] 3823.5 3891.6 +1.78%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 17.0 0.0 -100.00%
Fill Idle P99 [ms] 161.0 0.0 -100.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2185.9 2203.5 +0.80% ⚪
P50 [ms] 236.4 240.1 +1.57% ⚪
P90 [ms] 263.5 259.5 -1.52% ⚪
P99 [ms] 275.3 278.1 +1.02% ⚪

Observability

Samply Profiles

@mattsse

mattsse commented Jun 10, 2026

Copy link
Copy Markdown
Contributor Author

derek bench run-pairs=1 feature-args="--txpool.max-batch-size 500 --txpool.batch-timeout 5ms"

@decofe

decofe commented Jun 10, 2026

Copy link
Copy Markdown
Member

cc @mattsse

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs mattsse/test-txpool-batch-timeout
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20 duration=90 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=500 baseline=main feature=mattsse/test-txpool-batch-timeout baseline-hardfork="" feature-hardfork="" gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false feature-args="--txpool.max-batch-size 500 --txpool.batch-timeout 5ms"
  • Bloat: 100000 MiB
  • Preset: tip20
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 1
  • Baseline blocks: 161
  • Feature blocks: 164

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 22279 22163 -0.52% ⚪
Gas Throughput [Mgas/s] 985.6 980.5 -0.52% ⚪
Block Time Mean [ms] 554.3 545.9 -1.52% ⚪
Block Time P50 [ms] 553.0 545.0 -1.45% ⚪
Block Time P90 [ms] 584.0 581.0 -0.51% ⚪
Block Time P99 [ms] 610.0 605.0 -0.82% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.1 251.1 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2373.0 2368.2 -0.20% ⚪
P50 [ms] 229.0 225.8 -1.40% ⚪
P90 [ms] 243.2 238.6 -1.89% ⚪
P99 [ms] 275.5 249.1 -9.58% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 23.8 28.8 +21.01%
Finish P90 [ms] 38.2 40.4 +5.76%
Finish P99 [ms] 47.3 49.7 +5.07%
Pool Fetch P50 [ms] 10.2 12.8 +25.49%
Pool Fetch P90 [ms] 24.8 26.9 +8.47%
Pool Fetch P99 [ms] 30.7 31.6 +2.93%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 3044.5 3020.0 -0.81%
Serialized Block Size P90 [KiB] 3424.7 3422.7 -0.06%
Serialized Block Size P99 [KiB] 3755.1 3981.9 +6.04%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 40.0 0.0 -100.00%
Fill Idle P99 [ms] 103.0 0.0 -100.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2360.3 2333.2 -1.15% ⚪
P50 [ms] 231.9 229.3 -1.12% ⚪
P90 [ms] 253.2 254.7 +0.59% ⚪
P99 [ms] 271.3 265.8 -2.03% ⚪

Observability

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants