Skip to content

ci(bench): use existing recipients preset, multiple TIP-20 tokens#5618

Merged
klkvr merged 3 commits into
mainfrom
alexey/update-tip20-preset-token-selection
Jun 11, 2026
Merged

ci(bench): use existing recipients preset, multiple TIP-20 tokens#5618
klkvr merged 3 commits into
mainfrom
alexey/update-tip20-preset-token-selection

Conversation

@shekhirin

@shekhirin shekhirin commented Jun 11, 2026

Copy link
Copy Markdown
Member

Summary

  • Route TIP20 benchmark transfers through a generated token address set instead of a single fixed recipient.
  • Default the e2e benchmark preset to tip20_existing_recipients and pass the token count through the bench pipeline so all TIP20 presets share the same recipient pool.

Testing

  • Not run (not requested)

@shekhirin

shekhirin commented Jun 11, 2026

Copy link
Copy Markdown
Member Author

derek bench duration=10 run-pairs=1

@decofe

decofe commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

cc @shekhirin

⚠️ Benchmark cancelled. View logs

@decofe

decofe commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

cc @shekhirin

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs alexey/update-tip20-preset-token-selection
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20_existing_recipients duration=10 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=main feature=alexey/update-tip20-preset-token-selection baseline-hardfork=T6 feature-hardfork=T6 gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false
  • Bloat: 100000 MiB
  • Preset: tip20_existing_recipients
  • Target TPS: 50000
  • Duration: 10s
  • Run pairs: 1
  • Baseline blocks: 18
  • Feature blocks: 18

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 9613 9652 +0.41% ⚪
Gas Throughput [Mgas/s] 610.0 612.7 +0.44% ⚪
Block Time Mean [ms] 531.2 520.1 -2.09% ⚪
Block Time P50 [ms] 500.0 494.0 -1.20% ⚪
Block Time P90 [ms] 658.0 654.0 -0.61% ⚪
Block Time P99 [ms] 667.0 682.0 +2.25% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.2 251.2 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.2 251.2 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1358.0 1294.8 -4.66% ⚪
P50 [ms] 224.1 228.9 +2.14% ⚪
P90 [ms] 235.5 236.2 +0.30% ⚪
P99 [ms] 238.0 236.7 -0.55% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 11.0 9.7 -11.82%
Finish P90 [ms] 13.7 12.1 -11.68%
Finish P99 [ms] 13.8 14.0 +1.45%
Pool Fetch P50 [ms] 8.7 9.0 +3.45%
Pool Fetch P90 [ms] 10.8 18.2 +68.52%
Pool Fetch P99 [ms] 19.2 28.7 +49.48%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 1216.9 1157.4 -4.89%
Serialized Block Size P90 [KiB] 1572.1 1535.6 -2.32%
Serialized Block Size P99 [KiB] 1692.0 1687.3 -0.28%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1254.7 1209.0 -3.64% ⚪
P50 [ms] 231.4 232.2 +0.35% ⚪
P90 [ms] 376.1 366.8 -2.47% ⚪
P99 [ms] 385.1 403.1 +4.67% ⚪

Observability

@shekhirin shekhirin changed the title feat(bench): select TIP20 token recipients dynamically ci(bench): use existing recipients preset, multiple TIP-20 tokens Jun 11, 2026
@shekhirin shekhirin added the A-ci Related to github workflows or other build and lint tools label Jun 11, 2026
@shekhirin shekhirin marked this pull request as ready for review June 11, 2026 09:49
@shekhirin shekhirin requested a review from Zygimantass as a code owner June 11, 2026 09:49
@decofe

decofe commented Jun 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

cc @shekhirin

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: main vs alexey/update-tip20-preset-token-selection
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=tip20_existing_recipients duration=10 bloat=100 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=main feature=alexey/update-tip20-preset-token-selection baseline-hardfork=T6 feature-hardfork=T6 gas-limit=1000000000 run-pairs=1 otlp=true metrics=false no-cache=false force-bloat=false samply
  • Bloat: 100000 MiB
  • Preset: tip20_existing_recipients
  • Target TPS: 50000
  • Duration: 10s
  • Run pairs: 1
  • Baseline blocks: 17
  • Feature blocks: 18

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 9897 9111 -7.94% ⚪
Gas Throughput [Mgas/s] 628.0 577.9 -7.98% ⚪
Block Time Mean [ms] 549.6 523.4 -4.77% ⚪
Block Time P50 [ms] 557.0 499.0 -10.41% ⚪
Block Time P90 [ms] 648.0 667.0 +2.93% ⚪
Block Time P99 [ms] 668.0 675.0 +1.05% ⚪
Serialized Block Size / Tx P50 [B/tx] 251.1 251.1 +0.00% ⚪
Serialized Block Size / Tx P90 [B/tx] 251.2 251.2 +0.00% ⚪
Serialized Block Size / Tx P99 [B/tx] 251.2 251.2 +0.00% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1329.7 1315.9 -1.04% ⚪
P50 [ms] 237.4 222.6 -6.23% ⚪
P90 [ms] 247.5 234.2 -5.37% ⚪
P99 [ms] 247.8 239.7 -3.27% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 12.0 12.8 +6.67%
Finish P90 [ms] 14.7 15.6 +6.12%
Finish P99 [ms] 14.7 20.6 +40.14%
Pool Fetch P50 [ms] 5.6 7.0 +25.00%
Pool Fetch P90 [ms] 9.5 17.7 +86.32%
Pool Fetch P99 [ms] 25.4 20.0 -21.26%
Included Tx Exec P50 [ms] - -
Included Tx Exec P90 [ms] - -
Included Tx Exec P99 [ms] - -
Invalid Tx Exec P50 [ms] - -
Invalid Tx Exec P90 [ms] - -
Invalid Tx Exec P99 [ms] - -
Invalid Tx Attempts P50 0.0 0.0 0.00%
Invalid Tx Attempts P90 0.0 0.0 0.00%
Invalid Tx Attempts P99 0.0 0.0 0.00%
Invalid Tx Skips 0 0 0.00%
Nonce Too Low Skips 0 0 0.00%
Serialized Block Size P50 [KiB] 1272.6 1206.4 -5.20%
Serialized Block Size P90 [KiB] 1565.5 1523.8 -2.66%
Serialized Block Size P99 [KiB] 1732.7 1546.4 -10.75%
Fill Overhead P50 [ms] - -
Fill Overhead P90 [ms] - -
Fill Overhead P99 [ms] - -
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1188.8 1230.9 +3.54% ⚪
P50 [ms] 260.4 229.9 -11.71% ⚪
P90 [ms] 355.7 380.9 +7.08% ⚪
P99 [ms] 385.5 390.1 +1.19% ⚪

Observability

Samply Profiles

@klkvr klkvr added this pull request to the merge queue Jun 11, 2026
Merged via the queue into main with commit a4799ca Jun 11, 2026
36 checks passed
@klkvr klkvr deleted the alexey/update-tip20-preset-token-selection branch June 11, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@klkvr klkvr klkvr approved these changes

@Zygimantass Zygimantass Awaiting requested review from Zygimantass Zygimantass is a code owner

Assignees

No one assigned

Labels

A-ci Related to github workflows or other build and lint tools

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shekhirin @decofe @klkvr