Skip to content

node, chain: parallel aggregate submission for #907 - #919

Merged
ch4r10t33r merged 1 commit into
mainfrom
fix/issue-907-parallel-aggregate
May 24, 2026
Merged

node, chain: parallel aggregate submission for #907#919
ch4r10t33r merged 1 commit into
mainfrom
fix/issue-907-parallel-aggregate

Conversation

@ch4r10t33r

Copy link
Copy Markdown
Contributor

Summary

  • Merge #916 parallel aggregate work: replace serial std.Io.Threaded (concurrent_limit=1) with bounded non-blocking submits onto the shared ThreadPool (aggregate_max_inflight=4).
  • Check the in-flight cap before the expensive state cloneAndRelease, so saturated workers do not waste deep clones.
  • Add zeam_aggregate_skip_total{reason="spawn_failed"} when ThreadPool enqueue fails (never fall back to inline ~10s FFI on the libxev thread).
  • Clarify Prometheus HELP text: lean_pq_sig_aggregated_signatures_building_time_seconds measures per-att_data wrap time inside computeAggregatedSignatures, not the XMSS STARK worker; operators should use zeam_aggregate_worker_duration_seconds for end-to-end FFI cost (#907).

Context

On the ansible devnet, zeam aggregators were dropping ~50% of slot triggers as in_flight because the XMSS FFI takes ~11s while the slot interval is ~6s and only one aggregate worker could run at a time (#907). Live logs on zeam_8 with a #916-era image showed the skip rate improved to ~10%, but publish lag and timely=none merge timing remain separate follow-ups.

This branch is based on current main (includes #908 trivial-input pre-filter) and supersedes the open #916 branch for review/merge.

Test plan

  • zig build test --summary all
  • Deploy to devnet aggregator (zeam_8) and confirm zeam_aggregate_skip_total{reason="in_flight"} rate drops vs pre-parallelize aggregation as an aggregator #916 baseline
  • Confirm zeam_aggregate_worker_duration_seconds p50 stays ~10s for multi-sig inputs (parallelism helps throughput, not per-run FFI cost)
  • Confirm zeam_aggregator_publish_aggregations_total continues incrementing on duty subnet under load

@zclawz

zclawz commented May 24, 2026

Copy link
Copy Markdown
Contributor

Reviewed PR #919.

No blocking findings from my pass. The change correctly moves aggregate submission off the single dedicated Io.Threaded worker onto the shared bounded thread pool, does the in-flight cap check before the expensive state clone, releases the in-flight counter on all early/error paths I checked, joins outstanding aggregate workers before chain teardown, and updates the Prometheus help text/skip reasons consistently.

Validation:

  • GitHub checks on 6fa31fe5dc220e84f692a1e86999e14e47cf6600 are green (test, build, lint, docker/prover jobs).
  • I attempted local zig build test --summary all, but this workspace’s Zig/dependency combo fails before repo tests with dependency build-script API errors (Build.Graph.io / b.modules.put), so I’m treating CI as the authoritative validation here.

LGTM from me.

Document that lean_pq_sig_aggregated_signatures_building_time_seconds
measures per-att_data wrap time inside computeAggregatedSignatures,
not the XMSS STARK worker. Point operators at
zeam_aggregate_worker_duration_seconds for end-to-end FFI cost.
@ch4r10t33r
ch4r10t33r force-pushed the fix/issue-907-parallel-aggregate branch from 6fa31fe to fa84f37 Compare May 24, 2026 18:24

@zclawz zclawz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. I re-checked the current head after the force-push; the tree matches the version I reviewed, with no new diff from my prior pass.

@ch4r10t33r
ch4r10t33r merged commit c759e79 into main May 24, 2026
9 checks passed
@ch4r10t33r
ch4r10t33r deleted the fix/issue-907-parallel-aggregate branch May 24, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants