feat(staking,poll): incremental voter weight view + snapshot at PutPollResult (IIP-59 PR 2/6)

[envestcc]

Summary

Second PR of the IIP-59 protocol-native voter reward distribution series (spec: iotexproject/iips#73). Stacked on #4865 — PR 1 introduces the commissionRate schema + SetCommissionRate action; this PR keeps a live per-voter weight view up to date and freezes both the commission rate and the per-voter weights per-epoch, materializing what PR 3's GrantEpochReward will consume.

• No behavior change pre-flag. Gated on the same FeatureCtx.NoVoterRewardDistribution as PR 1. Reward distribution still runs the legacy path.
• What's new:
  • a live in-memory VoterWeightView in viewData, kept incrementally consistent by native handlers and by the contract-staking receipt-driven event dispatch
  • at PutPollResult (mid-epoch snapshot of next-epoch active delegates), we now (a) copy staking.Candidate.CommissionRate → state.Candidate.CommissionRate — poll's existing NxtCandidateKey PutState persists it, no extra key — and (b) read the sorted per-voter slice out of the view and write a byte-deterministic per-candidate blob under the new _voterWeightSnap state-trie namespace
• Reader for PR 3: VoterWeightsFromSnapshot(sr, candID) → []VoterWeight returns the frozen slice for one candidate.

Design change vs. the first draft on this branch

The prior version of this PR scanned all buckets at PutPollResult time to build the aggregate. That was O(all buckets) inside a hot state.db call and — worse — read from the contract staking indexer, a db independent of state.db, inside a consensus mutation path. Both problems are gone in this redesign:

• Incremental live view. Native handlers (handleCreateStake, handleUnstake, handleChangeCandidate, handleTransferStake, handleDepositToStake, handleRestake, handleCandidateEndorsement, handleStakeMigrate) push (cand, voter, Δw) deltas into VoterWeightView as they run. Contract-staking events push deltas through a VoterDeltaSink plumbed via context (no ContractStakeView interface widening, no mock breakage). SnapshotForEpochReward at PutPollResult reads O(voters-per-cand) and does zero bucket iteration.
• State.db-native consensus path. The consensus mutation path (Handle) never reads the contract staking indexer db. The view is fed through the same receipt-driven dispatch that already updates contractStakeView. The view itself is memory-only; on Start, CreateBaseView rebuilds it once from current buckets + each indexer's current per-voter aggregates. The durable half of the story is the frozen per-epoch snapshot blobs under _voterWeightSnap, not the live view.
• Cheap Snapshot/Revert. The view is a base map + a change-overlay wrapper (Wrap() layers a change map on top of the receiver so Revert throws the overlay away; Fork() deep-clones for parallel working sets; Commit() folds change into base and prunes zeros).

Storage layout

• New 1-byte namespace tag _voterWeightSnap = 5 in the staking namespace (protocol.go), following the _const / _bucket / _voterIndex / _candIndex / _endorsement pattern.
• Per-candidate blob key = _voterWeightSnap || candIdentifier.Bytes() → 21 bytes total, mirroring _voterIndex / _candIndex layout.
• Payload = stakingpb.VoterWeightSnapshot { repeated VoterWeightEntry (bytes voter, bytes weight) }. Entries pre-sorted by voter address bytes; marshalled output is byte-deterministic given identical logical state — a load-bearing invariant for consensus and for the byte-equality skip in the writer.

Correctness invariants (deliberate; PoC #4811 review issues folded in)

• Self-stake exclusion uses the strict native check ContractAddress == "" && Index == cand.SelfStakeBucketIdx. Fixes PoC review issue Docker build fails #5 (all contract buckets with Index=0 erroneously got self-stake weighting). Applied uniformly: self-stake buckets never enter the view, so all handler hooks that touch them skip the delta.
• Unstaked buckets skipped. handleUnstake emits the removal delta; buckets already unstaked at the time of handleTransferStake / handleChangeCandidate are skipped because they left the view earlier.
• Byte-equality skip in writeVoterWeightSnapshot: encode → compare to existing stored blob bytes → skip PutState when identical. Cuts write amplification in the steady state.
• Empty entries → DelState, not "write empty blob." VoterWeightsFromSnapshot returns (nil, nil) for the ErrStateNotExist path either way, but keeping state clean matches how other candidate-scoped namespaces behave.
• State read errors other than ErrStateNotExist propagate (wrapped). Fixes PoC review issue Implement IoTeX blockchain explorer API MVP #4 (all errors were being treated as "no buckets").
• Deterministic voter ordering — Weights(cand) returns a slice sorted by voter address bytes. No map iteration reaches the consensus-visible output. Fixes PoC review issue Batch update 2019-04-27 6PM PDT #2 (map iteration would have caused block-hash divergence between nodes).

Poll integration

• poll/util.go setCandidates: after the shortlist is built but before sm.PutState under NxtCandidateKey, call staking.FindProtocol(...).SnapshotForEpochReward(...). Must run first — the commission-rate copy mutates the candidate list in place, and the persisted state.Candidate needs the frozen value.
• FindProtocol lookup means poll still doesn't hard-depend on staking's concrete Protocol type; the nil-check keeps legacy tests that skip staking registration working.

Tests

• voter_weight_view_test.go (19 cases): base apply/aggregate/read, sorting stability, wrap overlay semantics, Snapshot/Revert parity against a Fork+Apply reference, Commit fold + zero-prune, nested Wrap, per-candidate isolation, sink context plumbing, deterministic incremental updates.
• vote_view_handler_test.go (9 cases, systemcontractindex/stakingindex): PutBucket fresh / same-cand-same-owner / owner-change / candidate-change; DeleteBucket / missing-bucket / muted-bucket; nil-sink tolerance; sink integration that a fresh-then-delete sequence balances to zero after Commit.
• voter_weight_snapshot_test.go: updated to construct a populated view directly rather than seeding buckets, then assert SnapshotForEpochReward writes the expected blob, the byte-equality skip triggers on unchanged inputs, and empty entries DelState the blob. End-to-end key layout, encoding determinism, reader roundtrip, and error propagation are all covered.

Test plan

• go build ./...
• go vet ./action/protocol/staking/... ./action/protocol/poll/... ./systemcontractindex/stakingindex/...
• go test ./action/protocol/staking/... ./systemcontractindex/stakingindex/... -count=1 — 390 passing (2 unrelated pre-existing gomonkey flakes in TestProtocol_FetchBucketAndValidate/validate_owner, confirmed to reproduce without this PR)
• go test ./action/... ./state/... -count=1 — smoke pass
• New view + handler + snapshot tests × 10 iterations — no flakes
• Behavior under IsToBeEnabled flipped mid-test — deferred to PR 5's e2e determinism harness
• Cross-node block-hash equality under load — deferred to PR 5's stress harness

Series roadmap

• PR 1: feat(action,staking,state): commissionRate schema + SetCommissionRate (IIP-59 PR 1/6) #4865 — commissionRate schema + SetCommissionRate action
• PR 2 (this PR): incremental voter weight view + snapshot @ PutPollResult
• PR 3: voter reward distribution (epoch path) — reads what PR 2 writes
• PR 4: block reward folding
• PR 5: stress test + e2e determinism harness
• PR 6: mainnet fork activation (flip NoVoterRewardDistribution gate)

🤖 Generated with Claude Code

[envestcc]

Holding review — reviewer flagged two architectural issues with computeCandidateVoterWeights:

1. Perf risk: O(voters) trie reads per top-N delegate at PutPollResult → mint-timeout risk at mainnet scale (potentially 10K+ reads on a single mid-epoch block).
2. Layering violation: ContractStakingIndexer is backed by an independent db.KVStore (blockindex/contractstaking/indexer.go:63), not state.db. Consensus paths must depend on state.db lineage only — reading the indexer here opens a divergence vector if any node's indexer is behind/inconsistent.

Both are the same root fix: move voter-weight aggregation to an incrementally-maintained viewData field, updated by (a) native staking handlers and (b) the existing contractStakeView.Handle(ctx, receipt) receipt-processing path (which is already state.db-native and per-block deterministic). PutPollResult then just reads the persisted per-candidate blob — O(top-N), no indexer.

Will redo PR 2 with that design on the same branch and force-push. Marking as draft in the meantime.

[envestcc]

Force-pushed the redesigned PR 2. Summary of the delta vs. the first draft:

• No more indexer reads on the consensus path. The first draft called contractStakingIndexer.BucketsByCandidate(...) inside SnapshotForEpochReward, which runs on the consensus mutation path. Since the indexer db is independent of state.db, that was a state-mismatch waiting to happen. The redesign feeds a live in-memory VoterWeightView through the same receipt-driven contractStakeView.Handle path that already updates contract staking state — no indexer reads reach Handle.
• PutPollResult is now O(voters-per-cand), not O(all buckets). Native handlers push (cand, voter, Δw) deltas as they run; PutPollResult reads the sorted slice out of the view and writes.
• Cheap Snapshot/Revert via a base + change-overlay wrapper (matches the shape of bucketPool / candidateCenter).
• Contract sink through context (VoterDeltaSink). Avoids widening the ContractStakeView interface — no mock breakage in existing tests or third-party impls.
• Same feature-flag guard, same _voterWeightSnap = 5 blob layout, same reader signature (VoterWeightsFromSnapshot) — PR 3 doesn't need to change based on this redesign.

Updated the PR body + title to reflect the new design. Ready for another look.

[guo]

Review: incremental voter-weight view looks sound, but the self-stake state machine isn't hooked

Overall the design holds up well. A few things I specifically checked and they're correct:

• No time-decay in the weight basis. CalculateVoteWeight reads only stored bucket fields (StakedDuration, AutoStake, StakedAmount) — a bucket's weight is constant between the actions that mutate it. That's the precondition that makes the delta-based view sound, so 👍.
• Snapshot determinism (sorted-by-voter + big-endian weight.Bytes() + byte-equal skip + DelState on empty + non-ErrStateNotExist errors propagating) is consensus-safe.
• Commission freeze ordering in poll/util.go setCandidates is right: SnapshotForEpochReward mutates state.Candidate.CommissionRate in place before the NxtCandidateKey PutState, so the frozen rate is persisted.
• viewData Snapshot/Revert/Commit/Fork mirror the existing contractsStake wrapper semantics.
• Contract-staking event deltas carry the correct signs (candidate/owner change → full move; same/same → net delta; muted/unstaked → zero, matching the native isUnstaked exclusion).

Blocking: three handlers that flip a bucket between voter-visible and self-stake-excluded don't emit a view delta

The PR adds ApplyVoterWeightDelta to the bucket-lifecycle handlers (create / unstake / change / transfer / deposit / restake) plus endorsement and migrate. But the self-stake designation state machine also moves buckets in and out of the view, and those handlers aren't touched by this PR:

Handler	Transition	Missing delta
handleCandidateActivate (handler_candidate_selfstake.go)	previous self-stake bucket → vote bucket and new vote bucket → self-stake bucket	+w(prev, false) and −w(new, false) (both legs)
handleCandidateDeactivate confirm op (handler_candidate_selfstake.go)	self-stake bucket → vote bucket (via csm.deactivate)	+w(bucket, false)
handleCandidateTransferOwnership (handler_candidate_transfer_ownership.go)	self-stake bucket → vote bucket (the needClear path)	+w(bucket, false)

The tell is that handleCandidateEndorsement already compensates right after csm.deactivate:

if err := csm.deactivate(cand, bucket, ...); err != nil { ... }
csm.ApplyVoterWeightDelta(cand.GetIdentifier(), bucket.Owner, p.calculateVoteWeight(bucket, false))

handleCandidateDeactivate's confirm op calls the same csm.deactivate (which does SubVote(w,true) + AddVote(w,false) + clears SelfStakeBucketIdx, i.e. converts the bucket back to a normal vote bucket) but does not apply the matching +w to the view. handleCandidateActivate and handleCandidateTransferOwnership have the analogous gap.

Why it matters — two layers:

1. The live view drifts, so the frozen snapshot carries wrong per-voter weights → rewards are misallocated (an owner whose bucket became self-stake keeps earning voter reward on it; an owner whose bucket reverted to a vote bucket is under-credited).
2. More seriously: buildVoterWeightBaseFromState rebuilds the base from current bucket state at startup, and it correctly excludes/includes these buckets. So rebuild-from-state ≠ the incrementally-maintained view after any of these three actions. A node that restarts between one of these actions and the next PutPollResult will freeze a different snapshot blob than a node that didn't restart → state-root divergence.

This is all gated behind NoVoterRewardDistribution, so there's no mainnet exposure before the PR 6 activation — but it needs to be closed before the PR 5 determinism harness, since this is exactly the class of drift that harness is meant to catch.

The fix is mechanical: add the ApplyVoterWeightDelta calls to those three handlers, mirroring the endorsement hook. Suggested invariant to enforce (and to cover in the PR 5 harness): every handler that writes SelfStakeBucketIdx must emit the corresponding view-membership delta, and the harness should assert rebuild-from-state == live view at each PutPollResult.

Non-blocking

• Dead code: voterDeltaSinkHandler (+ newVoterDeltaSinkHandler, effectiveWeight) in vote_view_handler.go has no references anywhere — the sink is actually wired through the modified voteViewEventHandler (voteView.Handle → VoterDeltaSinkFrom(ctx)). Go won't flag an unused package-level type, so it compiles, but it's ~60 lines of a second, parallel implementation of the same delta logic. Worth removing to avoid ambiguity about which path is authoritative.
• Stale snapshot blobs for delegates that drop out of the top-N are never deleted (SnapshotForEpochReward only iterates the next-epoch active list). Not a correctness issue — GrantEpochReward only reads delegates it pays — but the _voterWeightSnap namespace grows unbounded over churn. A lazy cleanup could be folded into PR 3/4.

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[envestcc]

Thanks for the careful read — all three misses are real. Fixed in 36e06e4d:

handleCandidateActivate (self-stake ↔ vote conversions). Two deltas were missing:

• After converting the previous self-stake bucket back to a vote bucket, emit +w(prev, false) so the view sees it rejoin.
• Before overwriting SelfStakeBucketIdx with the newly self-staked bucket, emit -w(new, false). Order matters: downstream isSelfStake checks need to see the pre-transition membership.

handleCandidateDeactivate (confirm op). csm.deactivate clears SelfStakeBucketIdx and folds the self-stake bucket back into the vote set — same conversion as above. Now emits +w(bucket, false) after the successful deactivate.

handleCandidateTransferOwnership (settled-bucket clear path). Capture the settled self-stake bucket before mutating candidate, then on the needClear path emit +w(prev, false). Guarded by !isUnstaked() since unstaked buckets are excluded from the view.

Dead code. You're right — voterDeltaSinkHandler had no callers; the live path is voteViewEventHandler.emitVoterDelta, wired via context by voteView.Handle. Deleted the struct + factory + PutBucket/DeleteBucket/effectiveWeight (~65 lines).

The 27 self-stake / deactivate / transfer-ownership handler tests still pass, and the full staking package (354 tests) is green. Please take another look.