fix: liveness bugs [skip-line-limit]

agent/INVARIANTS.md

@@ -634,13 +634,49 @@ design citation alone does not establish current runtime behavior.
   router's `on_event` path must not do synchronous store reads. — `flow-trace/06`
 - A well-formed `E3Requested` with an unsupported committee-size/preset enum is a benign skip (emit
   `Processed` so ordering advances); ABI-decode failures still fail closed. — INDEX concern #13
+- Every wait on a peer must be bounded and must end in an attributable outcome. The aggregator
+  bounds its wait for honest `NodeDkgFold` proofs and publishes `E3Failed{DKGTimeout}` when the
+  budget expires. A late party must not be dropped from the honest set instead, because C5 is signed
+  before the fold completes and binds exactly those H keyshares. — `flow-trace/04`
+- The node shutdown deadline and the fanout accept timeout come from one constant,
+  `NODE_SHUTDOWN_DEADLINE = FANOUT_ACCEPT_TIMEOUT + 30 s`. The daemon SIGKILL delay must stay above
+  that deadline, so a node is never killed while it still flushes. — `flow-trace/06`
+- A process that must outlive its spawner must not be started with `kill_on_drop`. The handle is
+  dropped as soon as the spawner returns, so the flag kills the very process it just started;
+  `spawn_detached_process` is the correct helper, and `spawn_process` is only for a caller that
+  retains the handle for the child's whole life. A detached start must then confirm readiness
+  through the child's own protocol, because it holds no handle to observe. — `flow-trace/06`
+- Every **external** supervisor grace must also exceed `NODE_SHUTDOWN_DEADLINE`: the
+  `const _: () = assert!` guards cover only the in-process `nodes daemon` path, not the container
+  runtimes that actually run production and DAppNode nodes. `deploy/docker-compose.yml` and
+  `dappnode/docker-compose.yml` carry `stop_grace_period`, and the DAppNode gate must compare the
+  parsed value against the deadline rather than pin a literal — a pinned literal silently asserts
+  the opposite of this rule once the Rust constant moves. — `flow-trace/06`
+- A deadline that bounds a wait must be persisted as an absolute instant and re-armed on recovery,
+  not held only in an actix `SpawnHandle`. The handle dies with the process, and a bound whose
+  arming events are not replayed is silently dropped by the restart it exists to survive. —
+  `flow-trace/04`
+- A local teardown that ends this node's ability to act on an on-chain window must derive its
+  deadline from that window, not from a fixed constant. `setAccusationVoteValidity` enforces only a
+  lower bound, so governance can outgrow any constant; because every node shares the grace, the
+  whole committee would go dark together while the chain still accepts a report. — `flow-trace/05`
+- An actor that holds in-memory state derived from an event below the persisted snapshot cursor must
+  persist that state, because replay starts at the cursor and never redelivers the event. Verified
+  caches, own-proof records, and collector inputs all follow this rule. — `flow-trace/06`
 
 ### Schema evolution
 
 - Rust type compatibility is **not** a storage-migration strategy: every durable payload carries an
   explicit schema version; add/remove/reorder of fields requires a compatibility test against
   checked-in fixtures; version mismatch runs a tested migration or fails startup with an actionable
   error. — `ARCHITECTURE.md`
+- `#[serde(default)]` does **not** make a new field backward compatible on a bincode-persisted type.
+  Bincode encodes a struct as a fixed sequence with no field names, so a record written before the
+  field existed fails to decode with "unexpected end of file" rather than defaulting; the attribute
+  only covers a value built in memory. Adding a field to any type reachable from
+  `Repository`/`Persistable` therefore requires bumping `SCHEMA_VERSION`, which converts the opaque
+  decode error into the explicit halt-and-migrate path. The JSON daemon socket is the exception: it
+  is field-named, so `serde(default)` behaves as expected there.
 
 ## Build / config sync
 

agent/flow-trace/00_INDEX.md

@@ -330,6 +330,7 @@ them are in the reference app (`examples/CRISP`), not the protocol.
 | Z-16 | **Safe slashing-manager retirement**                            | Resolved | Retiring managers remain authorized for their assigned E3s, bans, slash locks, proposals, and pending routes. `closeE3` now also waits for the objective accusation submission deadline. Revocation requires every canonical obligation counter to be zero.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Z-24 | **Build-bound crypto configuration**                            | Resolved | The generated configuration ID binds the encryption scheme, exact parameter hash, and circuit version. Requests accept only that append-only configuration and snapshot its verifier addresses. Rust validates the emitted ID against its local build, and the indexer uses local immutable parameters plus the E3's frozen ID instead of querying mutable live parameter bytes.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
 | Z-45 | **In-call request fee bound**                                   | Resolved | Each request supplies its expected fee token, expected crypto configuration, and maximum fee. Any change between quote and inclusion reverts before escrow transfer. The SDK obtains a fresh quote when the caller does not provide an explicit maximum.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+| L-01 | **Restart-visible liveness defects (live chaos audit)**         | Resolved | A 21-round kill/restart audit of a five-node swarm closed a class of defects that only a live restart exposes. Replay starts at the persisted snapshot cursor, so an actor that held state derived from an earlier event lost it silently: the commitment-consistency cache and the node's own C0 proof are now durable, and a restarted node no longer accuses honest peers or stalls its fold at N−1 of N. The router now tears down a slashably-failed E3 after a two-hour grace instead of holding it until process restart. The aggregator now bounds its wait for honest `NodeDkgFold` proofs and fails with `DKGTimeout` instead of stalling. `bb` subprocesses have a hard timeout. The shutdown deadline and the fanout accept timeout come from one constant. See `CHAOS_TEST_REPORT.md` for the per-round evidence.                                                                                                                               |
 
 ### Scope of the Zenith `Z-` entries
 

agent/flow-trace/04_DKG_AND_COMPUTATION.md

@@ -330,6 +330,26 @@ aggregation path can terminate deterministically instead of stalling on missing
 `PublicKeyAggregator` and `ThresholdPlaintextAggregator` dispatch the aggregator requests instead of
 pairwise folding.
 
+**Bounded node-proof collection:** a failed `NodeDkgFold` reports itself, but a member that dies
+mid-fold sends nothing. `PublicKeyAggregator` therefore arms a budget when it enters
+`GeneratingC5Proof` and cancels it when every honest proof arrives. If the budget expires,
+`fail_on_missing_node_proofs` names the parties that did not deliver and publishes
+`E3Failed { failed_at_stage: CommitteeFinalized, reason: DKGTimeout }`. The late parties are not
+dropped from the honest set instead: C5 is signed before the cross-node fold completes and binds
+exactly those H keyshares, so a different honest set would invalidate a published proof.
+
+The budget is durable, not only an in-process timer. `GeneratingC5Proof.node_proof_deadline_at`
+holds the absolute unix second, and `EffectsEnabled` re-arms it for the time that is actually left.
+The in-process handle is an actix `SpawnHandle` that dies with the process, and none of the three
+events that arm it are replayed on recovery, so without the persisted instant a restart would
+silently drop the bound and restore the unbounded stall it exists to prevent.
+
+The budget is `E3_DKG_NODE_PROOF_TIMEOUT_SECS`, and its default matches the DKG window (7200 s)
+because a node proof that arrives after the window cannot be used by its E3. Measured
+`ZkNodeDkgFold` at the `secure-8192` preset is 132 s at N=3, 380 s at N=5, and 904 s at N=9, and a
+member that restarts mid-DKG re-proves about 5500 s of inner circuits before it can fold again. Do
+not lower the budget below that restart-inclusive worst case for the deployed committee size.
+
 **Failure bridge:** `ProofRequestActor` now converts proof-generation worker failures and local
 proof-signing failures into terminal round failures instead of only logging that the proof-bearing
 artifact will not be published. DKG-path proofs (`C0` through `C5`) emit

agent/flow-trace/05_FAILURE_REFUND_SLASHING.md

@@ -1308,7 +1308,16 @@ When CommitteeMemberExpelled event arrives from EVM:
     │   │   → Single cleanup signal for all per-E3 actors
     │   │   NOTE: E3Failed with a misbehaviour reason (DKGInvalidShares, etc.) does
     │   │   NOT trigger E3RequestComplete — the accusation/slashing lifecycle must
-    │   │   complete first.
+    │   │   complete first. A slashable failure instead schedules a teardown after a
+    │   │   grace derived from the chain: ACCUSATION_REPORTING_WINDOW (1 day) plus the
+    │   │   registry's accusationVoteValidity plus a vote-in-flight margin, so the
+    │   │   accusation and vote windows can close while the E3 still leaves the router.
+    │   │   The grace is NOT a fixed constant: setAccusationVoteValidity enforces only a
+    │   │   lower bound, so governance can raise the window past any constant and every
+    │   │   node would tear down together while the chain still accepts a report.
+    │   │   SLASHABLE_FAILURE_TEARDOWN_GRACE (2 h) is the fallback used only when no
+    │   │   chain window is known. The deadline is persisted in the request-router
+    │   │   checkpoint and re-armed on restart.
     │   └─ E3StageChanged(Failed) and the same non-slashing E3Failed arriving after teardown
     │       are silently ignored (expected on-chain lag)
     │

agent/flow-trace/06_DEACTIVATION_AND_COMPLETION.md

@@ -239,7 +239,7 @@ interfold start → running node
     ├─ Persists Shutdown and waits for acknowledged EventBus fanout
     ├─ Flushes the sequencer and event-store pipeline
     ├─ Drains open snapshot batches in event order, flushes the backing store, and closes it
-    ├─ Enforces a 30-second deadline and exits unsuccessfully on failure
+    ├─ Enforces a 60-second deadline and exits unsuccessfully on failure
     └─ Flushes the optional operational JSON log collector
 
 On restart:
@@ -273,6 +273,17 @@ On restart:
 │        CiphernodeSelected events are likewise not guaranteed to replay.
 │      → Recovered aggregator roles, selected party IDs, and DHT document interests are injected
 │        directly from snapshots. Startup does not append synthetic recovery events.
+│      → Per-E3 durable caches are restored before replay, because replay starts at the snapshot
+│        cursor and never redelivers an earlier event:
+│        • CommitmentConsistencyChecker restores its verified-proof cache from
+│          `//commitment_consistency/v1/{e3_id}`. Without it the node holds no record of its own
+│          C0 proof, every peer C3 that points at that C0 fails the link check, and the node
+│          accuses honest peers.
+│        • ProofRequestActor restores its own C0 proof from `//own_c0_proof/v1/{e3_id}` and
+│          re-publishes `DKGInnerProofReady { seq: 0 }`. Without it the node fold waits forever
+│          at N−1 of N inner proofs.
+│        • ThresholdKeyshare rebuilds the decryption-share collector and its timeout whenever the
+│          state is `ReadyForDecryption`, including when no share has arrived yet.
 ├─ Sync module replays:
 │   → Arm the current NetReady listener before the network transport can publish readiness
 │   4. Replay EventStore events since the snapshot cut (effects still disabled)
@@ -312,6 +323,12 @@ event pipeline flushed, open snapshot batches drained, and the backing store flu
 deadline. Detached work that is not owned by those barriers can still be cancelled by process exit;
 operators must continue to follow the production shutdown precautions.
 
+`NODE_SHUTDOWN_DEADLINE` is derived as `FANOUT_ACCEPT_TIMEOUT + 30 s` in `e3-events`, and both the
+CLI and the daemon read that one constant. The daemon waits for the deadline plus five seconds
+before it sends `SIGKILL`. An external supervisor must allow at least as long: a systemd
+`TimeoutStopSec` or Docker stop timeout below 65 seconds kills a node while it still flushes its
+event log.
+
 The three long-lived libp2p `NetEvent` broadcast consumers (`NetEventTranslator`,
 `DocumentPublisher`, and `NetSyncManager`) treat Tokio's `Lagged(n)` receive result as a recoverable
 overload signal: they emit a bounded structured warning containing only the static consumer name and