WIP: Consume WAVS Test Harness

[JakeHartnell]

PoC companion to: Lay3rLabs/WAVS#1148

Consume wavs-test-harness end-to-end

Turns the PoC sketch on this branch into a runnable consumer of the new wavs-test-harness crate (Lay3rLabs/WAVS#1147). Three integration tests now exercise the full WAVS lifecycle in-process — real delta_neutral_strategy.wasm and wavs_farmer_aggregator.wasm running through wavs-engine, real signed envelopes, real deployed SmartVaultServiceHandler, real vault.executeRebalance — without bash, docker, or a running WAVS node.

What's shipped

Three test files on top of the existing common/ fixture:

File	Scope	Wall time
tests/test_rebalance_harness.rs	Single-scenario E2E: strategy WASM → aggregator WASM → envelope → sign → real handler → real vault → hedge closed, eventId marked processed	~50s
tests/test_quorum_harness.rs	2-of-3 happy path + 1-of-3 revert + out-of-order signer revert with sort_signature_data recovery	~130s (3 tests × full setup)
tests/test_scenarios_harness.rs	Two scenarios sequentially on one fixture (delta+ → oracle deviation skip)	~53s for both

Plus:

• Fixture migration (crates/integration-tests/tests/common/): deleted anvil.rs, dropped impersonate_funded from oracle.rs, dropped mine_blocks from avantis.rs. Consumers now use wavs_test_harness::chain::{spawn_fork, impersonate_funded, mine_blocks, set_balance, enable_auto_impersonate} directly.
• Shared helpers (tests/common/harness_handler.rs): deploy_service_manager, deploy_and_wire_vault_handler, spawn_mock_forecast_sidecar. ~140 lines covering everything the three harness tests share.
• Strategy bypass (components/delta_neutral_strategy/src/lib.rs): one host::config_var("test_skip_pending_check") check that lets fork-based tests skip Avantis's "wait for pending orders" gate (Avantis's production keeper that prunes the counter doesn't run on a fork). Production operators don't set the env var, so it's a no-op outside tests.

The speed-multiplier story

This is what the harness gives you that bash/docker E2E can't:

Approach	Setup cost	Per-scenario cost	4 scenarios total
Bash E2E (scripts/integration-test.sh)	~30s spawn + deploy	~20s spawn + deploy + execute	~120s+
Standalone fork test per scenario	~38s fresh anvil + deploy	~12s execute	~200s
test_scenarios_harness (this PR)	~38s once	~7s per scenario	~66s

The killer detail: scenario 2 onwards costs <10s because the strategy WASM is already loaded into Wasmtime, the vault + service manager are already deployed, the fork is already warm. Adding scenarios is nearly free.

Required cross-repo coordination

This PR depends on a one-line pin change in WAVS to unblock alloy unification: Lay3rLabs/WAVS@5a1231b4f on branch feat/test-harness-1147. The change loosens =1.0.42 exact pins to caret (1.0.42) so downstream consumers can resolve newer alloy-rpc-types-eth versions without hitting the BlobTransactionSidecarVariant mismatch the PoC doc documented. Once #1147 lands on WAVS main, the wavs-test-harness git rev in crates/integration-tests/Cargo.toml should be repointed to a main commit (comment in the file flags this).

Trade-offs and known limitations

Worth surfacing before review so reviewers know what's deferred vs broken:

• test_skip_pending_check strategy bypass. Avantis's pendingMarketOpenCount doesn't decrement on a fork because the production keeper bot never runs. We tried storage-cheating the counter via anvil_setStorageAt; it works for pendingOrderIds but zeroing the per-pair counter breaks Avantis's live-position bookkeeping (the trade disappears). A one-line config-var bypass in the strategy was the least invasive fix. Production behavior is unchanged.
• Mock forecast sidecar. The strategy's V2 decision engine requires per-cycle VPIN/sigma/fee-richness data from a Python sidecar. Tests spin up a tokio TcpListener that returns canned values satisfying evaluate_skip_reason. The harness doesn't yet ship a generic HTTP-mock helper; this lives in common::harness_handler::spawn_mock_forecast_sidecar.
• Snapshot/revert deferred. The original plan called for SnapshotGuard::take/revert to isolate scenarios. Anvil's evm_revert on a forked chain doesn't preserve node-level state like anvil_auto_impersonate_account — after revert, the next scenario's .from(deployer) tx silently hangs. The scenarios test uses sequential ordering instead, designed so state changes from prior scenarios don't invalidate subsequent assertions. The harness's snapshot primitive is the right tool for local-Anvil scenarios; the fork-revert interaction is brittle enough today that we sidestep it. (Worth filing upstream as a foundry/anvil issue.)
• Wallet-bound provider workaround. The harness's Contract::deploy(...) and envelope::submit_envelope build TransactionRequests without a default from, requiring a wallet-bound provider. The legacy fixture uses anvil auto-impersonation instead. We deploy contracts and submit envelopes through the legacy provider with explicit .from(f.deployer) to avoid running two NonceFillers on the same account (they diverge after each tx).
• Cargo.lock churn. The alloy bump cascades through ~9300 lines of lock-file diff. The functional change is small; the rest is cargo doing its job.

How to run

# Required env
export FORK_RPC_URL="https://base-mainnet.g.alchemy.com/v2/<key>"
export FUNDED_KEY="ac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80"

# Build WASM components first
cd components/delta_neutral_strategy && cargo build --release --target wasm32-wasip2 && cd ../..
cd components/aggregator && cargo build --release --target wasm32-wasip2 && cd ../..

# Or use the task runner
task build:wasi

# Each test independently
cargo test -p wavs-farmer-integration-tests --test test_rebalance_harness -- --nocapture
cargo test -p wavs-farmer-integration-tests --test test_scenarios_harness -- --nocapture
cargo test -p wavs-farmer-integration-tests --test test_quorum_harness -- --nocapture --test-threads=1

# Regression: legacy tests still compile
cargo check -p wavs-farmer-integration-tests --tests

Test plan

• test_rebalance_harness passes against a fresh Base fork (~50s)
• test_scenarios_harness passes; total wall time well under 100s
• test_quorum_harness — all 3 scenarios pass with --test-threads=1
• cargo check --tests reports zero errors across the whole crate
• Legacy test_rebalance still compiles (it has its own pre-existing flakiness unrelated to this PR)
• No new lints / warnings in cargo clippy

Future work

Items deliberately punted out of this PR. None block merge; all are tracked here so we don't lose them.

In the wavs-defi repo

• Migrate remaining tests to the harness pattern. test_deposit.rs, test_oi_cap.rs, test_close_trade.rs, test_out_of_range.rs, test_withdraw.rs still build payloads manually and impersonate the service handler placeholder. Re-skin each on top of harness_handler::deploy_and_wire_vault_handler to exercise the same SmartVaultServiceHandler path the rebalance test uses.
• Drop the manual bash E2E (scripts/integration-test.sh) once the harness suite covers the same ground — currently it's the only thing testing some of the close-and-reopen orchestration end-to-end.
• Remove test_skip_pending_check if/when there's a cleaner way to deal with Avantis's stale counter (proper storage cheat, upstream cleanup function we missed, or moving to a local Anvil with MockAvantisTrading).
• Repoint the wavs-test-harness git rev once Build reusable WAVS integration test harness WAVS#1147 lands on main. There's a comment in crates/integration-tests/Cargo.toml flagging this.

Upstream into wavs-test-harness

• WavsServiceHandler deploy helper. Today the harness ships MockHandler (= SimpleServiceManager + SimpleSubmit). Real consumers want SimpleServiceManager + their own handler. Lift the pattern from common::harness_handler::{deploy_service_manager, deploy_and_wire_vault_handler} into the harness as a generic "deploy any IWavsServiceHandler-shaped contract and wire it to the service manager" helper.
• HTTP-mock sidecar primitive. The pattern in common::harness_handler::spawn_mock_forecast_sidecar is reusable — any consumer with an external HTTP dependency needs the same shape. A mock::http_server(handler_fn) -> url helper in the harness would let consumers skip writing tokio TcpListener glue.
• Pyth oracle helper. The harness's chain::oracle ships Chainlink V3 mocking. Pyth uses the same anvil_set_code pattern — would let us drop the last hand-rolled mock in common/oracle.rs.
• Wallet-aware envelope submission. envelope::submit_envelope requires a wallet-bound provider. A variant that accepts an explicit from: Address would work with impersonated providers without forcing consumers to duplicate the function inline.

Anvil / foundry

• evm_revert node-state preservation. When anvil_auto_impersonate_account(true) is set before a snapshot and the snapshot is reverted, the auto-impersonation flag is cleared (or otherwise becomes inert). Reproduces on anvil 1.7.1 against a Base mainnet fork via Alchemy. Worth filing upstream — the workaround in test_scenarios_harness is to use sequential ordering instead of snapshot/revert, but this limits the harness's most compelling primitive on fork-tier tests.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
wavs-vault-demo	Ready	Preview, Comment	May 17, 2026 3:16am

[layertau]

Thanks for pushing this harness consumer forward — the single-scenario rebalance test does a good job wiring the real strategy/aggregator output through a real SmartVaultServiceHandler, and the PR clearly documents the fork-specific tradeoffs.

Pass 1 (spec compliance) found missing coverage relative to the PR description, so I’m stopping before the quality pass:

Critical — test_quorum_harness does not exercise the WAVS lifecycle promised by the PR.
The PR body says the three new integration tests exercise “real delta_neutral_strategy.wasm and wavs_farmer_aggregator.wasm running through wavs-engine, real signed envelopes, real deployed SmartVaultServiceHandler, real vault.executeRebalance”. However crates/integration-tests/tests/test_quorum_harness.rs builds a RebalancePayload manually (build_rebalance_payload) and submits it directly in an envelope; it never constructs a ServiceSpec, never runs InProcRunner, and never invokes the strategy or aggregator WASM. Please either update this test to drive the payload through the same strategy/aggregator harness path, or narrow the PR description/test scope so it doesn’t claim quorum coverage is an end-to-end WAVS lifecycle test.

Important — test_scenarios_harness only exercises the strategy, not the full submit path.
crates/integration-tests/tests/test_scenarios_harness.rs configures both WASMs, but the scenarios call run_component_full only and assert the strategy responses. They never run the aggregator, sign an envelope, submit to the handler, or verify vault.executeRebalance/event processing. That may be a useful strategy matrix, but it does not satisfy the PR’s stated “three integration tests now exercise the full WAVS lifecycle” claim. Please either add the missing aggregator/envelope/handler path where relevant, or adjust the PR/test naming and stated scope to match the actual coverage.

Verification note: I attempted cargo check -p wavs-farmer-integration-tests --tests. After setting local OpenSSL paths, compilation reached the test crate but could not proceed because this checkout lacked generated Foundry artifacts under contracts/out (and forge build was blocked by missing node_modules). I did not count that as a PR failure, but please ensure the documented setup commands produce those artifacts before re-requesting review.

[JakeHartnell]

@layertau appreciate the comments, could you please make a PR into this one that addresses them?

[layertau]

Opened #210 into this branch with the scope clarifications: #210

Note: after that merges, the parent PR body should be updated with the same narrower wording so the external spec matches the code: test_rebalance_harness is the full lifecycle test; quorum is handler/quorum coverage from encoded payloads; scenarios is a strategy-output matrix.

[JakeHartnell]

@layertau upstream code has completely changed and this PR doesn't make sense to merge any more. I would love to see a similar PR based off the latest main branch testing the new Aave strategy.

[layertau]

Acknowledged. Since the parent branch is obsolete, I’ll stop reviewing #209 and close the small helper PR I opened into it.

A fresh PR against latest main for the new Aave strategy sounds like the right follow-up; that should be tracked as a new implementation work stream rather than continuing this review thread.