Tests: Make main's suites runnable and green again, and reject a stale-ABI cached executable

[mho22]

Why

Four defects stop main from running or passing its own test suites. A fifth lets a stale binary through the package cache into the one place the host trusts.

The exact-ABI vitest runner cannot enumerate. scripts/ci-vitest-evidence-classes.tsv is an exact-cover whitelist, so drift in either direction is a hard stop. #1322 deleted tests/package-system/fetch-binaries-allow-stale.test.ts and tests/package-system/package-source-publish-contract.test.ts but left their rows; three later PRs added test files without classifying them. scripts/ci-run-test-suite.sh vitest exact-abi-source has been unable to start since.

Two host tests throw before collecting. host/test/kernel-scratch-contract.test.ts and host/test/kernel-blocking-retry-snapshot.test.ts read Rust sources by path, and #1321 moved those modules from crates/kernel/src/ to crates/runtime-core/. The scratch contract's regex also needs pub fn from_syscall rather than pub(crate) fn, because the module now crosses a crate boundary.

A third host test calls a method that no longer exists. host/test/privileged-projection.test.ts calls getMountSetIdCapability, which is absent from host/src/vfs/vfs.ts.

cargo test -p xtask is red. local_build::tests::clean_cascades_to_products_embedding_the_package asserts that cleaning a package nothing depends on removes only that package, and it named ruby. #1336 ships Ruby as a shell lazy-archive, so ruby now has a dependent and its removal set is fourteen nodes: eight packages and six browser products.

A cached executable can declare any ABI and still be trusted. A vim built before an ABI bump reached a desktop built after it, and the host refused it at exec with refusing unsafe program artifact before execution. Three things stack to produce that. packages/registry/vim/vim-src/ is a gitignored, persistent build tree; the ABI marker comes from libc/glue/channel_syscall.c, which wasm32posix-cc compiles at link time and which is not a make dependency, so after a bump make saw src/vim up to date, relinked nothing, and build-vim.sh re-collected the previous binary. wasm_artifact_policy_failures_for then let it through: it required __abi_version to be exported but compared its value only for side modules, so the resolver cached a stale executable under the current ABI's key. And build-vim.sh was outside vim's cache identity, because build_input_digests_from_repo hashes build.toml.inputs and the git inputs but not script_path, and vim declared no inputs.

What changed

Five commits, each one defect, plus one follow-up review commit (see below).

The evidence registry drops the two rows for files #1322 deleted and gains rows for the three unclassified files. A file is source-only only when its whole import closure stays clear of the binary resolver, which the runner enforces separately from the cover check; tests/package-system/build-input-crate-closure.test.ts qualifies, the two browser tests do too.

The two snapshot contracts follow the modules to crates/runtime-core/. The privileged-projection test follows the set-ID capability out of VirtualPlatformIO.

The clean-cascade test reads its leaf out of the plan graph — the first package no other node depends on — instead of naming one. The assertion keeps its meaning and no longer breaks the next time an image embeds whichever package the test happened to name. Nine packages in the checked-in graph still qualify.

The artifact policy compares the __abi_version value, not just its presence. A required export declaring anything other than wasm_posix_shared::ABI_VERSION is now a policy failure, guarded to non-side-modules so the existing dylink path keeps owning those; both call sites, cache-entry validation and build-script-output validation, go through the shared helper. Five hardcoded __abi_version -> 0 wasm literals in build_deps.rs would have failed that check, so they now derive from the existing emit_wasm_build_script helper and track ABI_VERSION. Two cache-mutation tests built their "different bytes" by rewriting that same body constant, which the new check rejects before the generation assertion runs; they now append a custom section the policy ignores. build-vim.sh drops $SRC_DIR/src/vim before make, as build-netcat.sh already does, and vim declares its inputs. vim's revision stays 4 — the added inputs already move the cache key.

Follow-up review (commit bceead629)

A local review of this branch (by Claude, at the author's request) hardened
one test and documented one deliberately-narrow check.

The clean-cascade leaf test still named its candidate by reconstructing a
package node from each graph.packages name under a hardcoded wasm32
arch. A package planned only for wasm64 would produce a node present neither
as a graph key nor in any dependency edge, so the "nothing depends on it"
predicate was trivially satisfied and clean_removal_set ran against a node
the graph does not contain — the assert passing while verifying nothing. The
test now reads the leaf out of the graph's real node set (the Package nodes
already in graph.dependencies), so the candidate is honest for any planned
target, fully realizing the "read the leaf out of the graph" intent.

The follow-up also considered making the executable __abi_version check as
strict as the side-module branch (failing Missing/Invalid/Err, not only
Present(mismatch)). That change was reverted because it is wrong by design:
genuine absence is already reported by the required-exports check (a Missing
branch double-reports), and legitimate executables carry markers
artifact_identity classifies Invalid — extraction only succeeds for a
constant-thunk shape, which the instrumenter guarantees for side modules but
not for arbitrary executables (the spidermonkey disabled-fork fixture is
accepted precisely in this state). Present(mismatch) is therefore the only
reliable staleness signal for executables. A comment at the check now records
that rationale so the narrower branch is not "fixed" into a regression later.

Validation

cargo test -p xtask --target aarch64-apple-darwin — 555 pass, 0 fail. Red on main before this branch.

cargo test --workspace --exclude xtask --target aarch64-apple-darwin, inside scripts/dev-shell.sh — 1999 pass, 0 fail.

scripts/ci-run-test-suite.sh vitest exact-abi-source — the runner enumerates again and reports 208 of 212 files passing, 3139 tests. The 11 remaining failures, in test/exec-state-tracking.test.ts, test/spawn-pid-authority.test.ts and test/vfork-production-mechanism.test.ts, are an artifact of the worktree, not of this branch: those tests load real binaries out of local-binaries/, that tier was built at ABI 44 by an unrelated branch, and main is at ABI 43. Decoding __abi_version out of local-binaries/programs/wasm32/exec-child.wasm gives i32.const 44. Every failure message is the ABI policy rejecting those bytes.

Follow-up commit bceead629 re-validated via scripts/dev-shell.sh at --target aarch64-apple-darwin: cargo test -p xtask build_deps module (287 pass, 0 fail) and clean_cascades_to_products_embedding_the_package (pass).

Two of the five defects were confirmed against a clean origin/main checkout rather than only in the working worktree.

[brandonpayton]

Kudos on getting the 1337 PR, @mho22 ;)

[brandonpayton]

Follow-up review commit: bceead629

I (Claude, reviewing this branch locally at Brandon's request) reviewed the
diff, found two candidate issues, and pushed one fix plus one clarifying
comment. Recording the reasoning here so the trail is legible.

1. Clean-cascade leaf test now reads the leaf from the graph's real node set

The leaf test built its candidate node by reconstructing a package node from
each graph.packages name under a hardcoded wasm32 arch. A package
planned only for wasm64 would then produce a node present neither as a graph
key nor in any dependency edge, so the "nothing depends on it" predicate was
trivially satisfied and clean_removal_set ran against a node the graph does
not actually contain. The assert would still pass while verifying nothing —
the exact robustness the "read the leaf out of the graph" rewrite was meant to
guarantee.

Fix: iterate the Package nodes already in graph.dependencies and pick one
nothing depends on, so the candidate is a node the graph truly contains for
any planned target.

2. Executable __abi_version check — investigated, intentionally left as-is (comment only)

I first suspected the executable branch was too permissive next to the
side-module branch: it only fails Present(mismatch) and ignores
Missing/Invalid/Err. Making it strict broke two existing tests, and they
were right to break:

• Genuine absence of __abi_version is already reported by the
  required-exports check, so a Missing branch double-reports.
• The spidermonkey disabled-fork fixture legitimately carries a marker that
  artifact_identity classifies Invalid (extraction only succeeds for a
  constant thunk / call-delegate shape — guaranteed for side modules by the
  instrumenter, not for arbitrary executables), and the test expects
  acceptance.

So Present(mismatch) really is the only reliable staleness signal for
executables, and the narrow check is correct by design. I reverted the
behavior change and instead added a comment at the check documenting why
it is deliberately narrower than the side-module branch, so the next reader
doesn't repeat my mistake.

Validation

Run via scripts/dev-shell.sh with --target aarch64-apple-darwin (xtask is
a host tool):

• cargo test -p xtask build_deps module — 287 pass, 0 fail
• clean_cascades_to_products_embedding_the_package — pass