Add a fuzzer for hax/aeneas - #41
Open
karthikbhargavan wants to merge 1 commit into
Open
Conversation
A source-level fuzzer that mutates the tests/src corpus, packs functions into crates, drives them through rustc -> charon -> aeneas, and classifies the outcome (crash / wrong-rejection / optional Lean elaboration) with dedup against a committed findings DB. Includes a borrow-weighted generator and a native-vs-Lean semantic differential. Runs in CI on cryspen's self-hosted nix runners (.github/workflows/ fuzz-nightly.yml): nix build .#aeneas gives a pin-matched aeneas+charon, and the harness runs inside nix develop. run --ci exits 3 only on a NEW (non-deduped) finding, so the job is red exactly when a change introduces a new crash/miscompilation. Nightly (fast) + weekly (longer) + manual dispatch. The Phase-3 pure-IR stage differential (needs the fork-only -dump-pure-ir) is not included here; it lives on the dump-pure-ir branch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A source-level fuzzer for the Aeneas Rust→Lean translation, under a
self-contained
fuzz/directory, plus a CI workflow that runs it on a schedule.It mutates the
tests/srcseed corpus (14 seeded mutators + a borrow-weightedgenerator), packs functions into crates, drives them through
rustc → charon → aeneas, and classifies each outcome — crash,wrong-rejection (rustc accepts but aeneas rejects), or optional Lean
elaboration failure — with dedup against a committed findings DB. A
native-vs-Lean semantic differential (
fuzz/semdiff/) is included forcatching miscompilations.
Running in CI (self-hosted nix runners)
.github/workflows/fuzz-nightly.ymlmirrorsci.yml:nix build .#aeneasyieldsresult/bin/aeneasand a pin-matchedresult/bin/charon(the flake symlinks charon into the aeneas output;flake.lock/charon-pinare the pin, already enforced by thecheck-charon-pinflake check) — nothing to pin by hand.nix develop --command(Rust viarustup,elan for the optional Lean oracle).
run --ciexit contract: exit3iff a NEW (non-deduped,non-expected-reject) finding was recorded versus the committed
fuzz/findings/db.json. Known bugs and feature-gate rejections keep the jobgreen. So CI goes red exactly when a change introduces a new
crash/miscompilation. The seed is derived from the run id and logged, so any
red run reproduces locally.
~90 min) +
workflow_dispatch(time_budget,seed,oracle_scope=lean).What it has already surfaced
Run against both the fork toolchain and upstream
mainduring development(details in
fuzz/FINDINGS.md):assert!of the same bool → "no bottoms" (InterpExpressions.ml:55),both targets. Filed as Crash: two
assert!of the same boolean aborts translation ("no bottoms in the value") #28.&mut→SymbolicToPureValues.ml:366(fork toolchain only; likely a version delta). Filed as Crash: a closure that reads through a captured
&mutaborts translation (SymbolicToPureValues.ml:366) #40.forloop with conditionalbreakand code after it: "Could not match the contexts" AeneasVerif/aeneas#1206, N4 → upstream bug: Matching on chars throws an Uncaught exception AeneasVerif/aeneas#797,F4/F6 → Crash:
returning a borrowed value from inside aloopaborts whole-crate translation (reorder_loop_outputs) #22/Inverted can_end guard in eliminate_shared_loans mutates frozen abstractions; valid programs rejected #24,from_fn→ Error with reference produced by a closure AeneasVerif/aeneas#804. Thedb.jsonbaseline records all ofthese so they don't re-flag.
Scope / notes
differential is a local/manual activity (
fuzz/setup/build_upstream.sh).pure-eval/,stage-diff/) needsthe fork-only
-dump-pure-irfeature and is not in this PR; it lives onthe dump-pure-ir branch. This PR is buildable on
main(harnesscargo test:50 unit + 6 integration green here).
to
mainfor the nightly/weekly lanes to run.Before it goes live (maintainer)
[self-hosted, linux, nix]) isavailable for this repo, and do a first manual
workflow_dispatchrun tovalidate the environment (rustup/elan in
nix develop, artifact upload).maintainer to triage and commit into
db.json+ repro dirs.🤖 Generated with Claude Code