Release v4.0.0

The record of what was tested for this release. Issues close when their PR reaches develop; release-level verification lives here. Each box gets its evidence beside it (job ID, SHA, measured number) when checked. Anything not done before the tag moves to Not verified with its reason.

Version

v4.0.0, decided 2026-10-04. docs-site/src/reference/versioning.md lists configuration-YAML keys as public API, and Unknown config keys are silently ignored — a typo can select the wrong model #496 makes a config carrying a previously ignored key stop the run, which is an incompatible change, so this is a MAJOR bump. The v3.6.0 milestone's issues and PRs moved to v4.0.0 and that milestone is closed.
SV representation: emit canonical SVTYPE for junctions instead of generic BND pairs #313 (canonical SVTYPE for junctions) moved to v4.1.0 for review, 2026-10-04. Remove the deprecated rneat alias shim (v4.0.0) #710 (remove the deprecated rneat alias) stays in v4.0.0.

Per PR (merged to `develop` since v3.5.1)

#800: rebuild the bundled COSMIC tumor models (#761)

CHANGELOG line (Changed): the seven bundled COSMIC tumor models were rebuilt with the current builder; only snp_distro (per-context somatic SNV placement weights) moved, by −7.4% to +12.9% per context, so gen-cancer-reads places somatic SNVs slightly differently — in the v4.0.0 CHANGELOG
gen-cancer-reads run end to end with the new model files (2026-10-08, develop 43e1611, release build). All seven tools/cosmic_*.json.gz on ecoli (4.6 Mb), 4x total, purity 0.5, tumor_mutation_rate 0.002, sv_rate_scale 0: each exited 0 and produced 8,567–9,036 somatic SNVs. Observed per-frame mutability (somatic SNVs ÷ frame occurrences in ecoli) against each model's de-cumulated snp_distro: r = 0.997–0.999 for every model against its own weights. Each per-tissue run matches its own model best (BRCA 0.998 vs ≤0.983; lung 0.997 vs ≤0.979; skin 0.998 vs ≤0.737). The three pan-cancer + SV models carry SNV weights identical to the pan-cancer model (max |diff| 0.0), so their rows tie by construction.
- Not verified here: the SV components (sv_rate_scale 0); Rebuild the bundled COSMIC tumor models with the current builder (#761) #800 reports they did not change.

#802: re-freeze somatic_snv_recall at 0.840 (#680)

Owes nothing for release. Not a claim that 0.84 is correct; the cause stays open in somatic_snv_recall sits 4.5 sigma below baseline, replicated on two builds #680.

#807: reject config keys a subcommand does not read (#496)

CHANGELOG line: unknown keys now stop the run (behavior change for users) — in the v4.0.0 CHANGELOG
Moved to Not verified: configs written by Delta-only scripts were checked by search, not by running them

#808: close issues on merge to develop; this release ticket — process only, owes nothing.

#809: straddling deletion no longer widens its mutated copy to the contig end (#691)

CHANGELOG line: input_vcf runs with a literal SV deletion crossing a symbolic SV's boundary had inflated coverage (+48% on ecoli) — in the v4.0.0 CHANGELOG
Moved to Not verified: SCN reproducer rerun on Delta
Known limitation, not addressed: the deleted bases past the sub-region boundary still get reference coverage

#810: max_reads samples uniformly, not from the head of the FASTQ (#721)

CHANGELOG line: capped gen-seq-error-model fits read the whole file and give different models — in the v4.0.0 CHANGELOG
Moved to Not verified: head-sampled vs uniform capped fit on HG002
Moved to Not verified: capped-run timing

#811: fold soft-masked bases to A/C/G/T at parse time (#790)

CHANGELOG line: soft-masked references now simulate exactly like uppercase ones (de novo SNP placement in masked sequence changes); input breakend ALTs are written uppercase — in the v4.0.0 CHANGELOG

#812: write NM on golden BAM records (#536)

CHANGELOG line: golden BAM records carry NM — in the v4.0.0 CHANGELOG
golden_bam_nm (#[ignore], samtools oracle) green in release-gates: run 37216181098 (PR Document unknown-key rejection and the golden BAM's NM tag #815, after Write NM on golden BAM records (#536) #812 merged), 2,892 records, calmd agrees on all
Not run on a real genome; the oracle runs on H1N1

#813: harness --mutate modes and the CI guard (#755)

Merged as 7d62f36. delta-harness green on the PR run (f78d59a): all 18 suites end with a mutation summary. CI runs on PRs, not on pushes to develop.
Owes no CHANGELOG line (CI only).

#814: keep observed indel lengths when one type is absent (#766)

Merged as a40fa42, docs included
CHANGELOG line: an insertions-only or deletions-only training VCF keeps its observed lengths — in the v4.0.0 CHANGELOG

#815: docs for #496 and #536

Merged as 040d654

#817: name the cause when gen-seq-error-model has no weight to fit (#767)

Merged as 21429e4
CHANGELOG line: a BAM whose MD tags disagree with its reads is refused by name, with the count and a samtools calmd fix, instead of failing as a bare InvalidWeights (any disagreement refuses; no best-effort fit); a mate population with no bases is refused by name — in the v4.0.0 CHANGELOG
PATCH-level; not run on a real BAM with stale MD (synthetic fixtures, by design)

#818: stop warning that a breakend has no END/SVLEN (#497)

Merged as faaf4a9
CHANGELOG line: a breakend supplied via input_vcf no longer logs a WARN about missing END/SVLEN; a <DEL> or other span-requiring SV without one still does — in the v4.0.0 CHANGELOG
PATCH-level; only input_vcf breakends tested, de novo BNDs share the code path untested

#821: remove the deprecated rneat alias (#710)

Merged as f278b17
CHANGELOG line (Compatibility): the rneat alias is removed; invoke eidolon. Release binaries never shipped it; conda installs no longer include it — in the v4.0.0 CHANGELOG
MAJOR-level; no actual conda build run, only its cargo install line

#822: test check_query_denominator; measured harness coverage (#819 steps 1–2)

Merged as 534c818
No CHANGELOG line (tests, CLAUDE.md, workflow comment only)

#823: make 22 weak unit tests assert content (#819 step 3)

Merged as 219a5dd
No CHANGELOG line (test code only)

#826: gen-mut-model REF checks (#819 decisions)

Merged as 8992df2
CHANGELOG line (Compatibility): gen-mut-model refuses a VCF in which any SNP or indel REF disagrees with the reference, as NEAT2 did; it stops at the first one, names it with both bases, and gives the bcftools norm --check-ref x command to drop such records. SNPs at contig edges are no longer counted toward the rate. A BED naming no reference contig says so — in the v4.0.0 CHANGELOG
Not run on a real VCF/reference pair

#827: filter-reads output naming (#824)

Merged as acc1186
CHANGELOG line (Fixed): filter-reads no longer crashes on plain .fastq/.vcf input or mis-names files in a directory containing a dot; output is always .gz, so plain input gains the suffix — in the v4.0.0 CHANGELOG
PATCH-level; fixtures only

#828: test audit steps 4–5 (#819)

Merged as 43e1611
No CHANGELOG line (test code only)

Still in the milestone, not started

none: all milestone work merged (#819 closed 2026-10-08)

#658 re-measured 2026-10-06 from job 22583881: simulated mapq0_pct unchanged at 0.0001 after #657, so the cause is separate from error placement. Per plan the investigation waits; removed from this milestone.

#746 moved to v4.1.0 (2026-10-05): changing a default alters simulated content, which versioning.md places outside the public API, so it needs no MAJOR bump.

Release procedure

cargo test --workspace --no-fail-fast: passed inside release_shakeout.sh step 1 (2026-10-08, develop 43e1611). Re-run on release/v4.0.0 (Release prep: v4.0.0 #829): 1,104 passed.
release-gates (cargo test --workspace -- --ignored) green on the release PR: Release prep: v4.0.0 #829, run 37787703543. All 8 ignored tests ran and passed (5 realigned SV gates, transition-matrix round trip, golden-BAM NM vs calmd, validate differential). All other Release prep: v4.0.0 #829 checks pass. Release PR Release v4.0.0 #830 (develop → main, dbc7c518): all checks pass, including release-blockers (first run against main) and release-gates run 37879367133 (8 ok, 0 failed).
cargo clippy --workspace --all-targets -- -D warnings: clean, same run. Re-run on release/v4.0.0 (Release prep: v4.0.0 #829): 1,104 passed.
scripts/release_shakeout.sh, full mode (DATA_DIR=~/code/data/rneat_shakeout, hg38 chr22 + NA12878 chr22 BAM): ALL STEPS PASSED, 2026-10-08, develop 43e1611. First run with fragment_model: corrected by Reject config keys a subcommand does not read (#496) #807. Final gen-reads step: 999,186 read pairs, 0:54 wall, 272 MB peak.
- Resources vs the prior logged shakeout (2026-05-22, several releases older, config not confirmed identical): every step lower in time and peak RSS (frag 753→255 MB, reads 544→272 MB); no regression.
- Two Placed 0/6 GC-weighted fragments in a 340bp region (extension_budget=0) warnings are the two 340 bp non-N islands on chr22 (14,419,554–14,419,894 and 14,419,994–14,420,334), each bounded by N gaps, shorter than the 350 bp mean fragment. True sequence termini; expected.
Junk-file check: no empty tracked files, no odd path names (develop 43e1611). Re-run on release/v4.0.0 (Release prep: v4.0.0 #829): clean.
Five binary assets on the release page (2026-10-09, run 37932211311, all 5 jobs success): aarch64-unknown-linux-gnu-rhel8, x86_64-apple-darwin, x86_64-pc-windows-msvc.exe, x86_64-unknown-linux-gnu, x86_64-unknown-linux-gnu-rhel8 (7.7–8.8 MB). The downloaded linux-gnu asset, run under its download name, reports eidolon 4.0.0+b7b3742 and runs gen-reads.
Docs site serves: docs.yml deployed from b7b37420; index, getting-started/installation.html and the new upgrading/from-3.html all HTTP 200, the last with its content.
Conda sha256 follow-up PR: v4.0.0 follow-ups: conda sha256, complete CHANGELOG Not verified list #831 into develop (one-way flow), sha256 ab0c433a…bb17f, identical across two downloads. Open; merges at the user's discretion.

Not verified for this release

Each was owed by a PR and not done before the tag. None blocks the release; all need Delta or real data.

Reject config keys a subcommand does not read (#496) #807: configs written by Delta-only scripts were checked by search for unknown keys, not by running the scripts. A Delta script carrying an unread key would now fail at job start, with the key named.
Stop a straddling deletion widening its alt haplotype to the contig end (#691) #809: the SCN reproducer was not rerun on Delta (needs a rebuild). Known limitation, not addressed: deleted bases past the sub-region boundary still get reference coverage.
Sample max_reads uniformly across the FASTQ, not from its head (#721) #810: head-sampled vs uniform capped gen-seq-error-model fits not compared on HG002; a capped run not timed, so the HPC chapter's 1:30:00 is not measured.
Rebuild the bundled COSMIC tumor models with the current builder (#761) #800: the rebuilt tumor models' SV components not exercised end to end (sv_rate_scale 0 in the run above); Rebuild the bundled COSMIC tumor models with the current builder (#761) #800 reports they did not change.
gen-mut-model: refuse REF mismatches as NEAT2 did, with a bcftools fix in the error #826, filter-reads: name outputs from the file name, accept plain input (#824) #827: the gen-mut-model REF check and the filter-reads fix tested on fixtures only, not real data.
Stop warning that a breakend has no END/SVLEN (#497) #818: de novo breakends share the BND-warning code path; no test runs them.

CHANGELOG v4.0.0: complete the Not verified list from #816

v4.0.0 released 2026-10-09.

Tag: v4.0.0 on b7b37420, the merge of Release v4.0.0 #830 (develop → main, first release under the one-way flow).
Assets: all 5 (run 37932211311). The linux-gnu binary run under its download name reports eidolon 4.0.0+b7b3742.
Docs: deployed from b7b37420; index, a deep page and upgrading/from-3.html return 200.
Release notes: written from the CHANGELOG entry, including the full Not verified list.
Follow-ups: v4.0.0 follow-ups: conda sha256, complete CHANGELOG Not verified list #831 (conda sha256 plus the completed CHANGELOG Not verified list) is open into develop.
Not verified: every item is in the section above and in the release notes.

Post-release Delta validation of v4.0.0, submitted 2026-10-09 on the Delta checkout at the v4.0.0 tag, rebuilt: eidolon 4.0.0+b7b3742.

job	run	compared against
22772429	`model_builders` on HG002 (BAM, v4.2.1 truth VCF, FASTQ, GRCh38)	must build; tests #826's REF check on a real VCF
22772430	`realism_panel` on HG002, GRCh38	job 22583881
22772431	`germline_e2e`, eidolon only, chr22	recall on eidolon's own truth
22772432	`cancer_pipeline`, chr22 default	job 22596356 (SNV recall 0.856)
22772433	`sv_pipeline` smoke, chr22, `SV_RATE_SCALE=30`	all stages run; no BND (one contig)
22772434? / 22772435	capped `gen-seq-error-model` (5 M reads, timed; #810) and the SCN reproducer (#809)	HPC chapter's 1:30:00; truncated "no R2 window" count = 0

Results to follow here.

Delta validation results, first round (all on eidolon 4.0.0+b7b3742):

gen-mut-model: refuse REF mismatches as NEAT2 did, with a bcftools fix in the error #826 verified on real data. model_builders (22772429): gen-mut-model passed on the HG002 v4.2.1 truth VCF against GRCh38 (2:04, 2,085 MB), so the REF check accepts it. gen-seq-error-model passed on the full 29 GB FASTQ (13:34, 52 MB).
Germline e2e (22772431, chr22 30x, HaplotypeCaller + hap.py): SNP recall 0.9897 / precision 0.9995 / F1 0.9946 (51,124 of 51,655); INDEL 0.9780 / 0.9873 / 0.9826 (7,455 of 7,623); truth Ti/Tv 2.23. No earlier germline_e2e run compared.
Cancer (22772432, chr22 30x, purity 0.6, Mutect2): SNV recall 0.853 (314/368, CI 0.814–0.887) vs 0.856 in job 22596356. No regression; the somatic_snv_recall sits 4.5 sigma below baseline, replicated on two builds #680 gap is unchanged. Indels 16/23.
SV smoke (22772433, chr22, SV_RATE_SCALE=30): all stages ran. BND coverage hole (one contig, expected); manta_INS wholly filtered by --passonly (sv_pipeline: check_denominator verifies only the truth side, so precision rests on an unverified query denominator #511, expected).
Sample max_reads uniformly across the FASTQ, not from its head (#721) #810 timed. Capped fit (22772434), 5 M reads from the 29 GB HG002 FASTQ: 11:08 wall, 53 MB, 1 CPU, against 13:34 uncapped. The HPC chapter's 1:30:00 is generous.
Stop a straddling deletion widening its alt haplotype to the contig end (#691) #809 SCN reproducer (22772442): 0 truncated "no R2 window" lines, but 0 such lines in total; confirming that debug logging was on and the SVs were processed before counting it.

Not run yet:

realism_panel (22772430): REAL_BAM not found at neat_data/hg002/hg002.bam.
The BAM-based builders, skipped for the same reason.

Harness defect found: model_builders.sbatch skips each BAM step when INPUT_BAM is set but missing ([[ -n && -f ]], line 115), and reports only the downstream round-trip failure. To fix on develop.

Delta validation, second round: the HG002 BAM had been removed in a cleanup and is rebuilt from the retained reads. stage_hg002.sh defaults (one chunk, all reads) reuse the existing FASTQs, the same inputs as before, so job 22583881 remains the realism baseline.

22776603: stage_hg002.sh (bwa-mem2 → hg002.bam)
22776604: realism_panel on HG002 (afterok 22776603)
22776605: model_builders BAM steps + round-trip (afterok 22776603)

Also open: the SCN run (22772442) wrote 2.59 M pairs at coverage: 10, implying about 78 Mb covered. On ecoli, the same settings give 9.95x actual depth. Checking the assembly's non-N size and the skip counts before recording #809 as verified.

#809 verified on its original trigger. SCN reproducer (22772442: debug logging on, SV VCF loaded, 2,587,978 pairs written) had 0 truncated "no R2 window" lines. The low read count is explained by the input. The SCN Delly call set's het DELs span 524 Mb and het DUPs 454 Mb over a 144.4 Mb assembly, and 26.7 Mb is under homozygous DELs; overlapping SVs compose multiplicatively. The same run without input_vcf (22776712) wrote 4,781,880 pairs, matching the 4.78 M expected at 10x.

Staging (22776603) built and indexed hg002.bam, then exited silently on a SIGPIPE in its contig check: fixed in #833. The BAM is complete; realism_panel and the model_builders BAM steps are being resubmitted against it. Harness fix for the earlier silent skip: #832.

Rebuilt hg002.bam verified: samtools quickcheck OK, 68,846,158 mapped / 102,252 unmapped (0.15%). Resubmitted against it: 22781361 realism_panel (vs 22583881), 22781362 model_builders BAM steps + round-trip.

model_builders BAM steps (22781362): all PASS on the rebuilt HG002 BAM. frag_length 1:04 / 248 MB; gc_bias 2:35 / 11.5 GB; bam_models 2:39 / 11.5 GB; round-trip produced 188,421 valid read pairs from the built models. With 22772429, all five builders have now passed at full scale on real human data with v4.0.0.
realism_panel (22781361) refused to measure, correctly. The resubmit omitted READ_LEN=250 for this 2x250 library, so the read-length band (145–151) excluded 89,628 of 89,670 real reads, and the guard stopped the run. Resubmitting with READ_LEN=250. 22583881's other settings are not recorded beyond its BAM and reference, so the comparison assumes defaults otherwise.

Delta validation of v4.0.0: complete

All on eidolon 4.0.0+b7b3742 (the release commit), 2026-10-09/10.

check	job(s)	result
model builders at full scale on real HG002 (BAM, v4.2.1 truth VCF, 29 GB FASTQ, GRCh38)	22772429, 22781362	all five PASS; round-trip 188,421 valid pairs from the built models
#826 REF check on a real VCF	22772429	verified: HG002 v4.2.1 truth VCF accepted against GRCh38
germline e2e (chr22 30x, HaplotypeCaller + hap.py)	22772431	SNP recall 0.9897 / prec 0.9995; INDEL 0.9780 / 0.9873
cancer e2e (chr22 30x, purity 0.6, Mutect2)	22772432	SNV recall 0.853 vs 0.856 baseline (22596356): no regression
SV smoke (chr22, `SV_RATE_SCALE=30`)	22772433	all stages run; BND hole by construction
#810 capped-fit timing	22772434	measured: 11:08, 53 MB (HPC chapter updated in #832)
#809 on the SCN reproducer	22772442, 22776712	verified: 0 truncated R2 windows; read count explained by the input VCF
realism panel	22783100	snapshot, not a comparison (below)

Realism snapshot (22783100): not comparable to 22583881

Settings:

HG002 hg002.bam rebuilt from one chunk (real depth 5.31x in the windows), GRCh38, READ_LEN=250;
fragments Normal(400, 90), the panel's override when no FRAGMENT_MODEL is set;
adapters off; 10 regions; defaults otherwise.

22583881 differs in all three that matter: real depth 20.59x (a deeper BAM, about 4 chunks), simulated insert SD 93.1 (the shipped empirical fragment model, so FRAGMENT_MODEL was set), and simulated clip 0.0262 (adapters on). Its settings were never archived and its log is gone, so it cannot be reproduced exactly.

metric	REAL	SIM	gap
clip_pct	0.0475	0.001	47.5x (adapters off)
improper_pct	0.0055	0	inf
mapq0_pct	0.0043	0	inf (#658)
depth_vmr	1.486	1.346	1.1x
depth_excess	0.0887	0.0692	1.3x
depth_acf	0.028	0.013	2.2x
ins_sd	87.7	78.9	1.1x
ins_skew	0.587	0.345	1.7x
ins_p99	657	605	1.1x

Not a regression signal either way. Variant-free read generation is unchanged in v4.0.0: same-seed v3.5.1 vs v4.0.0 gives identical reads apart from the NM tag. Harness fix: realism_panel will archive its effective settings so the next run can be compared (PR to follow).

Harness defects found and fixed

model_builders: refuse an input that is set but missing; record capped-fit timing #832: model_builders skipped builders silently for a set-but-missing input.
stage_hg002: stop a SIGPIPE in the contig check ending the job #833: stage_hg002 SIGPIPE exit after building the BAM.

Correction. Job 22583881's log was not lost. archive_run copied it: eidolon-access-results/realism/job_22583881/eidolon-realism_22583881.out, 47,687 bytes, alongside panel.tsv. Earlier comments here said its settings were unrecoverable and the run could not be reproduced; that was wrong. Its configuration block is being compared with 22783100's to confirm which settings differed.

Matched realism rerun submitted: 22789530, eidolon 4.0.0+b7b3742, with 22583881's settings as recorded in its archived log: REAL_BAM=/work/nvme/bhrd/jallen17/hg002/hg002.bam (21x), FRAGMENT_MODEL=$SCRATCH/hg002_defaults/frag_hg002.json.gz, ADAPTERS=truseq, READ_LEN=250, GRCh38, defaults otherwise. It replaces 22783100 (one-chunk BAM, Normal fragments, adapters off) as the comparison. Expectation: variant-free read generation is unchanged in v4.0.0, so metrics should match 22583881 within region and seed noise.

Harness fixes from the Delta validation are merged to develop: #832 (a6da2b93, model_builders refuses a set-but-missing input; HPC chapter timing), #833 (049aea81, stage_hg002 SIGPIPE), #835 (adc2f95c, archive_run finds job logs through scontrol), #836 (51f6be45, realism panel says what it simulates). #834 closed in favor of the archived logs. Remaining: matched realism rerun 22789530.

Matched realism rerun (22789530): no regression. Delta validation complete.

eidolon 4.0.0+b7b3742, with 22583881's settings: 21x /work/nvme BAM, fitted HG002 fragment model, truseq, READ_LEN=250. COMPLETED in 1:15:46.

Real side identical to 22583881 on every metric (cand_per_mb 127.5, depth 20.59, VMR 1.697, ins_sd 88.4, …): same BAM, same 10 regions, same settings.

sim metric	22583881	22789530
improper_pct	0.0004	0.0004
clip_pct	0.0262	0.026
mapq0_pct	0.0001 [0–0.206]	0.0003 [0–0.2047]
depth_mean	20.91	20.91
depth_vmr	1.403	1.369
depth_excess	0.0193	0.0177
depth_acf	0.012	0.021
ins_sd / skew / p99	93.1 / 0.233 / 648	93.1 / 0.231 / 650

All within window noise; mapq0_pct is at its floor in both. The simulated side is not bit-identical, as expected: the panel carries model SNPs and indels (#836), and the mutation model was reshipped (#770) and soft-mask handling changed (#811) between the two builds. The standing gaps (cand_per_mb, improper 14x, mapq0, ins_skew 2.5x, depth_acf 2.2x) are the subject of #837.

22789530 replaces 22583881 as the realism baseline (settings in its archived log). 22783100, the snapshot with different settings, is superseded.

Validation summary: all items checked on real data at genome scale. Remaining "not verified" items are listed above.

Navigation Menu

Repository navigation

Release v4.0.0 #816

Version

Per PR (merged to `develop` since v3.5.1)

Still in the milestone, not started

Release procedure

Not verified for this release

Delta validation of v4.0.0: complete

Realism snapshot (22783100): not comparable to 22583881

Harness defects found and fixed

Matched realism rerun (22789530): no regression. Delta validation complete.

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Release v4.0.0 #816

Description

Version

Per PR (merged to develop since v3.5.1)

Still in the milestone, not started

Release procedure

Not verified for this release

Activity

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 9, 2026

joshfactorial commented on Oct 10, 2026

joshfactorial commented on Oct 10, 2026

Delta validation of v4.0.0: complete

Realism snapshot (22783100): not comparable to 22583881

Harness defects found and fixed

joshfactorial commented on Oct 10, 2026

joshfactorial commented on Oct 10, 2026

joshfactorial commented on Oct 10, 2026

joshfactorial commented on Oct 10, 2026

Matched realism rerun (22789530): no regression. Delta validation complete.

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Per PR (merged to `develop` since v3.5.1)