Repository navigation
Release v4.0.1 - #855
Merged
Merged
Release v4.0.1#855
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each builder ran only when its input file existed, so an INPUT_* path pointing at a deleted file skipped that builder silently: job 22772429 skipped three BAM builders and reported only the round-trip failure that followed. Check every set input at startup and stop naming it. Also record the measured capped gen-seq-error-model timing in the HPC chapter (Delta, v4.0.0: 11 min for a 5 M-record cap over a 29 GB FASTQ). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The truth-VCF contig check ran zcat | head inside a command substitution under set -euo pipefail. head closes the pipe after one line, zcat takes SIGPIPE (141), and set -e ended the job silently after the BAM was already built and indexed (job 22776603). stage_hcc1395.sh hit the same line in job 19901611 and disables pipefail around both checks; do the same here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
archive_run guessed the job log as <jobname>_<jobid>.out in the current directory, so fit_hg002_defaults, fit_overlap_matrix and stage_raw_pairs, whose #SBATCH --output names it differently, never had their logs archived, silently. The log is where every script prints its configuration. Ask scontrol for StdOut/StdErr, keep the guess as the fallback, and warn when neither finds it. New test_archive_run.sh fails 5 of 10 against the old function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The panel's banner and comment said it simulates 'with no variants'. Its config leaves the mutation model at default, so the simulated genome has germline SNPs and small indels at the model's rate and positions; only SVs are off. candidates_vs_truth.sh repeated the claim as 'simulated reads cannot' disagree with the reference at small indels. Both now state what is simulated and what differs from the real donor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v4.0.0 follow-ups: conda sha256, complete CHANGELOG Not verified list
model_builders: refuse an input that is set but missing; record capped-fit timing
stage_hg002: stop a SIGPIPE in the contig check ending the job
archive_run: find the job log through scontrol, warn when it is missing
realism panel: say what the simulated genome carries
The indel pass counted error indels with variants separated but never counted substitutions, so indel_probability had no denominator. Cache SEQ, find mismatches by walking each CIGAR against the region reference (indel_context_subs.awk), and class both kinds of position by the same support rule. indel_probability.awk reports error events, the observed indel share, the shipped curve's background mean, and the fit share / mean, since the model scales indel_probability by that curve. A zero count is fatal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
indel_context: measure indel_probability in the same pass (#746)
HG002 at 21x (job 22794642) gave 0.0092 counting only low-support error positions and 0.0104 counting the ambiguous positions too, which are almost all errors at low local depth (116 of 29,380 are HG002 SNVs). The inherited 0.01 sits inside that range and is kept. model_data/README.md and the default-model comment now say so, and drop an unsourced 1e-5 per-base figure the same run contradicts (3.8e-5). indel_probability.awk now counts ambiguous events and reports the fit with them, so the next run shows the range without a hand calculation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page said the bundled model's het/hom is ~2.0 (homozygous_frequency 0.333). v3.5.0 refit it from HG002 to 0.388, so ~1.6. The speed and memory figures (v1.19.1) and the germline fidelity comparison predate the recalibration; they now say so and point at their re-run (#840). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
indel_probability: record it as measured; report both fits (#746)
Comparison page: current het/hom default, and date the stale comparisons (#840)
The config loaders for gen-reads, gen-mut-model and filter-reads, and gen_reads/mod.rs, panicked on user input: an unreadable or invalid config, a missing key, a missing file, an unknown extension or key, an existing output. Those panics predate the per-subcommand error types and were never converted; #807 and #827 extended them. eidolon-core's create_output_file, check_parent and check_create_dir panicked too, so an existing output panicked in every subcommand. All now return the subcommand's error with a message naming the key or path. gen-reads' ConfigError carries its reason instead of logging it and returning 'Error generating configuration!'. main prints the error's message rather than its Debug form and exits 1, and a missing config file names its own subcommand instead of reporting a filter-reads error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ign (#840) baseline_metrics.tsv: ecoli and chr22 fidelity rows and the adapter rows take the campaign's values. The ecoli_tstv band moves 2.0:2.6 -> 2.8:3.5, centered on what the HG002 context model predicts from E. coli's composition (3.05; observed 3.183). Perf, somatic SNV, and somatic indel rows are not re-frozen; somatic indel (16/23 on one draw) waits on replicates. ACCESS §3.1/3.2/3.3/3.8/3.11, the NEAT comparison page, and the perf and variety charts carry the v4.0.0 numbers, with the Phase 1 values stated alongside where they moved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bad configs return errors instead of panicking
The previous text said the memory growth was accounted for; only the +22 MB quality tables are. Replace it with local measurements: per-read cost unchanged, the gap is per-run startup, most of it parsing the 20.8 MB default sequencing error model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serde_json::from_reader over a GzDecoder parses through io::Read a byte at a time. On the 20.8 MB default sequencing error model that took ~720 ms of every gen-reads run. Decompressing into memory and calling from_slice takes ~70 ms. Every model load now goes through one helper, from_gzip_json: the embedded defaults and model_reader for user files. model_reader and model_unzipped_reader return an error naming the file when it cannot be opened, instead of panicking. Output is byte-identical to develop at the same seed (H1N1 and E. coli: FASTQ, VCF body, BAM records). Co-Authored-By: Claude <noreply@anthropic.com>
Re-freeze fidelity baselines and refresh ACCESS from the v4.0.0 campaign (#840)
Load models by decompressing first, then parsing (#845)
The README, installation page, comparison page and ONBOARDING advertised `conda install -c bioconda eidolon`, which finds nothing: the add-recipe PR (bioconda-recipes#67420) has been awaiting review since July, and Bioconda carries only `rneat` 1.20.1. Install instructions now lead with the release binaries and say the Bioconda package is pending. The release skill gains a Bioconda check after the sha256 step, and the report step names where Bioconda stands. Updating the in-repo recipe publishes nothing, which is how this went unnoticed. Co-Authored-By: Claude <noreply@anthropic.com>
Adds Stephens et al. (2016, PLOS ONE) and Allen et al. (2026, JOSS) as references, as listed on the NEAT README. The README and comparison page already cite both; CITATION.cff, which GitHub's "Cite this repository" and Zenodo read, cited neither. The version had stayed at 3.1.0 because the release skill's version-bump step did not list the file. It does now. Co-Authored-By: Claude <noreply@anthropic.com>
…847) Six runs pool to 121/163 = 0.742 (CI 0.67-0.80), excluding the 0.845 baseline. Counting reads at each planted indel in a kept BAM shows eidolon put them there: found indels carry 4-13 supporting reads, and 8 of the 10 missed carry as many. Ruled out: germline proximity, homopolymer context, tumor-model indel lengths. Co-Authored-By: Claude <noreply@anthropic.com>
Docs: eidolon is not on Bioconda yet; point installs at release binaries
CITATION.cff: cite both NEAT papers; version 4.0.0
Re-freeze somatic_indel_recall at 0.742: the misses are the caller's (#847)
Records the post-fix numbers: startup 0.09 s locally, E. coli at 10x level with v1.19.1, and chr22 on Delta 67.44 s (12.0x NEAT), quoting only chr22 from job 22985666 because of a possible filesystem incident during it. The remaining chr22 gap is #848. Co-Authored-By: Claude <noreply@anthropic.com>
Release prep: v4.0.1
ACCESS §3.2: the startup cost was the reader, fixed in v4.0.1 (#846)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release ticket: #852. PATCH: output, config keys and model formats are unchanged.
For users
Also on
developsince v4.0.0: harness fixes, the v4.0.0 baseline re-freeze and ACCESS refresh (#840, #847),indel_probabilitymeasured (#746), and docs (#849: Bioconda not yet available; #850:CITATION.cff).Checks
cargo test --workspace(1,117 passed),clippy -D warningsclean, release shakeout PASS (SKIP_LARGE_DATA=1), junk-file check clean.release-blockersandrelease-gatesrun on this PR.🤖 Generated with Claude Code