diff --git a/CHANGELOG.md b/CHANGELOG.md index f6af710ec8..a637e48bb5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +- 2026-09-02: Emitted the `sampling_factor` replicas of `flexref`, `emref` and `mdref` in rounds - every model is refined once before any model is refined a second time - so raising `sampling_factor` appends model numbers instead of renumbering the models of every input after the first. +- 2026-09-01: Fixed a pre-existing parallel scheduler ordering defect exposed by reproducible CNS job identity work; task results now retain submission order rather than completion order. +- 2026-09-01: Removed run-volatile CNS provenance headers (`REMARK FILENAME=`, `REMARK DATE:`, `REMARK HADDOCK stats for`, and `REMARK initial structure`) from published PDB files, and volatile filename/date title lines from PSF files. +- 2026-09-01: Preserved all CNS parameter families assembled through symbol splicing, including interaction matrices, random AIRs, flexible segments, symmetry, and NCS parameters. +- 2026-09-01: Restored per-model `tolerance` handling for concatenated batch CNS jobs and made CNS stderr, stdout error markers, and non-zero exit statuses fail explicitly. +- 2026-09-01: Published CNS PDB/PSF outputs only after complete-set validation and normalization, using same-filesystem atomic replacement on local, batch, and grid paths. +- 2026-09-01: Made coarse-grained topology generation reproducible from `iniseed`. +- 2026-09-01: Made rigid-body sampling prefix-stable: it now schedules exactly `sampling` jobs round-robin across model combinations, so raising `sampling` appends jobs instead of renumbering them. +- 2026-09-01: Derived every CNS random seed from the job itself - from `iniseed`, from the content of the models the job starts from, and from which repeat of that job it is - rather than from its position in the schedule or from process-wide random-number draw order. `iniseed` keeps its meaning: changing it changes every seed in the run. Results change for every run of `rigidbody`, `flexref`, `emref`, `mdref` and `mdscoring`, and `sampling_factor` now buys additional sampling rather than duplicate models. - 2026-08-02: Fixed logging/warning leaks - Issue #1647 - 2026-07-31: Fixed D-amino acid detection - Issue #1636 - 2026-07-31: Fixed topocg issue removing ligands - Issue #1638 diff --git a/caching-publication-plan.md b/caching-publication-plan.md new file mode 100644 index 0000000000..9d8c7a88ab --- /dev/null +++ b/caching-publication-plan.md @@ -0,0 +1,195 @@ +# Staged publication of the CNS caching feature + +## Standing of this document + +This note records **how** the CNS caching feature is being published and +reviewed, not what it does or how it works. It sits alongside +`caching-use-cases.md` (what must hit, what must miss, and why), +`caching-test-suite-plan.md` (how those verdicts are observed), and +`canonical-representation-consumption.md` (why the canonical form is not +executed in production). + +## Why the work is staged + +Caching is a **user-facing feature**, and the work behind it touches three +concerns that are best judged separately, by different people, against +different criteria. + +**Usability.** Does the feature do what a HADDOCK user expects and wants — is +the reuse it offers the reuse they would ask for, and is it offered in a form +they can actually use? This is a domain judgement, made against behaviour, and +it requires no reading of the implementation. It is settled in Stage 2, where +the behaviour is written down as a readable contract, and confirmed in Stage 3, +where users exercise the feature as a black box. + +**Code quality.** Is the implementation sound, maintainable, and a reasonable +thing to carry in HADDOCK3? This is a core developer judgement, made against +the diff. It is the content of Stage 3. + +**Reproducibility.** Content-based caching rests on it entirely: reuse is only +meaningful if the same job, run again, produces the same result, and only safe +if a job's identity accounts for everything that can change that result. +Reproducibility is addressed twice — in Stage 1 independently of caching, as +fixes and a canonicalization library that stand on their own merits, and in +Stage 4 together with caching, where a job can be dumped and re-executed from +its declared dependencies alone. + +Keeping these apart is what the staging buys. A user asked to review a diff +would be reviewing the wrong artifact; a core developer asked whether reusing a +refinement result across a changed cluster rank is scientifically acceptable +would be answering the wrong question. Reproducibility, meanwhile, is worth +having whether or not caching is ever merged, so it is not made to wait on a +review of caching. + +The stages also order the reviews so that the promised behaviour is fixed +**before** the implementation that has to satisfy it exists. + +## The stages + +Each stage is the previous stage's branch with further work applied on top, so +the stages are cumulative branches rather than four independent pull requests. + +| Stage | Adds | Concern | Reviewed by | +|---|---|---|---| +| 1 | Reproducibility fixes + canonicalization library | Reproducibility, independent of caching | Core devs | +| 2 | The caching test suite | Usability, as a written contract | HADDOCK users | +| 3 | The caching implementation | Usability in practice, then code quality | Users, then core devs | +| 4 | `seamless-run` job dumps | Reproducibility, together with caching | Core devs | + +### Stage 1 — reproducibility and the canonicalization library + +Branch: `bitwise-reproducibility`. + +Two kinds of work, deliberately kept in separate commits: + +- **General reproducibility fixes**, ten commits, each standing on its own: + parallel scheduler results returned in submission order; deterministic + coarse-grained topology generation and deterministic CNS seeding; a + prefix-stable rigid-body sampling schedule; generated CNS input restricted to + the parameters the recipes can actually read; run-volatile provenance + normalized out of published PDB and PSF artifacts; complete output sets + published atomically on the local, batch and grid paths; and three unrelated + pre-existing defects corrected along the way (coarse-grain back-mapping input + alignment, the unhonoured `sampling_factor` clamp, discarded grid worker + exceptions). These contain no caching, and each is justifiable to a reviewer + who never hears the word. +- **A canonicalization library**, two commits. It derives a stable canonical + representation of a CNS transformation — the script with locators erased, plus + the set of inputs bound to canonical pin names — from which a job identity can + be computed, together with a committed golden canonical form for each of the + nine CNS job shapes. + +The library is **virtual** at this stage: nothing consumes its output. CNS +continues to execute the existing generated input through the existing +scheduler path, and the canonical form exists only to be checksummed. Making +CNS execute the canonical form directly was investigated and rejected; see +`canonical-representation-consumption.md` for the operational reasons. + +Reviewed by core devs, on reproducibility grounds alone. The bar is that Stage 1 +stands up **even if caching is never merged**: bitwise-reproducible CNS results +are worth having in their own right, and the canonicalization library is a +self-contained component. Stage 1 is not "caching, part one". + +### Stage 2 — the behavioural contract + +Stage 1 plus the caching test suite at `end-to-end_tests/caching/`. + +The suite is nothing but ordinary invocations of +`haddock3 config.cfg --cache OLD-RUN-DIR`. Each case declares, per output file, +which earlier run's file it must be reused from, or that it must be recomputed. +There are no mocks, no test-only entry points, and no inspection of internals. + +That constraint is what makes the user review possible. A case is legible to +someone who runs dockings and has never opened the source: it names a config +change, an earlier run, and the files that must or must not be recomputed as a +result. + +Reviewed by **HADDOCK users, not core devs**. The questions put to them are +answerable from domain knowledge alone: + +- Is anything declared a required hit that they would regard as a *different* + calculation? +- Is anything declared a required miss that they would expect to be reused — + and would therefore be surprised to pay for again? +- Is the feature offered in a shape they can use: opting in per run, naming + earlier run directories directly, drawing on several of them at once? + +The suite does not pass at Stage 2. There is nothing yet that could make it +pass. That is the point: freezing the expectations before the implementation +exists is what prevents the suite from being quietly written to describe +whatever the implementation happens to do. + +Stage 2 exits when users agree the declared behaviour is the behaviour they +want. + +### Stage 3 — the implementation + +Stage 2 plus the caching implementation. Cache-key construction becomes the +canonicalizer's first production caller: the canonical mapping is computed +before cache lookup, a verified result is reused on a hit, and the existing CNS +path executes on a miss. + +Stage 3 is reviewed twice, in order: + +1. **The Stage 2 reviewers check it out and run it.** The test suite gives them + a mechanical answer; their own configurations and molecular systems give + them the answer that actually matters. This is the confirmation that the + contract they signed off in Stage 2 has been met. +2. **On their green light, core devs review Stage 3 as code.** + +The order is deliberate. Core developer review is the expensive, scarce +resource, and it is spent once the behaviour is settled rather than twice +across a moving specification. + +### Stage 4 — dump to a workdir and run with `seamless-run` + +The ability to dump a single CNS job to a working directory as a +self-contained `seamless-run` command. + +Stage 4 returns to the reproducibility concern, this time bound to caching. A +dumped job is reproducible in the strongest available sense — re-executable +anywhere from its declared dependencies alone — and that same property is what +makes it a test of job identity. + +Its primary purpose is therefore to test the Stage 1 canonicalization library +rigorously, which Stages 2 and 3 structurally cannot do. Their suite observes +the cache from outside: it can see whether a file was reused, never the key +that decided it. Two jobs that ought to differ but receive the same key are +invisible to it until some case happens to catch the resulting wrong hit. + +A dump closes that gap, because it is self-contained by construction — it +carries precisely the declared dependencies and nothing else: + +- running one in an isolated working directory **is** the strong test for an + under-declared dependency set: an undeclared file is simply not there, so a + silent key defect becomes a loud execution failure; +- the key and the pin table become directly inspectable, so canonical forms can + be diffed and frozen against golden forms, with no CNS execution required; +- two dumps of one job that produce different results make a cache conflict + *classifiable* — nondeterministic artifact versus incomplete key — rather + than merely observable. + +Stage 4 is sequenced last because it is an assurance instrument, not a +prerequisite: Stage 1 is reviewed on its own merits and already retains an +executable canonical-workspace probe for representative jobs, which Stage 4 +generalizes into a suite. It does not change how HADDOCK executes CNS. +Executable canonical workspaces remain a test and audit instrument, never the +production execution path. + +## What this staging is not + +- **Not a schedule.** The stages are a review order, not a set of deadlines. +- **Not four independent pull requests.** Each stage is the previous branch plus + work on top and is reviewed as a whole. +- **Not two reviews of the same thing.** The user review in Stages 2 and 3 + cannot substitute for core developer code review, and code review cannot + substitute for it. + +## Numbering note + +`canonical-representation-consumption.md` was written under an earlier +numbering in which Stage 2 was the caching implementation. Under the scheme in +this document, that is Stage 3. Where that document says "the caching +implementation in Stage 2" or "Stage 2 must append a successful cache record +only after the complete declared output set has been published", read +**Stage 3**. diff --git a/canonical-representation-consumption.md b/canonical-representation-consumption.md new file mode 100644 index 0000000000..477bf71aca --- /dev/null +++ b/canonical-representation-consumption.md @@ -0,0 +1,293 @@ +# Consumption of the canonical CNS representation + +## Standing of this document + +This note records the investigation into whether CNS should execute the same +canonical representation that will be used to calculate CNS-job identities. +It belongs to the staged caching design discussion and captures a rejected +production architecture, the reasons for rejecting it, and a smaller related +change that remains desirable. + +The conclusion is: + +- the canonical representation should remain a **checksum-side identity + representation**; +- normal CNS execution should continue to use the existing run and scheduler + architecture; +- the caching implementation in Stage 2 should become the canonicalizer's + production caller when it calculates a job key before cache lookup; +- executable canonical workspaces remain valuable as a test and audit + instrument, but should not become the normal execution path; and +- incomplete CNS output publication should be solved independently using + hidden partial output files and atomic rename. + +## The question + +The Stage 1 branch currently exposes canonicalization through +`CNSJob.canonical_mapping()`. Normal runs do not call it. `CNSJob.run()` sends +the original generated input to CNS, using the real input and output filenames, +and only normalizes selected output artifacts afterward. + +This led to a reasonable question: instead of maintaining a canonical form only +for job identity, should CNS itself execute with canonical input and output +filenames? + +Doing so would make the relationship between execution and identity especially +direct. The canonical script and canonical input bindings used to construct the +key would also be the script and bindings observed by CNS. + +## Initially considered production architecture + +The investigated design gave every CNS job a private execution directory. A job +would: + +1. build its canonical mapping; +2. create a private job directory; +3. materialize job-specific dependencies under canonical names; +4. execute a canonical CNS input producing `canonical-output.pdb` and, + optionally, `canonical-output.psf`; +5. validate and normalize the outputs; and +6. publish them under HADDOCK's user-facing filenames. + +Only job-specific files need to be materialized. The CNS executable and the +installed module and toppar trees do not need to be copied for ordinary local +execution. Stable `MODULE:relative/path` and `TOPPAR:relative/path` references +can continue to resolve through the execution environment. + +This distinction is important. A dependency has at least two relevant names: + +- its canonical pin name, used in transformation identity; and +- the stable reference spelling CNS receives, which may be a `MODULE:` or + `TOPPAR:` reference rather than a workspace path. + +Conflating these names is one cause of the current `MODULE:module/...` rewrite +defect. + +## Empirical feasibility + +The basic execution model was tested with the real CNS executable and generated +HADDOCK inputs. + +### Rigid-body job + +A rigid-body job was run with only its two input PDB files and two input PSF +files materialized under canonical names. CNS, the module scripts, and toppar +remained in their installed locations. + +The job completed successfully. After output normalization, its PDB was +byte-identical to the output of the original job. + +### Topology job + +A `topoaa` job was run with only its input PDB materialized under a canonical +name. Its canonical PDB and PSF outputs both completed successfully and, after +normalization, were byte-identical to the original outputs. + +The probe also demonstrated that an executable canonical form would immediately +expose several current defects that a checksum-only form can hide: + +- rewriting an input alias must not rewrite the CNS variable name on the + assignment's left-hand side; +- `MODULE:relative` and `TOPPAR:relative` execution references must not acquire + an extra `module/` or `toppar/` prefix; +- the current textual `canonical-count` replacement is not executable CNS; and +- output bindings must not be consumed by an input basename collision. + +These findings establish that direct canonical execution is technically +possible after the known canonicalization defects are fixed. + +## Why technical feasibility is not sufficient + +HADDOCK commonly runs many jobs concurrently on HPC systems backed by a network +filesystem. A production design must account for filesystem metadata operations, +data movement, scheduler topology, local scratch lifetime, and many simultaneous +HADDOCK runs. + +### Step-local workspaces + +Putting one workspace per job in the step directory permits hardlinking inputs +and same-filesystem atomic rename of outputs. It also adds substantial metadata +traffic to the shared filesystem. + +For 10,000 rigid-body jobs with two input molecules, a naive implementation can +add approximately: + +- 10,000 workspace directories; and +- 40,000 input hardlinks for the two PDB/PSF pairs. + +Creating and removing these entries is precisely the kind of workload that can +stress NFS, Lustre, GPFS, and similar metadata services, especially when several +HADDOCK runs do it concurrently. + +This is a material regression from the current non-debug local path, where CNS +inputs may remain in memory and no per-job input tree is created. + +### Node-local workspaces + +Node-local scratch avoids shared-filesystem metadata pressure and makes CNS +working I/O local. However, it prevents hardlinking directly from network-hosted +inputs. A naive implementation would copy every input for every job. + +That could be mitigated with a node-local content pool: + +```text +shared filesystem -> one local copy per unique input per node + | + +-> hardlinks into private job directories +``` + +Checksumming, decompression, and local materialization would also need to be +fused so an input is not read from the network once for its checksum and again +for staging. Static module/toppar dependency scans and checksums would need +caching rather than being repeated for every job. + +These optimizations are feasible, but together they constitute a new execution +and data-staging subsystem. They require decisions about: + +- scheduler-provided scratch such as `SLURM_TMPDIR` and `TMPDIR`; +- scratch capacity and exhaustion; +- allocation- or node-scoped content pools; +- concurrent population and cleanup; +- cross-filesystem output publication; +- batch-job preparation on the submission host versus the compute node; +- MPI processes distributed across nodes; +- debug artifact retention; and +- grid payload construction, where invariant files really do need transport. + +This is far beyond the intended responsibility of Stage 1. + +### Scheduler integration + +Local and MPI modes eventually call `CNSJob.run()`, so they could share a +central canonical executor. Batch mode bypasses that method and generates shell +commands that invoke CNS directly. It would require a preparation and +finalization protocol, including per-job status and output handling for +concatenated batch jobs. + +Grid mode already creates remote workspaces, but it has its own input rewrite, +payload, and output-retrieval implementation. Converting it to the same model +would be another substantial refactor. Unlike ordinary local execution, a grid +payload must transport the executable and invariant dependencies because the +remote site cannot be assumed to contain the HADDOCK installation. + +Thus direct consumption of the canonical form would not be a contained change +to `CNSJob.run()`. It would alter every CNS execution backend or make execution +semantics mode-dependent. + +## Decision: do not execute the canonical representation in production + +The proposed execution architecture is rejected for the staged caching work. +It is technically workable, but its operational cost and scope are not +proportionate to the identity problem it solves. + +The intended separation is instead: + +### Stage 1 + +- Produce a stable canonical representation of each CNS transformation. +- Normalize result artifacts so irrelevant output locators and volatile headers + do not affect result identity. +- Test canonicalization against real generated CNS inputs and golden canonical + forms. +- Keep an executable canonical-workspace synthesizer or equivalent companion + test for representative jobs. This proves that the declared dependencies are + complete and that the canonical form remains meaningful. +- Do not alter normal CNS input materialization or scheduler execution merely to + make CNS observe the canonical pin names. + +### Stage 2 + +- Make cache-key construction the canonicalizer's production caller. +- Calculate the canonical mapping before cache lookup. +- Reuse a verified result on a hit. +- Execute the existing CNS path on a miss. + +The executed input and the checksum-side representation will therefore not be +byte-identical objects. Their equivalence is a contract maintained by focused +canonicalization tests, golden forms, adjacent MUST-MISS tests, and the +executable isolated-workspace companion suite. + +This is the same deliberate separation already described in the caching design: +canonicalization erases irrelevant locators for identity without requiring a +change to the run-directory layout or normal CNS execution. + +## Independent issue: incomplete CNS output publication + +The workspace investigation highlighted a separate correctness problem that +does not require canonical input staging. + +At present, CNS writes directly to the final HADDOCK output filename. If CNS or +its worker dies while writing, a truncated PDB can remain at that path. +`Persistent.is_present()` checks only whether the path exists, so later module +logic can treat that truncated file as a generated result. In addition, +`CNSJob.run()` currently does not use the subprocess return code as a success +condition. + +This should be corrected independently. + +### Proposed output-publication protocol + +For each declared output, CNS writes to a hidden partial filename in the existing +step directory, retaining the logical suffix, for example: + +```text +.rigidbody_1.partial.pdb +.molecule_haddock.partial.psf +``` + +Keeping `.pdb` or `.psf` as the final suffix matters because CNS recipes derive +auxiliary names from those suffixes. + +After CNS terminates, the job must require all of the following before publishing +anything: + +1. the subprocess exit status is acceptable; +2. no known CNS failure is present in its output; +3. every declared output exists; and +4. every declared output is non-empty. + +The job then normalizes the partial PDB/PSF and publishes it with +same-filesystem `os.replace()` to the expected HADDOCK filename. + +If CNS is interrupted while writing, only the hidden partial file is affected. +The public output remains absent. A subsequent attempt removes stale partial +files and treats the declared output set as a unit. + +PDB and PSF cannot be made filesystem-atomic as a pair while retaining the flat +step-directory layout. They can nevertheless be validated as a pair before +publication, published only during job finalization, and cleared together before +a retry. Downstream modules do not start until the producing module finishes. +Stage 2 must append a successful cache record only after the complete declared +output set has been published. + +### Filesystem impact + +This targeted protocol does not create per-job directories, stage inputs, or +copy large data trees. Compared with current execution, it adds temporary output +names and one metadata rename per successfully published artifact. CNS still +writes the same amount of output data to the same filesystem. + +Grid retrieval should use the analogous cross-filesystem-safe form: copy the +retrieved artifact to a hidden temporary file in the destination step directory, +verify and normalize it there, and then use `os.replace()`. An interrupted copy +cannot expose a truncated public output. + +## Commit boundaries + +The following concerns should remain separate: + +1. Scheduler results returned in submission order. This is a pre-existing + Scheduler correction whose importance was exposed by output normalization. +2. Canonicalization correctness: context-aware path rewriting, stable + `MODULE:`/`TOPPAR:` references, output-binding validation, count handling, + archive restraints, and indexed `cgtoaa` references. +3. Atomic CNS output publication through hidden partial outputs. +4. Coarse-grained random-number determinism. +5. CNS parameter inclusion and exclusion, including removal of `tolerance` and + the recorded treatment of `log_level`. +6. Real generated-input golden canonical forms. +7. Stage 2 cache lookup and recording, which supplies the canonicalizer's normal + production caller. + +Direct production execution of the canonical representation should not be one +of these commits. It is intentionally not part of the staged caching feature. diff --git a/docs/pages/architecture.md b/docs/pages/architecture.md index 5562dda3d5..2d81448a04 100644 --- a/docs/pages/architecture.md +++ b/docs/pages/architecture.md @@ -183,7 +183,7 @@ Stateless or near-stateless helpers used across modules and gears. | [libmpi.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libmpi.py) | `MPIScheduler` — MPI execution | | [libgrid.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libgrid.py) | `GRIDScheduler` — DIRAC grid execution | | [libpdb.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libpdb.py), [libstructure.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libstructure.py) | Parse and manipulate PDB structures | -| [libalign.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libalign.py), [libmath.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libmath.py) | Alignment and RMSD/geometry maths | +| [libalign.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libalign.py) | Alignment and RMSD/geometry maths | | [libclust.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libclust.py), [libfcc.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libfcc.py) | Clustering helpers | | [librestraints.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/librestraints.py) | Restraint (`.tbl`) handling | | [libaa2cg.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libaa2cg.py), [libligand.py](https://github.com/haddocking/haddock3/blob/main/src/haddock/libs/libligand.py) | Coarse-grain mapping, ligand topology | diff --git a/docs/pages/intro.md b/docs/pages/intro.md index 69198d98c9..138a52a502 100644 --- a/docs/pages/intro.md +++ b/docs/pages/intro.md @@ -98,6 +98,12 @@ ambig_fname = "data/e2a-hpr_air.tbl" In this example only very few parameters are defined as most correspond to the default ones defined for each module. The sampling in this case would be 1000 rigidbody models and 200 for the refinement stages. Clustering is based on the fraction of common contacts and the final clusters are analysed using the best model generated as a reference (the `caprieval` module). +For `rigidbody`, `sampling` is the exact number of jobs. When several input-model combinations are available, jobs are assigned round-robin across them. + +Every CNS random seed is derived from the job itself: from `iniseed`, from the content of the models the job starts from, and from which repeat of that job it is. A job therefore keeps its seed when the run around it grows, shrinks or is reordered, and changing `iniseed` still changes every seed in the run. + +For `flexref`, `emref` and `mdref`, the `sampling_factor` replicas of the input models are emitted in rounds: every model is refined once before any model is refined a second time, so raising `sampling_factor` appends model numbers rather than renumbering them. Each replica carries its own seed and is genuinely additional sampling. + Detailed explanations on how to configure a workflow through the configuration files can be found here. diff --git a/docs/titles.yaml b/docs/titles.yaml index 061e01b0dc..fc531d0e0a 100644 --- a/docs/titles.yaml +++ b/docs/titles.yaml @@ -73,7 +73,6 @@ reference: libs.libinteractive: "libinteractive: functions related to interactive tasks" libs.libio: "libio: I/O helping functions" libs.liblog: "liblog: Logging helping functions" - libs.libmath: "libmath: Math helping functions" libs.libmpi: "libmpi: MPI execution functions" libs.libontology: "libontology: module communication" libs.libparallel: "libparallel: multiprocessing helping functions" diff --git a/integration_tests/test_cnsjob.py b/integration_tests/test_cnsjob.py index fc227f388e..0c098c49b2 100644 --- a/integration_tests/test_cnsjob.py +++ b/integration_tests/test_cnsjob.py @@ -6,6 +6,7 @@ from pathlib import Path from typing import Generator +from haddock.core.exceptions import CNSRunningError from haddock.gear.known_cns_errors import KNOWN_ERRORS from haddock.libs.libsubprocess import CNSJob @@ -75,7 +76,7 @@ def cns_inp_str(cns_seed_filename, cns_output_pdb_filename): write coordinates format=pdbo output={cns_output_pdb_filename} end set display={cns_seed_filename} end -display evaluate($seed=42) +display 42 close {cns_seed_filename} end stop""" @@ -143,12 +144,13 @@ def test_cnsjob_run_uncompressed_err( "haddock.libs.libsubprocess.subprocess.Popen.communicate", return_value=(bytes(random_error, encoding="utf-8"), b""), ) - cnsjob.run( - compress_inp=False, - compress_out=False, - compress_err=False, - compress_seed=False, - ) + with pytest.raises(CNSRunningError): + cnsjob.run( + compress_inp=False, + compress_out=False, + compress_err=False, + compress_seed=False, + ) # Check that error file was created assert Path(f"{cns_error_filename}").exists() assert Path(f"{cns_error_filename}").stat().st_size > 0 @@ -166,12 +168,13 @@ def test_cnsjob_run_compress_err( "haddock.libs.libsubprocess.subprocess.Popen.communicate", return_value=(bytes(random_error, encoding="utf-8"), b""), ) - cnsjob.run( - compress_inp=False, - compress_out=False, - compress_err=True, - compress_seed=False, - ) + with pytest.raises(CNSRunningError): + cnsjob.run( + compress_inp=False, + compress_out=False, + compress_err=True, + compress_seed=False, + ) # Check that error file was created and compressed ! assert Path(f"{cns_error_filename}.gz").exists() assert Path(f"{cns_error_filename}.gz").stat().st_size > 0 diff --git a/integration_tests/test_mdref.py b/integration_tests/test_mdref.py index 1bd4fa6705..1156e0c569 100644 --- a/integration_tests/test_mdref.py +++ b/integration_tests/test_mdref.py @@ -132,4 +132,8 @@ def test_mdref_mutliple_fle(mdref_module, calc_fnat): native=Path(GOLDEN_DATA, "2oob.pdb"), ) - assert fnat == pytest.approx(0.8, abs=0.1) + # The same band as the two tests above. It was centred lower when the + # seed a job received depended on where it sat in the schedule; with + # the seed derived from the model being refined, this run lands at the + # same quality as its siblings. + assert fnat == pytest.approx(0.9, abs=0.1) diff --git a/src/haddock/libs/libaa2cg.py b/src/haddock/libs/libaa2cg.py index 70893666de..cbc8f01819 100644 --- a/src/haddock/libs/libaa2cg.py +++ b/src/haddock/libs/libaa2cg.py @@ -373,7 +373,7 @@ def ss_classification(ss, program="dssp"): charged = ["ARG", "LYS", "ASP", "GLU"] -def add_dummy(bead_list, dist=0.11, n=2): +def add_dummy(bead_list, dist=0.11, n=2, rng=None): """ Args: @@ -387,11 +387,8 @@ def add_dummy(bead_list, dist=0.11, n=2): new_bead_dic = {} # Generate a random vector in a sphere of -1 to +1, to add to the bead position - v = [ - random.random() * 2.0 - 1, - random.random() * 2.0 - 1, - random.random() * 2.0 - 1, - ] + rng = rng or random.Random() + v = [rng.random() * 2.0 - 1 for _ in range(3)] # Calculated the length of the vector and divide by the final distance of the dummy bead norm_v = norm(v) / dist @@ -409,7 +406,7 @@ def add_dummy(bead_list, dist=0.11, n=2): return new_bead_dic -def map_cg(chain): +def map_cg(chain, rng=None): """ Args: @@ -515,7 +512,7 @@ def map_cg(chain): # add to data structure # this special beads have no HADDOCK code bead_list = [(b, m_dic[r][b][0]) for b in m_dic[r]] - dummy_bead_dic = add_dummy(bead_list, dist=d, n=n) + dummy_bead_dic = add_dummy(bead_list, dist=d, n=n, rng=rng) for db in dummy_bead_dic: db_coords = dummy_bead_dic[db] # code should be the same as the residue @@ -930,6 +927,7 @@ def martinize( input_pdb: str, output_path: str, skipss: bool, + seed: Optional[int] = None, ) -> tuple[str, bool]: """ Converts an all-atom (AA) PDB structure into a coarse-grained (CG) model @@ -960,6 +958,7 @@ def martinize( raise ModuleError(emsg) p = PDBParser() + rng = random.Random(seed) io = PDBIO() # Parse PDB and run DSSP @@ -1003,7 +1002,7 @@ def martinize( structure_builder.init_chain(chain.id) structure_builder.init_seg(chain.id) - mapping_dic = map_cg(chain) + mapping_dic = map_cg(chain, rng=rng) for residue in mapping_dic: if residue.id[0] != " ": # filter HETATMS diff --git a/src/haddock/libs/libcns.py b/src/haddock/libs/libcns.py index f76a0d5395..08e66a696c 100644 --- a/src/haddock/libs/libcns.py +++ b/src/haddock/libs/libcns.py @@ -1,23 +1,141 @@ """CNS scripts util functions.""" +import gzip +import hashlib import itertools import math +import re from functools import partial from os import linesep from pathlib import Path from haddock import EmptyPath, log from haddock.core import cns_paths -from haddock.core.typing import Any, FilePath, FilePathT, Optional, Union +from haddock.core.typing import Any, FilePath, FilePathT, Optional, Sequence, Union from haddock.libs import libpdb from haddock.libs.libfunc import false, true -from haddock.libs.libmath import RandomNumberGenerator -from haddock.libs.libontology import PDBFile +from haddock.libs.libontology import PDBFile, Persistent from haddock.libs.libpdb import check_combination_chains from haddock.libs.libutil import transform_to_list -RND = RandomNumberGenerator() +#: Ceiling for a derived CNS seed. +#: +#: CNS holds numbers as double-precision floats, so an integer above 2**53 +#: stops being exactly representable by the time a recipe reads it. +#: ``iniseed`` accepts values past that point, which is a separate matter -- +#: it is the user's number and is used as given -- but a *derived* seed is +#: ours to choose, and 2**31 leaves four orders of magnitude of margin while +#: staying inside the range CNS has always been handed. +SEED_CEILING = 2**31 + +#: Memo for :func:`content_checksum`, keyed by path, size and mtime. +_CONTENT_CHECKSUMS: dict[tuple[str, int, int], str] = {} + + +def content_checksum(path: FilePath) -> str: + """Checksum a file by its content, transparently to gzip storage. + + Memoized for the life of the process. A run hashes the same topology + once per sampling job that docks it, and the files concerned are written + once and then only read; the memo is keyed on size and modification time + as well as on the path, so a file that is rewritten in place is hashed + again rather than remembered wrongly. + """ + resolved = Path(path).resolve() + stat = resolved.stat() + key = (str(resolved), stat.st_size, stat.st_mtime_ns) + checksum = _CONTENT_CHECKSUMS.get(key) + if checksum is None: + data = resolved.read_bytes() + if resolved.name.endswith(".gz"): + data = gzip.decompress(data) + checksum = hashlib.sha256(data).hexdigest() + _CONTENT_CHECKSUMS[key] = checksum + return checksum + + +def model_path(model: Persistent) -> Path: + """Where a model's bytes can be read from, now. + + A model records both an absolute location and a step-relative one, and + the absolute one goes stale as soon as a run directory is copied or + moved. Both spellings are tried, and each of them compressed, because a + cleaned step holds ``model.pdb.gz`` where an uncleaned one holds + ``model.pdb`` and the two are the same content. + """ + candidates = (Path(model.rel_path), Path(model.path, model.file_name)) + for candidate in candidates: + for spelling in (candidate, Path(f"{candidate}.gz")): + if spelling.is_file(): + return spelling + raise FileNotFoundError( + f"cannot read {model.file_name}: tried " + + ", ".join(str(candidate) for candidate in candidates) + ) + + +def derive_seed( + iniseed: int, + inputs: Union[Persistent, Sequence[Persistent]], + repeat: int = 0, +) -> int: + """Return a CNS job's random seed as a function of the job itself. + + The seed depends on ``iniseed``, on the content of the models the job + starts from, and on which repeat of that job this is. It depends on + nothing else: not on the job's index in the schedule, not on the + schedule's length, and not on how many molecules or conformers the run + happens to contain. A job therefore keeps its seed when the run around + it grows, shrinks, or is reordered, which is the property that lets the + same computation be recognised as the same computation. + + ``iniseed`` keeps its meaning exactly: changing it changes every seed in + the run. + + Seeds need not be unique, and this function does not try to make them so. + Two unrelated jobs drawing the same number is harmless -- they start from + different structures and compute different things. What must never + collide is two repeats of *one* job, which would make them duplicates + rather than additional sampling, and ``repeat`` is what separates those. + """ + digest = hashlib.blake2b(digest_size=8) + digest.update(f"iniseed={int(iniseed)}\nrepeat={int(repeat)}\n".encode()) + for model in transform_to_list(inputs): + digest.update(f"{content_checksum(model_path(model))}\n".encode()) + return 1 + int.from_bytes(digest.digest(), "big") % (SEED_CEILING - 1) + + +def refinement_schedule( + n_models: int, + sampling_factor: int, +) -> list[tuple[int, int]]: + """Return ``(model index, replica index)`` for every refinement job. + + Replicas are emitted in rounds: every input model gets its first replica + before any model gets its second. The alternative -- all of one model's + replicas, then all of the next model's -- makes a model's number depend + on ``sampling_factor``, so raising it renumbers every job belonging to an + input after the first. + + This is a numbering property, not a reuse property. Seeds are derived + from content (see :func:`derive_seed`), so the order in which jobs are + emitted changes neither what any of them computes nor whether it can be + recognised again. What it changes is whether ``flexref_3.pdb`` means the + same thing in two runs that differ only in ``sampling_factor`` -- for a + person comparing two runs, and for every downstream step that carries a + model's number. + + The three refinement modules share this schedule. They are one job shape + and are tested through one representative, so a change applied to one of + them and not the others would leave the representative representing + nothing while every test stayed green. + """ + return [ + (model_index, replica) + for replica in range(sampling_factor) + for model_index in range(n_models) + ] def generate_default_header( @@ -29,9 +147,7 @@ def generate_default_header( link = load_link(Path(path, cns_paths.LINK_FILE)) scatter = load_scatter(Path(path, cns_paths.SCATTER_LIB)) tensor = load_tensor(**cns_paths.get_tensors(path)) - trans_vec = load_trans_vectors( - **cns_paths.get_translation_vectors(path) - ) + trans_vec = load_trans_vectors(**cns_paths.get_translation_vectors(path)) water_box = load_boxtyp20(cns_paths.get_water_box(path)["boxtyp20"]) else: @@ -245,10 +361,7 @@ def load_boxtyp20(waterbox_param: Path) -> str: # This is used by docking -def prepare_multiple_input( - pdb_input_list: list[str], - psf_input_list: list[str] -) -> str: +def prepare_multiple_input(pdb_input_list: list[str], psf_input_list: list[str]) -> str: """Prepare multiple input files.""" input_str = f"{linesep}! Input structure{linesep}" for psf in psf_input_list: @@ -307,15 +420,12 @@ def prepare_single_input( for i, segid in enumerate(chainsegs, start=1): input_str += write_eval_line(f"prot_segid_{i}", segid) - seed = RND.randint(100, 99999) - input_str += write_eval_line("seed", seed) - return input_str def _add_cg_backmapping_arguments( - input_element: Union[PDBFile, list[PDBFile]], - ) -> str: + input_element: Union[PDBFile, list[PDBFile]], +) -> str: """Build CG backmapping CNS arguments string. Args: @@ -334,24 +444,32 @@ def _add_cg_backmapping_arguments( cgtoaa_tbl_list: list[Path] = [] input_str: str = "" # Structure composed of multiple entries - if isinstance(input_element.aa_topology, (list, tuple,)): - for psf in input_element.aa_topology: + if isinstance( + input_element.aa_topology, + ( + list, + tuple, + ), + ): + for psf, tbl, is_shape in zip( + input_element.aa_topology, + input_element.cgtoaa_tbl, + input_element.shape, + ): if psf is None: raise ValueError( f"All-Atom Topology not found {input_element.rel_path}. " "Conversion to all-atom requires a topology generated " "with [topoaa] and [topocg]." - ) - else: - aa_psf_list.append(psf.rel_path.as_posix()) - for i, tbl in enumerate(input_element.cgtoaa_tbl): - if tbl is None and not input_element.shape[i]: + ) + if tbl is None and not is_shape: raise ValueError( - f"Coarse-Crain to All-Atom restraint file not found " + "Coarse-Grain to All-Atom restraint file not found " f"{input_element.rel_path}. Conversion to all-atom " "requires a restraint file generated with [topocg]." - ) - elif not input_element.shape[i]: + ) + if not is_shape: + aa_psf_list.append(psf.rel_path.as_posix()) cgtoaa_tbl_list.append(tbl.as_posix()) # Structure composed of only one entry else: @@ -362,24 +480,28 @@ def _add_cg_backmapping_arguments( f"All-Atom Topology not found {input_element.rel_path}." "Conversion to all-atom requires a topology generated with" " [topoaa] and [topocg]." - ) - aa_psf_list.append(pdb.aa_topology.rel_path.as_posix()) + ) if pdb.cgtoaa_tbl is None and not shape: raise ValueError( "Coarse-Crain to All-Atom restraint file not found for" f" entry: {input_element.rel_path}. Conversion to all-atom" " requires a restraint file generated with [topocg]." - ) - cgtoaa_tbl_list.append(pdb.cgtoaa_tbl.as_posix()) + ) + if not shape: + aa_psf_list.append(pdb.aa_topology.rel_path.as_posix()) + cgtoaa_tbl_list.append(pdb.cgtoaa_tbl.as_posix()) # Loop over all the files that need to be added - for ind, (aa_psf, cg2aa_tbl) in enumerate(zip(aa_psf_list, cgtoaa_tbl_list), start=1): + for ind, (aa_psf, cg2aa_tbl) in enumerate( + zip(aa_psf_list, cgtoaa_tbl_list), + start=1, + ): # eval line for psf input_str += write_eval_line(f"input_aa_psf_filename_{ind}", aa_psf) # eval line for pdb input_str += write_eval_line( f"input_aa_pdb_filename_{ind}", f"{aa_psf[:-4]}.pdb" - ) + ) # eval line for tbl input_str += write_eval_line(f"input_cgtbl_filename_{ind}", cg2aa_tbl) return input_str @@ -398,6 +520,7 @@ def prepare_cns_input( default_params_path: Optional[Path] = None, debug: Optional[bool] = False, seed: Optional[int] = None, + chainid_list: Optional[list[str]] = None, ) -> Union[Path, str]: """ Generate the .inp file needed by the CNS engine. @@ -410,16 +533,48 @@ def prepare_cns_input( input_element : `libs.libontology.Persisten`, list of those """ # TODO: Refactor this function into smaller functions or classes + pdb_files = transform_to_list(input_element) + if isinstance(input_element, (list, tuple)): + component_count = len(input_element) + elif isinstance(input_element.topology, (list, tuple)): + component_count = len(input_element.topology) + else: + component_count = 1 + + # Direct module callers do not pass through workflow IO and therefore do + # not trigger BaseCNSModule's molecule-parameter expansion. Complete each + # ``mol_*`` family here from its declared molecule-1 default, where the + # actual input structure provides an authoritative component count. + default_values = dict(defaults) + molecule_defaults = { + match.group("base"): value + for name, value in default_values.items() + if (match := re.fullmatch(r"(?Pmol_.+)_1", name)) is not None + } + for base, value in molecule_defaults.items(): + for molecule_index in range(2, component_count + 1): + default_values.setdefault(f"{base}_{molecule_index}", value) + # read the default parameters - default_params = load_workflow_params(**defaults) + # ``ambig_fname`` can name an archive in the configuration, but CNS reads the + # per-job extracted table supplied below. Do not leave the stale archive + # assignment in the generated input. + default_values.pop("ambig_fname", None) + default_params = load_workflow_params(**default_values) default_params += write_eval_line("ambig_fname", ambig_fname) # Check if any PDBFile has ligand files and override global parameters # This is important for ensembles with autotoppar where each model has different ligand files - pdb_files = transform_to_list(input_element) + # + # Only the parameter file is emitted. `$ligand_top_fname` is read solely + # by the two topology recipes, and neither of them comes through this + # function -- so for every module that does, the assignment is dead text. + # Its value is a step-folder path, and being unread it is not a declared + # dependency and nothing rewrites it, so it would otherwise carry that + # locator into the job's canonical form and make the same job + # unrecognisable from another directory. `tests/test_libcns.py` pins the + # premise, so a recipe that starts reading it cannot go unnoticed. for pdb in pdb_files: - if hasattr(pdb, "ligand_top_fname") and pdb.ligand_top_fname: - default_params += write_eval_line("ligand_top_fname", pdb.ligand_top_fname) if hasattr(pdb, "ligand_param_fname") and pdb.ligand_param_fname: default_params += write_eval_line( "ligand_param_fname", pdb.ligand_param_fname @@ -474,7 +629,8 @@ def prepare_cns_input( segid_str = "" if native_segid: if isinstance(input_element, (list, tuple)): - chainid_list = check_combination_chains(input_element) + if chainid_list is None: + chainid_list = check_combination_chains(input_element) for i, _chainseg in enumerate(chainid_list, start=1): segid_str += write_eval_line(f"prot_segid_{i}", _chainseg) @@ -492,10 +648,10 @@ def prepare_cns_input( output += write_eval_line("count", model_number) - # Set pseudo-random seed - if seed is None: - seed = RND.randint(100, 99999) - seed_str = write_eval_line("seed", seed) + # A seed is emitted only for recipes which are explicitly given one, and + # the caller derives it from the job rather than drawing it. See + # `derive_seed`. + seed_str = write_eval_line("seed", seed) if seed is not None else "" # Combine all input parts inp = default_params + input_str + seed_str + output + segid_str + recipe_str @@ -535,7 +691,6 @@ def prepare_expected_pdb( # Single model / complex else: pdb.topology = model_obj.topology - pdb.seed = model_obj.seed pdb.aa_topology = model_obj.aa_topology pdb.cgtoaa_tbl = model_obj.cgtoaa_tbl pdb.restr_fname = model_obj.restr_fname diff --git a/src/haddock/libs/libcnscanonical.py b/src/haddock/libs/libcnscanonical.py new file mode 100644 index 0000000000..ac7e4988eb --- /dev/null +++ b/src/haddock/libs/libcnscanonical.py @@ -0,0 +1,991 @@ +"""Virtual canonical representations for CNS job identity. + +The representation is checksum-side only: ordinary HADDOCK runs execute their +generated inputs in the normal step layout. Cache-key construction will become +the production consumer in the caching stage; executable canonical-workspace +validation is deliberately deferred to the later audit stage. +""" + +from __future__ import annotations + +import gzip +import hashlib +import os +import re +import tempfile +from dataclasses import dataclass +from pathlib import Path + +from haddock.core.typing import Optional, Sequence, Union + + +_ASSIGNMENT_PATTERNS = ( + re.compile( + r"eval(?:uate)?\s*\(\s*\$(?P[A-Za-z0-9_]+)\s*=\s*" + r"(?P.*)\s*\)" + ), + re.compile(r"\{===>\}\s*(?P[A-Za-z0-9_]+)\s*=\s*(?P.*?)\s*;"), +) + +_REFERENCE_PATTERN = re.compile( + r"@@?(?P" + r"MODULE:[^\s;]+|TOPPAR:[^\s;]+|" + r"MODULE/[^\s;]+|TOPPAR/[^\s;]+|" + r"[$&][A-Za-z0-9_]*_[$&][A-Za-z0-9_]+|" + r"[$&][A-Za-z0-9_]+|[^\s;]+)" +) +_INDEXED_VARIABLE_REFERENCE_PATTERN = re.compile( + r"(?P[$&][A-Za-z0-9_]*_)(?P[$&][A-Za-z0-9_]+)$" +) +_DYNAMIC_TOPPAR_PREFIX_PATTERN = re.compile( + r'"?(?PTOPPAR(?::|/)[^"\s]+?)"?\s*\+\s*encode\(' +) +_COUNT_SUFFIX_REFERENCE_PATTERN = re.compile( + r"(?P[$&][A-Za-z0-9_]+|[^+\s]+)" + r'\s*\+\s*"(?P[^"]*)"\s*\+\s*' + r"encode\(\s*(?P[$&][A-Za-z0-9_]+|\d+)\s*\)" +) +_LOCATOR_COUNT_ASSIGNMENT_PATTERN = re.compile( + r"(?im)(?P" + r"(?:eval(?:uate)?\s*\(\s*\$|\{===>\}\s*)" + r"count\s*=\s*" + r")(?P[^);\r\n]*)(?P[);])" +) + + +#: What the ``canonical-cns`` pin is bound to. +#: +#: The CNS executable is not an input to the computation; it is the machine +#: that evaluates it. The computation a job declares is its script together +#: with the data that script reads, and the binary is the interpreter of that +#: declaration. So the pin exists -- the executable is a named position in +#: the mapping, not something the representation forgets about -- but it is +#: bound to a constant rather than to the executable's own bytes. +#: +#: Binding it to the bytes would make an identity that no two installations +#: can ever share, because no two of them compile or download the same binary. +#: That is not a corner case: sharing identities across installations is the +#: principal use of ever computing one -- a lab-wide store, a store published +#: alongside a paper, a workstation result reused on a cluster. +#: +#: Nor would the bytes buy safety. They cannot detect a CNS build that +#: computes *different* results; they only prevent two builds that agree from +#: being recognised as agreeing, which is the overwhelmingly common case. A +#: build that genuinely disagrees is a reproducibility problem this +#: representation cannot fix and should not pretend to: pointing HADDOCK3 at a +#: different engine and expecting different numbers is a user error that no +#: identity scheme catches for free. The honest answer is to record which +#: executable produced a result as *provenance*, where a mixture of builds is +#: visible and auditable without being part of identity. +_CNS_EXECUTABLE_POLICY_PIN = hashlib.sha256( + b"haddock3-canonical-cns-executable-is-not-an-input" +).hexdigest() + + +@dataclass(frozen=True) +class CanonicalDependency: + """One immutable CNS input in its location-independent role.""" + + original_path: Path + canonical_name: str + checksum: str + + +@dataclass(frozen=True) +class CanonicalMapping: + """The complete, immutable identity mapping of a CNS job.""" + + canonical_script: str + dependencies: tuple[CanonicalDependency, ...] + cns_exec: Path + output_paths: tuple[Path, ...] + canonical_output_names: tuple[str, ...] + output_shape: str + invariant_dependencies: tuple[str, ...] + work_dir: Path + module_dir: Path + toppar_dir: Path + + @property + def checksums(self) -> dict[str, str]: + """Return the canonical checksum tree for this CNS job.""" + return { + "canonical-cns": _CNS_EXECUTABLE_POLICY_PIN, + "canonical.inp": _checksum_bytes(self.canonical_script.encode()), + **{dep.canonical_name: dep.checksum for dep in self.dependencies}, + } + + @property + def dependency_paths(self) -> dict[Path, str]: + """Map resolved original paths to their canonical names.""" + return { + dependency.original_path: dependency.canonical_name + for dependency in self.dependencies + } + + +@dataclass +class CNSDependencyScan: + """Resolved CNS read dependencies for one job.""" + + read_files: list[Path] + unresolved_reads: list[str] + + +class _IgnoredReference: + """Sentinel for optional references that do not resolve to a file.""" + + +IGNORED_REFERENCE = _IgnoredReference() + + +def build_canonical_mapping( + input_file: Path | str, + *, + envvars: dict[str, str], + cns_exec: Path, + output_files: Sequence[Path] = (), + output_pdb_files: Sequence[Path] = (), + work_dir: Path | None = None, +) -> CanonicalMapping: + """Resolve every CNS read and rewrite the job into canonical names.""" + work_dir = (work_dir or Path.cwd()).resolve() + module_dir = _resolve_env_path(envvars["MODULE"], work_dir) + toppar_dir = _resolve_env_path(envvars["TOPPAR"], work_dir) + script = _cns_job_script(input_file, work_dir) + scan = _scan_cns_job_dependencies(input_file, envvars, work_dir) + if scan.unresolved_reads: + unresolved = ", ".join(scan.unresolved_reads) + raise ValueError(f"Canonical CNS input has unresolved reads: {unresolved}") + + paths = list(dict.fromkeys(path.resolve() for path in scan.read_files)) + variables = _script_path_variables(script, work_dir, module_dir, toppar_dir) + canonical_names = _canonical_dependency_names( + paths, + module_dir, + toppar_dir, + variables, + ) + dependencies = tuple( + CanonicalDependency( + path, + canonical_names[path], + compression_transparent_checksum(path), + ) + for path in paths + ) + + normalized_outputs = tuple(_absolute_path(path, work_dir) for path in output_files) + pdb_outputs = {_absolute_path(path, work_dir) for path in output_pdb_files} + for output in normalized_outputs: + if output.suffix.lower() == ".pdb": + pdb_outputs.add(output) + if len(pdb_outputs) != 1 or len(normalized_outputs) not in (1, 2): + raise ValueError( + "A canonical CNS job must declare exactly one PDB and at most one " + "PSF output." + ) + psf_outputs = [ + output for output in normalized_outputs if output.suffix.lower() == ".psf" + ] + if len(psf_outputs) > 1: + raise ValueError("A canonical CNS job may declare at most one PSF output.") + + pdb_output = next(iter(pdb_outputs)) + canonical_output_names = ("canonical-output.pdb",) + ( + ("canonical-output.psf",) if psf_outputs else () + ) + output_name_map = {pdb_output: "canonical-output.pdb"} + if psf_outputs: + output_name_map[psf_outputs[0]] = "canonical-output.psf" + + canonical_script = _rewrite_canonical_script( + script, + work_dir, + module_dir, + toppar_dir, + { + dependency.original_path: dependency.canonical_name + for dependency in dependencies + }, + output_name_map, + text_names=_script_dependency_aliases( + script, + work_dir, + module_dir, + toppar_dir, + canonical_names, + ), + ) + _assert_canonical_script( + canonical_script, + work_dir, + module_dir, + toppar_dir, + [ + *[ + dependency.original_path + for dependency in dependencies + if not dependency.canonical_name.startswith(("module/", "toppar/")) + ], + *normalized_outputs, + ], + [ + *[dependency.canonical_name for dependency in dependencies], + *output_name_map.values(), + ], + canonical_output_names, + ) + cns_exec = _absolute_path(cns_exec, work_dir) + return CanonicalMapping( + canonical_script=canonical_script, + dependencies=dependencies, + cns_exec=cns_exec, + output_paths=(pdb_output, *psf_outputs), + canonical_output_names=canonical_output_names, + output_shape="pdb+psf" if psf_outputs else "pdb", + invariant_dependencies=tuple( + sorted( + ["canonical-cns"] + + [ + dependency.canonical_name + for dependency in dependencies + if dependency.canonical_name.startswith(("module/", "toppar/")) + ] + ) + ), + work_dir=work_dir, + module_dir=module_dir, + toppar_dir=toppar_dir, + ) + + +def canonical_mapping_for_job(job) -> CanonicalMapping: + """Build the canonical mapping for a CNSJob without importing its class.""" + return build_canonical_mapping( + job.input_file, + envvars=job.envvars, + cns_exec=Path(job.cns_exec), + output_files=job.output_files, + output_pdb_files=job.output_pdb_files, + work_dir=job.work_dir, + ) + + +def scan_cns_dependencies( + input_file: Path, + envvars: dict[str, str], +) -> CNSDependencyScan: + """Resolve explicit CNS read dependencies.""" + input_file = input_file.resolve() + workdir = input_file.parent + module_dir = _resolve_env_path(envvars["MODULE"], workdir) + toppar_dir = _resolve_env_path(envvars["TOPPAR"], workdir) + + read_files: list[Path] = [] + seen_read_files: set[Path] = set() + unresolved_reads: list[str] = [] + visited: set[Path] = set() + + def add_read_file(path: Path) -> None: + path = path.resolve() + if path in seen_read_files: + return + seen_read_files.add(path) + read_files.append(path) + + def scan_file(path: Path, variables: dict[str, str]) -> None: + path = path.resolve() + if path in visited: + return + visited.add(path) + + if not path.exists(): + unresolved_reads.append(str(path)) + return + + add_read_file(path) + local_vars = dict(variables) + guard_stack: list[tuple[str, bool]] = [] + + for raw_line in path.read_text(encoding="utf-8", errors="ignore").splitlines(): + line = raw_line.split("!", 1)[0].strip() + if not line: + continue + + lowered = line.lower() + if lowered.startswith("if ("): + guard_var = _extract_nonempty_guard_variable(line) + if guard_var is not None: + guard_stack.append((guard_var, True)) + elif lowered.startswith("end if") and guard_stack: + guard_stack.pop() + + for pattern in _ASSIGNMENT_PATTERNS: + match = pattern.search(line) + if match: + value = _normalize_assignment_value(match.group("value")) + local_vars[match.group("name")] = value + dynamic_prefix = _extract_dynamic_toppar_prefix(value) + if dynamic_prefix is not None: + local_vars[f"__dynamic_prefix__{match.group('name')}"] = ( + dynamic_prefix + ) + break + + for match in _REFERENCE_PATTERN.finditer(line): + token = match.group("target") + resolved = _resolve_reference( + token=token, + current_file=path, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=local_vars, + ) + if resolved is IGNORED_REFERENCE: + continue + if resolved is None: + if _is_guarded_optional_reference(token, guard_stack): + continue + unresolved_reads.append(token) + continue + if isinstance(resolved, list): + for resolved_path in resolved: + if not resolved_path.exists(): + unresolved_reads.append(str(resolved_path)) + continue + add_read_file(resolved_path) + if resolved_path.suffix.lower() == ".cns": + scan_file(resolved_path, local_vars) + continue + if not isinstance(resolved, Path): + continue + if not resolved.exists(): + unresolved_reads.append(str(resolved)) + continue + + add_read_file(resolved) + if resolved.suffix.lower() == ".cns": + scan_file(resolved, local_vars) + + scan_file(input_file, {}) + return CNSDependencyScan( + read_files=read_files, + unresolved_reads=sorted(set(unresolved_reads)), + ) + + +def compression_transparent_checksum(path: Path) -> str: + """Checksum a file by its logical bytes, transparently to gzip storage.""" + path = path.resolve() + data = path.read_bytes() + if path.name.endswith(".gz"): + data = gzip.decompress(data) + return _checksum_bytes(data) + + +def _checksum_bytes(data: bytes) -> str: + return hashlib.sha256(data).hexdigest() + + +def _cns_job_script(input_file: Path | str, work_dir: Path) -> str: + """Read a materialized CNS input or return its in-memory contents.""" + if isinstance(input_file, Path): + return _absolute_path(input_file, work_dir).read_text(encoding="utf-8") + return input_file + + +def _scan_cns_job_dependencies( + input_file: Path | str, + envvars: dict[str, str], + work_dir: Path, +) -> CNSDependencyScan: + """Resolve the files read by a materialized or in-memory CNS input.""" + if isinstance(input_file, Path): + script_path = _absolute_path(input_file, work_dir) + scan = scan_cns_dependencies(script_path, envvars) + return CNSDependencyScan( + [path for path in scan.read_files if path != script_path], + scan.unresolved_reads, + ) + + with tempfile.NamedTemporaryFile( + mode="w", + encoding="utf-8", + dir=work_dir, + suffix=".inp", + delete=False, + ) as handle: + handle.write(input_file) + temporary_input = Path(handle.name) + try: + scan = scan_cns_dependencies(temporary_input, envvars) + return CNSDependencyScan( + [path for path in scan.read_files if path != temporary_input.resolve()], + scan.unresolved_reads, + ) + finally: + temporary_input.unlink(missing_ok=True) + + +def _script_path_variables( + script: str, + work_dir: Path, + module_dir: Path, + toppar_dir: Path, +) -> dict[Path, str]: + """Associate assignment variables with resolved paths for named roles.""" + result: dict[Path, str] = {} + variables: dict[str, str] = {} + for line in script.splitlines(): + for pattern in _ASSIGNMENT_PATTERNS: + match = pattern.search(line) + if match is None: + continue + variable = match.group("name") + value = _normalize_assignment_value(match.group("value")) + variables[variable] = value + resolved = _resolve_count_suffix_reference( + token=value, + current_file=work_dir / "canonical.inp", + workdir=work_dir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=set(), + ) + if resolved is None: + resolved = _resolve_reference( + token=value, + current_file=work_dir / "canonical.inp", + workdir=work_dir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + ) + if isinstance(resolved, Path) and resolved.exists(): + result[resolved.resolve()] = _canonical_role_variable( + variable, + value, + ) + break + return result + + +def _script_dependency_aliases( + script: str, + work_dir: Path, + module_dir: Path, + toppar_dir: Path, + canonical_names: dict[Path, str], +) -> dict[str, str]: + """Map textual dependency aliases to the resolved canonical role name.""" + aliases: dict[str, str] = {} + variables: dict[str, str] = {} + for line in script.splitlines(): + for pattern in _ASSIGNMENT_PATTERNS: + match = pattern.search(line) + if match is None: + continue + variable = match.group("name") + value = _normalize_assignment_value(match.group("value")) + variables[variable] = value + dynamic_prefix = _extract_dynamic_toppar_prefix(value) + if dynamic_prefix is not None: + variables[f"__dynamic_prefix__{variable}"] = dynamic_prefix + + resolved = _resolve_count_suffix_reference( + token=value, + current_file=work_dir / "canonical.inp", + workdir=work_dir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=set(), + ) + if not isinstance(resolved, Path): + resolved = _resolve_reference( + token=value, + current_file=work_dir / "canonical.inp", + workdir=work_dir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + ) + if not isinstance(resolved, Path): + break + + canonical_name = canonical_names.get(resolved.resolve()) + if canonical_name is None: + break + if canonical_name.startswith(("module/", "toppar/")): + # install-tree files keep their MODULE:/TOPPAR: spelling + break + + if not value.startswith(("$", "&")): + # a variable *reference* is already location-independent; rewriting it + # would replace the CNS symbol name itself rather than a filename + aliases[value] = canonical_name + count_match = _COUNT_SUFFIX_REFERENCE_PATTERN.fullmatch(value.strip()) + if count_match is not None: + base_value = _resolve_variable_text( + count_match.group("base"), + variables, + ) + if base_value: + aliases[base_value] = canonical_name + break + return aliases + + +def _resolve_variable_text(token: str, variables: dict[str, str]) -> Optional[str]: + """Resolve a textual CNS variable reference without resolving filesystem paths.""" + seen: set[str] = set() + while token.startswith(("$", "&")): + variable_name = token[1:] + if variable_name in seen or variable_name not in variables: + return None + seen.add(variable_name) + token = variables[variable_name] + return token + + +def _canonical_role_variable(variable: str, value: str) -> str: + """Return the variable name that carries a dependency's stable role.""" + match = _COUNT_SUFFIX_REFERENCE_PATTERN.fullmatch(value.strip()) + if match is not None and match.group("base").startswith(("$", "&")): + return match.group("base")[1:].lower() + return variable.lower() + + +def _canonical_dependency_names( + paths: Sequence[Path], + module_dir: Path, + toppar_dir: Path, + variables: dict[Path, str], +) -> dict[Path, str]: + """Assign stable roles in dependency first-reference order.""" + names: dict[Path, str] = {} + counters = {"pdb": 0, "psf": 0, "generic": 0} + named_roles = ( + ("unambig", "canonical-unambig.tbl"), + ("hbond", "canonical-hbond.tbl"), + ("ambig", "canonical-ambig.tbl"), + ("dihe", "canonical-dihe.tbl"), + ("sym", "canonical-symmetry.tbl"), + ("tensor", "canonical-tensor.tbl"), + ("ligand_top", "canonical-ligand.top"), + ("ligand_param", "canonical-ligand.param"), + ) + for path in paths: + if path.is_relative_to(module_dir): + names[path] = f"module/{path.relative_to(module_dir).as_posix()}" + continue + if path.is_relative_to(toppar_dir): + names[path] = f"toppar/{path.relative_to(toppar_dir).as_posix()}" + continue + variable = variables.get(path, "") + named = next( + ( + name + for marker, name in named_roles + if re.search(rf"(?:^|_){re.escape(marker)}(?:_|$)", variable) + ), + None, + ) + if named is not None and named not in names.values(): + names[path] = named + continue + logical_name = _strip_known_compression_suffix(path.name) + suffix = Path(logical_name).suffix.lower() + if suffix in (".pdb", ".psf"): + counters[suffix[1:]] += 1 + names[path] = f"canonical-input-{counters[suffix[1:]]}{suffix}" + elif suffix == ".tbl" and variable.startswith(("input_aa_", "input_cgtbl_")): + counters["generic"] += 1 + names[path] = f"canonical-cg-input-{counters['generic']}.tbl" + else: + counters["generic"] += 1 + names[path] = f"canonical-input-{counters['generic']}{suffix}" + return names + + +def _canonicalize_install_paths( + script: str, + module_dir: Path, + toppar_dir: Path, +) -> str: + """Give absolute install-tree paths their environment-relative CNS spelling. + + ``MODULE:`` and ``TOPPAR:`` are resolved by CNS through environment variables, + so a reference spelled that way is already independent of where HADDOCK3 is + installed. Paths written out in full are not, and reach the canonical form even + when the recipe never reads them -- which is how the install path used to enter + the identity of every topology job. + """ + + def replace(match: re.Match[str]) -> str: + candidate = Path(match.group("path")).resolve() + for root, prefix in ((module_dir, "MODULE:"), (toppar_dir, "TOPPAR:")): + if candidate.is_relative_to(root): + return f"{prefix}{candidate.relative_to(root).as_posix()}" + return match.group(0) + + return re.sub(r'(?/[^\s"\';]+)', replace, script) + + +def _rewrite_canonical_script( + script: str, + work_dir: Path, + module_dir: Path, + toppar_dir: Path, + dependency_names: dict[Path, str], + output_names: dict[Path, str], + text_names: Optional[dict[str, str]] = None, +) -> str: + """Replace only resolved job-specific path spellings in CNS text. + + Two rules keep the result both canonical and executable: + + - install-tree references keep their ``MODULE:``/``TOPPAR:`` spelling. The + ``module/`` and ``toppar/`` names are pin names for the checksum tree; writing + them into the script instead would leave the top-level input and the module + scripts it includes disagreeing about what ``MODULE`` points at. + - only *path* spellings are rewritten, never a bare basename, so that an input + and an output sharing a basename cannot consume each other's binding. + """ + result = _canonicalize_install_paths(script, module_dir, toppar_dir) + job_paths = { + path: name + for path, name in dependency_names.items() + if not name.startswith(("module/", "toppar/")) + } + for path, name in {**job_paths, **output_names}.items(): + candidates = {str(path), path.as_posix()} + candidates.add(os.path.relpath(path, start=work_dir)) + for match in re.finditer( + r'(?(?:\.\.?/|/)[^\s"\';]+)', + result, + ): + candidate = match.group("path") + if _absolute_path(Path(candidate), work_dir) == path: + candidates.add(candidate) + for candidate in sorted(candidates, key=len, reverse=True): + if candidate: + result = _replace_script_token(result, candidate, name) + for candidate, name in sorted( + (text_names or {}).items(), + key=lambda item: len(item[0]), + reverse=True, + ): + if candidate: + result = _replace_script_token(result, candidate, name) + return _canonicalize_logging_level(_canonicalize_locator_count_assignment(result)) + + +def _replace_script_token(script: str, token: str, replacement: str) -> str: + """Replace a path-like token without matching inside another path token.""" + pattern = re.compile(rf"(? str: + """Erase generated structure index values from canonical CNS input.""" + return _LOCATOR_COUNT_ASSIGNMENT_PATTERN.sub( + r"\gcanonical-count\g", + script, + ) + + +def _canonicalize_logging_level(script: str) -> str: + """Normalize CNS interpreter verbosity, which cannot affect job artifacts.""" + return re.sub( + r'(?P\$log_level\s*=\s*)"[^"]*"', + r'\g"canonical-log-level"', + script, + ) + + +def _assert_output_bindings( + script: str, + canonical_output_names: Sequence[str], +) -> None: + """The script must write exactly the outputs the mapping declares. + + Output shape is part of the transformation, so a script whose + ``$output_pdb_filename`` says something other than the declared canonical name + is describing a different computation from the one being checksummed. + """ + for variable, canonical in ( + ("output_pdb_filename", "canonical-output.pdb"), + ("output_psf_filename", "canonical-output.psf"), + ): + match = re.search(rf'\${variable}\s*=\s*"(?P[^"]*)"', script) + if match is None: + continue + if canonical not in canonical_output_names: + raise ValueError( + f"Canonical CNS script writes ${variable} but no {canonical!r} " + "output was declared" + ) + if match.group("name") != canonical: + raise ValueError( + f"Canonical CNS script binds ${variable} to " + f"{match.group('name')!r} instead of the declared {canonical!r}" + ) + + +def _assert_canonical_script( + script: str, + work_dir: Path, + module_dir: Path, + toppar_dir: Path, + paths: Sequence[Path], + canonical_names: Sequence[str] = (), + canonical_output_names: Sequence[str] = (), +) -> None: + """Reject location-dependent canonical scripts before they become identity.""" + _assert_output_bindings(script, canonical_output_names) + scanned = script + for name in sorted(canonical_names, key=len, reverse=True): + if name: + scanned = scanned.replace(name, "\x00") + leaked = [str(work_dir), str(module_dir), str(toppar_dir)] + leaked.extend(path.name for path in paths if path.name) + for token in leaked: + if token and token in scanned: + raise ValueError( + f"Canonical CNS script leaked {token!r} from job in {work_dir}" + ) + match = re.search(r"(?:^|/)\d+_[A-Za-z][A-Za-z0-9_]*", scanned) + if match: + raise ValueError( + f"Canonical CNS script leaked step-folder token {match.group(0)!r}" + ) + + +def _absolute_path(path: Path, work_dir: Path) -> Path: + return path.resolve() if path.is_absolute() else (work_dir / path).resolve() + + +def _resolve_env_path(raw_path: str, workdir: Path) -> Path: + path = Path(raw_path) + if path.is_absolute(): + return path.resolve() + return (workdir / path).resolve() + + +def _resolve_reference( + *, + token: str, + current_file: Path, + workdir: Path, + module_dir: Path, + toppar_dir: Path, + variables: dict[str, str], + seen_variables: Optional[set[str]] = None, +) -> Optional[Union[Path, list[Path], _IgnoredReference]]: + seen_variables = seen_variables or set() + indexed_reference = _INDEXED_VARIABLE_REFERENCE_PATTERN.fullmatch(token) + if indexed_reference is not None: + prefix = indexed_reference.group("prefix")[1:] + indexed_variables = sorted( + variable + for variable in variables + if re.fullmatch(rf"{re.escape(prefix)}\d+", variable) + ) + if not indexed_variables: + return None + resolved_paths: list[Path] = [] + for variable in indexed_variables: + resolved = _resolve_reference( + token=f"${variable}", + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=set(seen_variables), + ) + if not isinstance(resolved, Path): + return None + resolved_paths.append(resolved) + return resolved_paths + resolved_from_variable = False + if token.startswith(("$", "&")): + variable_name = token[1:] + if variable_name in seen_variables: + return None + seen_variables.add(variable_name) + dynamic_prefix = variables.get(f"__dynamic_prefix__{variable_name}") + if variable_name not in variables: + if dynamic_prefix is not None: + prefix_path = _resolve_reference( + token=dynamic_prefix, + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=seen_variables, + ) + if isinstance(prefix_path, Path): + return sorted(prefix_path.parent.glob(f"{prefix_path.name}*")) + return None + token = variables[variable_name] + if token == "": + return IGNORED_REFERENCE + resolved_from_variable = True + dynamic_count_path = _resolve_count_suffix_reference( + token=token, + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=seen_variables, + ) + if dynamic_count_path is not None: + return dynamic_count_path + if token.startswith(("$", "&")): + return _resolve_reference( + token=token, + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=seen_variables, + ) + if dynamic_prefix is not None and any( + fragment in token for fragment in ("+", "encode(") + ): + prefix_path = _resolve_reference( + token=dynamic_prefix, + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=seen_variables, + ) + if isinstance(prefix_path, Path): + return sorted(prefix_path.parent.glob(f"{prefix_path.name}*")) + + if not token or any(fragment in token for fragment in ("+", "encode(", "$", "&")): + return None + + if token.startswith("MODULE:"): + return (module_dir / token.split(":", 1)[1].lstrip("/")).resolve() + + if token.startswith("TOPPAR:"): + return (toppar_dir / token.split(":", 1)[1].lstrip("/")).resolve() + + if token.startswith("MODULE/"): + return (module_dir / token.removeprefix("MODULE/").lstrip("/")).resolve() + + if token.startswith("TOPPAR/"): + return (toppar_dir / token.removeprefix("TOPPAR/").lstrip("/")).resolve() + + candidate = Path(token) + if candidate.is_absolute(): + return candidate.resolve() + relative_base = workdir if resolved_from_variable else current_file.parent + return (relative_base / candidate).resolve() + + +def _extract_nonempty_guard_variable(line: str) -> Optional[str]: + match = re.search(r'\$([A-Za-z0-9_]+)\s*#\s*""', line) + if match is not None: + return match.group(1) + + match = re.search(r"&BLANK%([A-Za-z0-9_]+)\s*=\s*false", line, re.IGNORECASE) + if match is not None: + return match.group(1) + + return None + + +def _resolve_count_suffix_reference( + *, + token: str, + current_file: Path, + workdir: Path, + module_dir: Path, + toppar_dir: Path, + variables: dict[str, str], + seen_variables: set[str], +) -> Optional[Path]: + match = _COUNT_SUFFIX_REFERENCE_PATTERN.fullmatch(token.strip()) + if match is None: + return None + + base = _resolve_reference( + token=match.group("base"), + current_file=current_file, + workdir=workdir, + module_dir=module_dir, + toppar_dir=toppar_dir, + variables=variables, + seen_variables=set(seen_variables), + ) + count = _resolve_literal_or_variable( + match.group("count"), + variables=variables, + ) + if not isinstance(base, Path) or count is None: + return None + + counted = Path(f"{base}{match.group('separator')}{count}") + if counted.exists(): + return counted.resolve() + if base.exists(): + return base.resolve() + return None + + +def _resolve_literal_or_variable( + token: str, + *, + variables: dict[str, str], +) -> Optional[str]: + if token.isdigit(): + return token + if not token.startswith(("$", "&")): + return None + variable = variables.get(token[1:]) + return variable if variable and variable.isdigit() else None + + +def _is_guarded_optional_reference( + token: str, + guard_stack: list[tuple[str, bool]], +) -> bool: + if not token.startswith(("$", "&")): + return False + variable_name = token[1:] + return any( + guard_var == variable_name and is_optional + for guard_var, is_optional in guard_stack + ) + + +def _extract_dynamic_toppar_prefix(value: str) -> Optional[str]: + match = _DYNAMIC_TOPPAR_PREFIX_PATTERN.search(value) + if match is None: + return None + return match.group("prefix") + + +def _normalize_assignment_value(value: str) -> str: + value = value.strip() + if len(value) >= 2 and value[0] == '"' and value[-1] == '"': + return value[1:-1] + return value + + +def _strip_known_compression_suffix(name: str) -> str: + if name.endswith(".gz"): + return name[:-3] + return name diff --git a/src/haddock/libs/libcnsoutput.py b/src/haddock/libs/libcnsoutput.py new file mode 100644 index 0000000000..01e10e28dd --- /dev/null +++ b/src/haddock/libs/libcnsoutput.py @@ -0,0 +1,144 @@ +"""Helpers for normalizing CNS-generated output artifacts.""" + +import gzip +import os +import uuid +from pathlib import Path + +from haddock.core.typing import FilePath + + +CNS_PDB_VOLATILE_PREFIXES = ( + b"REMARK FILENAME=", + b"REMARK initial structure ", + b"REMARK DATE:", + b"REMARK HADDOCK stats for ", +) +CNS_PSF_STABLE_TITLE = b"; HADDOCK3 normalized topology" + + +def _is_volatile_psf_line(line: bytes) -> bool: + """Whether a PSF line is the wall-clock stamp CNS writes into its title.""" + stripped = line.strip() + return stripped.startswith(b"DATE:") and b"created by user:" in stripped + + +def _normalize_psf_line(line: bytes) -> bytes: + """Return stable PSF title content.""" + if line.strip().startswith(b"; FILENAME="): + return CNS_PSF_STABLE_TITLE + _line_ending(line) + if _is_volatile_psf_line(line): + return b"" + return line + + +def _rewrite_atomically(path: Path, stable: bytes) -> None: + """Replace a file's bytes without writing into it in place.""" + temporary = path.with_name(f".{path.name}.normalize-{uuid.uuid4().hex}") + try: + temporary.write_bytes(stable) + os.replace(temporary, path) + finally: + temporary.unlink(missing_ok=True) + + +def normalize_cns_pdb(path: FilePath) -> bool: + """Remove run-volatile CNS header lines from a PDB file. + + Returns ``True`` when the file bytes were changed. + """ + pdb_path = Path(path) + if not pdb_path.exists(): + return False + + original = pdb_path.read_bytes() + logical = gzip.decompress(original) if pdb_path.name.endswith(".gz") else original + stable = normalize_cns_pdb_bytes(logical) + + if stable == logical: + return False + + _rewrite_atomically( + pdb_path, + gzip.compress(stable, mtime=0) if pdb_path.name.endswith(".gz") else stable, + ) + return True + + +def normalize_cns_pdb_bytes(pdb_bytes: bytes) -> bytes: + """Return PDB bytes without CNS run-volatile header lines.""" + stable_lines = [ + line + for line in _split_lf_records(pdb_bytes) + if not line.startswith(CNS_PDB_VOLATILE_PREFIXES) + ] + return b"".join(stable_lines) + + +def is_normalized_cns_pdb(path: FilePath) -> bool: + """Return whether a PDB, compressed or not, has stable CNS headers.""" + return _is_normalized(path, normalize_cns_pdb_bytes) + + +def normalize_cns_psf(path: FilePath) -> bool: + """Remove the run-volatile CNS date stamp from a PSF file. + + Returns ``True`` when the file bytes were changed. + """ + psf_path = Path(path) + if not psf_path.exists(): + return False + + original = psf_path.read_bytes() + logical = gzip.decompress(original) if psf_path.name.endswith(".gz") else original + stable = normalize_cns_psf_bytes(logical) + + if stable == logical: + return False + + _rewrite_atomically( + psf_path, + gzip.compress(stable, mtime=0) if psf_path.name.endswith(".gz") else stable, + ) + return True + + +def normalize_cns_psf_bytes(psf_bytes: bytes) -> bytes: + """Return PSF bytes without the CNS date stamp.""" + stable_lines = [_normalize_psf_line(line) for line in _split_lf_records(psf_bytes)] + return b"".join(stable_lines) + + +def is_normalized_cns_psf(path: FilePath) -> bool: + """Return whether a PSF, compressed or not, has a stable CNS title.""" + return _is_normalized(path, normalize_cns_psf_bytes) + + +def _is_normalized(path: FilePath, normalize) -> bool: + """Whether a file, compressed or not, is unchanged by ``normalize``.""" + artifact_bytes = Path(path).read_bytes() + if Path(path).name.endswith(".gz"): + artifact_bytes = gzip.decompress(artifact_bytes) + return normalize(artifact_bytes) == artifact_bytes + + +def _split_lf_records(data: bytes) -> list[bytes]: + """Split bytes into LF-terminated records without interpreting text.""" + records: list[bytes] = [] + start = 0 + for idx, byte in enumerate(data): + if byte == 0x0A: + records.append(data[start : idx + 1]) + start = idx + 1 + if start < len(data): + records.append(data[start:]) + return records + + +def _line_ending(line: bytes) -> bytes: + """Return the existing line ending for an LF-split record.""" + if line.endswith(b"\r\n"): + return b"\r\n" + if line.endswith(b"\n"): + return b"\n" + return b"" diff --git a/src/haddock/libs/libgrid.py b/src/haddock/libs/libgrid.py index d0c302659c..85c44cef4a 100644 --- a/src/haddock/libs/libgrid.py +++ b/src/haddock/libs/libgrid.py @@ -18,6 +18,7 @@ from haddock import log from haddock.core.defaults import cns_exec_linux as CNS_EXEC +from haddock.libs.libcnsoutput import normalize_cns_pdb, normalize_cns_psf from haddock.libs.libsubprocess import CNSJob from haddock.libs.libutil import parse_ncores @@ -101,7 +102,6 @@ class Tag(Enum): class GridInterface(ABC): - def __init__( self, input: Union[Path, str, list[str]], @@ -342,7 +342,15 @@ def retrieve_output(self) -> None: for output_f in self.expected_outputs: src = Path(f"{self.loc}/{self.id}/{output_f}") dst = Path(self.wd / f"{output_f}") - shutil.copy(src, dst) + temporary = dst.with_name(f".{dst.name}.retrieve-{uuid.uuid4().hex}") + try: + shutil.copy(src, temporary) + if not temporary.is_file() or temporary.stat().st_size == 0: + raise RuntimeError(f"GRID returned incomplete output: {src}") + self._normalize_output(temporary, logical_name=dst.name) + os.replace(temporary, dst) + finally: + temporary.unlink(missing_ok=True) def clean_timings(self) -> None: """Clean the timings dictionary.""" @@ -352,6 +360,18 @@ def clean(self) -> None: """Clean up the temporary directory where the job lives.""" shutil.rmtree(self.loc) + @staticmethod + def _normalize_output(path: Path, logical_name: Optional[str] = None) -> None: + """Normalize CNS output artifacts copied back from the grid.""" + name = logical_name or path.name + if name.endswith(".gz"): + name = name[:-3] + suffix = Path(name).suffix.lower() + if suffix == ".pdb": + normalize_cns_pdb(path) + elif suffix == ".psf": + normalize_cns_psf(path) + @staticmethod def parse_output(output_str: str) -> dict[str, str]: """Parse the output string from DIRAC commands into a dictionary.""" @@ -452,7 +472,6 @@ def process_input_f(self) -> None: self.payload_fnames.append(inp_name) with open(inp_name, "w") as f: for line in self.input_str.splitlines(keepends=True): - # Parse this line and try to identify output files output = self._find_output(line) if output: @@ -472,7 +491,6 @@ def process_input_f(self) -> None: class CompositeGridJob(GridInterface): - def __init__( self, input: list[str], @@ -656,7 +674,7 @@ def wait_for_completion(self) -> None: if jobs_to_check: log.debug(f"+ Checking status of {len(jobs_to_check)} payload(s)...") with ThreadPoolExecutor(max_workers=self.ncores) as executor: - executor.map(self.process_job, jobs_to_check) + list(executor.map(self.process_job, jobs_to_check)) else: complete = True @@ -670,10 +688,10 @@ def submit_jobs(self, tag: Tag = Tag.DEFAULT) -> None: log.info(f"++ Submitting {len(queue)} '{tag.value}' payloads to the grid...") with ThreadPoolExecutor(max_workers=self.ncores) as executor: - executor.map(lambda job: job.package(), queue) + list(executor.map(lambda job: job.package(), queue)) with ThreadPoolExecutor(max_workers=self.ncores) as executor: - executor.map(lambda job: job.submit(), queue) + list(executor.map(lambda job: job.submit(), queue)) def probe_grid_efficiency(self) -> None: """Submit a small number of jobs to probe the efficiency of the GRID.""" diff --git a/src/haddock/libs/libhpc.py b/src/haddock/libs/libhpc.py index 651eb870d2..be623371ef 100644 --- a/src/haddock/libs/libhpc.py +++ b/src/haddock/libs/libhpc.py @@ -8,6 +8,7 @@ from pathlib import Path from haddock import log, modules_defaults_path +from haddock.core.exceptions import CNSRunningError from haddock.core.typing import Any, Container, FilePath, Optional from haddock.gear.yaml2cfg import read_from_yaml_config from haddock.libs.libsubprocess import CNSJob @@ -95,9 +96,25 @@ def prepare_job_file(self, queue_type: str = "slurm") -> None: ) job_file_contents += f"cd {self.moddir}{os.linesep}" - for job in self.tasks: + self.partial_inputs: list[Path] = [] + for index, job in enumerate(self.tasks, start=1): + input_path = ( + job._output_path(job.input_file) + if isinstance(job.input_file, Path) + else None + ) + if input_path is not None and not input_path.exists(): + # Leave the shell's normal missing-input diagnostic intact. This + # also keeps job-file construction useful for dry-run callers. + job_file_contents += ( + f"{job.cns_exec} < {job.input_file} > {job.output_file}{os.linesep}" + ) + continue + partial_input = Path(self.moddir, f"{self.job_num}_{index}.partial.inp") + partial_input.write_text(job.prepare_execution_input(), encoding="utf-8") + self.partial_inputs.append(partial_input) cmd = ( - f"{job.cns_exec} < {job.input_file} > {job.output_file}" f"{os.linesep}" + f"{job.cns_exec} < {partial_input.name} > {job.output_file}{os.linesep}" ) job_file_contents += cmd @@ -125,6 +142,18 @@ def update_status(self) -> str: return self.job_status + def normalize_outputs(self) -> None: + """Publish each successful task without overriding module tolerance.""" + try: + for task in self.tasks: + try: + task.publish_outputs(check_output_log=True) + except CNSRunningError as error: + log.warning(f"CNS batch task did not produce output: {error}") + finally: + for partial_input in getattr(self, "partial_inputs", []): + partial_input.unlink(missing_ok=True) + def cancel(self, bypass_statuses: Container[str] = ("finished", "failed")) -> None: """Cancel the execution.""" if self.update_status() not in bypass_statuses: @@ -149,9 +178,7 @@ def __init__( # split tasks according to concat level if concat > 1: - log.info( - f"Concatenating, each .job will produce {concat} " "(or less) models" - ) + log.info(f"Concatenating, each .job will produce {concat} (or less) models") job_list = [task_list[i : i + concat] for i in range(0, len(task_list), concat)] self.worker_list = [HPCWorker(t, j) for j, t in enumerate(job_list, start=1)] @@ -190,9 +217,7 @@ def run(self) -> None: worker.update_status() # Log status if not finished if worker.job_status != "finished": - log.info( - f">> {worker.job_fname.name}" f" {worker.job_status}" - ) + log.info(f">> {worker.job_fname.name} {worker.job_status}") # Increment number of terminated works if worker.job_status in TERMINATED_STATUS: terminated_count += 1 @@ -215,6 +240,12 @@ def run(self) -> None: time.sleep(sleep_timer) per = (float(batch_num) / float(total_batches)) * 100 + for worker in worker_list: + # A failed concatenated worker can still contain successful + # tasks. Validate and publish each task independently; the + # module's tolerance setting decides whether missing models + # make the overall step fail. + worker.normalize_outputs() log.info( f">> Batch {batch_num}/{total_batches} took " f"{elapsed:.2f}s to finish, {per:.2f}% complete" diff --git a/src/haddock/libs/libmath.py b/src/haddock/libs/libmath.py deleted file mode 100644 index 09d9ffecb2..0000000000 --- a/src/haddock/libs/libmath.py +++ /dev/null @@ -1,26 +0,0 @@ -"""Mathematical functions.""" - -import random - - -class RandomNumberGenerator: - """ - Generate random numbers. - - Python uses the Mersenne Twister as the core generator. - It produces 53-bit precision floats and has a period of 2**19937-1 - """ - - def __init__(self, seed: int = 494) -> None: - """494 = sum([ord(i) for i in 'HADDOCK']).""" - self.seed = seed - self.random = random.Random() - self.random.seed(self.seed) - - def __call__(self, lower_limit: float = 0.0, upper_limit: float = 1.0) -> float: - """Generate a random number between range.""" - return self.random.uniform(lower_limit, upper_limit) - - def randint(self, lower_limit: int = 0, upper_limit: int = 9) -> int: - """Generate a random integer.""" - return lower_limit + int(self() * (upper_limit - lower_limit + 1)) diff --git a/src/haddock/libs/libparallel.py b/src/haddock/libs/libparallel.py index f577d1ceaa..ecaa5949b7 100644 --- a/src/haddock/libs/libparallel.py +++ b/src/haddock/libs/libparallel.py @@ -79,23 +79,29 @@ def run(self): class Worker(Process): """Work on tasks.""" - def __init__(self, tasks: Sequence[SupportsRunT], results: Queue) -> None: + def __init__( + self, + tasks: Sequence[SupportsRunT], + results: Queue, + task_indices: Optional[Sequence[int]] = None, + ) -> None: super(Worker, self).__init__() self.tasks = tasks self.result_queue = results + self.task_indices = list(task_indices or range(len(tasks))) log.debug(f"Worker ready with {len(self.tasks)} tasks") def run(self) -> None: """Execute tasks.""" results = [] - for task in self.tasks: + for task_index, task in zip(self.task_indices, self.tasks): r = None try: r = task.run() except Exception as e: log.warning(f"Exception in task execution: {e}") - results.append(r) + results.append((task_index, r)) # Put results into the queue self.result_queue.put(results) @@ -149,8 +155,16 @@ def __init__( else: sorted_task_list = tasks - job_list = split_tasks(sorted_task_list, self.num_processes) - self.worker_list = [Worker(jobs, self.queue) for jobs in job_list] + indexed_tasks = list(enumerate(sorted_task_list)) + job_list = split_tasks(indexed_tasks, self.num_processes) + self.worker_list = [ + Worker( + [task for _, task in jobs], + self.queue, + [task_index for task_index, _ in jobs], + ) + for jobs in job_list + ] log.info(f"Using {self.num_processes} cores") log.debug(f"{self.num_tasks} tasks ready.") @@ -191,7 +205,15 @@ def run(self) -> None: for w in self.worker_list: w.join() - self.results = [item for sublist in all_results for item in sublist] + # Workers finish independently, so queue arrival order is not submission + # order. Several callers pair these values with their submitted jobs. + self.results = [ + result + for _, result in sorted( + (item for sublist in all_results for item in sublist), + key=lambda item: item[0], + ) + ] log.info(f"{self.num_tasks} tasks finished") diff --git a/src/haddock/libs/libsubprocess.py b/src/haddock/libs/libsubprocess.py index 768c4316f3..575503fb4e 100644 --- a/src/haddock/libs/libsubprocess.py +++ b/src/haddock/libs/libsubprocess.py @@ -1,6 +1,7 @@ """Run subprocess jobs.""" import os +import re import shlex import subprocess from contextlib import suppress @@ -10,13 +11,18 @@ from haddock.core.exceptions import ( CNSRunningError, JobRunningError, - KnownCNSError, ) -from haddock.core.typing import Any, FilePath, Optional, ParamDict +from haddock.core.typing import Any, FilePath, Iterable, Optional, ParamDict from haddock.gear.known_cns_errors import KNOWN_ERRORS as KNOWN_CNS_ERRORS +from haddock.libs.libcnsoutput import normalize_cns_pdb, normalize_cns_psf from haddock.libs.libio import gzip_files +CNS_DENORMAL_STDERR = ( + b"Note: The following floating-point exceptions are signalling: IEEE_DENORMAL\n" +) + + class BaseJob: """Base class for a subprocess job.""" @@ -105,6 +111,8 @@ def __init__( error_file: Optional[FilePath] = None, envvars: Optional[ParamDict] = None, cns_exec: Optional[FilePath] = None, + output_files: Optional[Iterable[FilePath]] = None, + output_pdb_files: Optional[Iterable[FilePath]] = None, ) -> None: """ CNS subprocess. @@ -124,12 +132,41 @@ def __init__( A dictionary containing the environment variables needed for the CNSJob. These will be passed to subprocess.Popen.env argument. + output_pdb_files : iterable + PDB files expected from this CNS job. When provided, run-volatile + CNS header lines are removed after successful execution. + output_files : iterable + Files expected from this CNS job. Files with ``.pdb`` or ``.psf`` + suffixes are normalized after successful execution. """ self.input_file = input_file + self.work_dir = Path.cwd().resolve() self.output_file = output_file self.error_file = error_file self.envvars = envvars self.cns_exec = cns_exec + self.output_files = [Path(output_file) for output_file in output_files or []] + self.output_pdb_files = list( + dict.fromkeys( + [ + *( + Path(output_pdb_file) + for output_pdb_file in output_pdb_files or [] + ), + *( + output_file + for output_file in self.output_files + if output_file.suffix.lower() == ".pdb" + ), + ] + ) + ) + for output_pdb_file in self.output_pdb_files: + if output_pdb_file not in self.output_files: + self.output_files.append(output_pdb_file) + self._pending_partial_outputs: dict[Path, Path] = {} + self._assert_declared_output_bindings() + self._declared_input_script = self._input_script() def __repr__(self) -> str: _input_file = self.input_file @@ -201,63 +238,226 @@ def run( ``False``. """ - if isinstance(self.input_file, str): - p = subprocess.Popen( - self.cns_exec, - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - close_fds=True, - env=self.envvars, - ) - out, error = p.communicate(input=self.input_file.encode()) - p.kill() - - elif isinstance(self.input_file, Path) and self.output_file is not None: - with open(self.input_file) as inp: - p = subprocess.Popen( - self.cns_exec, - stdin=inp, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - close_fds=True, - env=self.envvars, - ) - out, error = p.communicate() - p.kill() - # Write out file - with open(self.output_file, "wb+") as outf: - outf.write(out) - - if compress_inp: - gzip_files(self.input_file, remove_original=True) - - if compress_out: - gzip_files(self.output_file, remove_original=True) + script = self.prepare_execution_input() - if compress_seed: - with suppress(FileNotFoundError): - gzip_files( - Path(Path(self.output_file).stem).with_suffix(".seed"), - remove_original=True, - ) + p = subprocess.Popen( + self.cns_exec, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + close_fds=True, + env=self.envvars, + cwd=self.work_dir, + ) + out, error = p.communicate(input=script.encode()) + p.kill() - # If undetected error or detect an error in the STDOUT - if error or self.contains_cns_stdout_error(out): + # GNU Fortran emits this note for otherwise complete CNS calculations. + error = (error or b"").replace(CNS_DENORMAL_STDERR, b"").strip() + failed = bool(error) or self.contains_cns_stdout_error(out) + if isinstance(p.returncode, int) and p.returncode != 0: + failed = True + error = error or f"CNS exited with status {p.returncode}".encode() + if failed: # Write .err file (only if an error file was provided, otherwise # the diagnostic raise below would be masked by a TypeError) if self.error_file is not None: - with open(self.error_file, "wb+") as errf: + error_file = self._output_path(Path(self.error_file)) + with open(error_file, "wb+") as errf: errf.write(out) # Compress it if compress_err: - gzip_files(self.error_file, remove_original=True) - if error: - raise CNSRunningError(error) + gzip_files( + error_file, + remove_original=True, + ) + self._discard_partial_outputs() + raise CNSRunningError(error or out) + + self.publish_outputs() + + if isinstance(self.input_file, Path) and compress_inp: + gzip_files(self._output_path(self.input_file), remove_original=True) + if isinstance(self.input_file, Path) and self.output_file is not None: + output_file = self._output_path(Path(self.output_file)) + output_file.write_bytes(out) + if compress_out: + gzip_files(output_file, remove_original=True) + if compress_seed: + with suppress(FileNotFoundError): + seed_file = output_file.with_suffix(".seed") + if not seed_file.exists(): + seed_file = self.work_dir / seed_file.name + gzip_files( + seed_file, + remove_original=True, + ) # Return STDOUT return out + def normalize_outputs(self) -> None: + """Normalize known CNS-generated outputs.""" + self._normalize_paths( + self._output_path(output_file) for output_file in self.output_files + ) + + @staticmethod + def _normalize_paths(output_files: Iterable[Path]) -> None: + """Normalize the supplied public or hidden CNS artifacts.""" + for output_file in output_files: + suffix = output_file.suffix.lower() + if suffix == ".pdb": + normalize_cns_pdb(output_file) + elif suffix == ".psf": + normalize_cns_psf(output_file) + + def _input_script(self) -> Optional[str]: + """Return the materialized CNS input, if it is available now.""" + if isinstance(self.input_file, str): + return self.input_file + path = self._output_path(self.input_file) + if path.exists(): + return path.read_text(encoding="utf-8") + return None + + def _assert_declared_output_bindings(self) -> None: + """Reject a job whose metadata names outputs other than its CNS script.""" + script = self._input_script() + if script is None: + return + expected = { + ".pdb": self._output_path(next(iter(self.output_pdb_files))).resolve() + if self.output_pdb_files + else None, + ".psf": next( + ( + self._output_path(output).resolve() + for output in self.output_files + if output.suffix.lower() == ".psf" + ), + None, + ), + } + matches = {} + for variable, suffix in ( + ("output_pdb_filename", ".pdb"), + ("output_psf_filename", ".psf"), + ): + match = re.search(rf'\${variable}\s*=\s*"(?P[^"]+)"', script) + if match is None: + continue + matches[suffix] = match + declared = self._output_path(Path(match.group("path"))).resolve() + if expected[suffix] is None or declared != expected[suffix]: + raise ValueError( + f"CNS ${variable}={match.group('path')!r} does not match " + f"the declared output for this job" + ) + if matches: + missing = [ + suffix + for suffix, output in expected.items() + if output and suffix not in matches + ] + if missing: + raise ValueError( + "CNS input does not bind every declared output: " + + ", ".join(missing) + ) + + def _partial_output_script(self) -> tuple[str, dict[Path, Path]]: + """Direct CNS to same-directory staging outputs until they are complete.""" + script = self._declared_input_script or self._input_script() + if script is None: + raise ValueError(f"CNS input {self.input_file!r} does not exist") + self._declared_input_script = script + partial_outputs: dict[Path, Path] = {} + for variable in ("output_pdb_filename", "output_psf_filename"): + match = re.search(rf'\${variable}\s*=\s*"(?P[^"]+)"', script) + if match is None: + continue + public = self._output_path(Path(match.group("path"))).resolve() + partial = public.with_name(f"{public.stem}.partial{public.suffix}") + partial_outputs[public] = partial + partial_name = str(Path(match.group("path")).with_name(partial.name)) + script = ( + script[: match.start("path")] + + partial_name + + script[match.end("path") :] + ) + return script, partial_outputs + + def prepare_execution_input(self) -> str: + """Return and retain the exact CNS script that will be executed. + + Batch backends call this before writing their temporary input file; the + matching :meth:`publish_outputs` call then validates and publishes the + complete output set after the process has succeeded. Path-backed inputs + are updated so the retained ``.inp`` and CNS stdout describe the same + execution. + """ + script, partial_outputs = self._partial_output_script() + for partial in partial_outputs.values(): + partial.unlink(missing_ok=True) + self._pending_partial_outputs = partial_outputs + if isinstance(self.input_file, Path): + self._output_path(self.input_file).write_text(script, encoding="utf-8") + return script + + def publish_outputs(self, check_output_log: bool = False) -> None: + """Publish a prepared complete output set, or normalize legacy outputs.""" + if check_output_log and self.output_file is not None: + output_file = self._output_path(Path(self.output_file)) + if output_file.exists() and self.contains_cns_stdout_error( + output_file.read_bytes() + ): + self._discard_partial_outputs() + raise CNSRunningError(f"CNS reported an error in {output_file}") + if self._pending_partial_outputs: + self._publish_partial_outputs(self._pending_partial_outputs) + self._pending_partial_outputs = {} + else: + self.normalize_outputs() + + def _publish_partial_outputs(self, partial_outputs: dict[Path, Path]) -> None: + """Validate and atomically publish a complete normalized output set.""" + missing = [ + path + for path in partial_outputs.values() + if not path.is_file() or path.stat().st_size == 0 + ] + if missing: + self._discard_partial_outputs(partial_outputs) + raise CNSRunningError( + "CNS did not produce complete outputs: " + + ", ".join(str(path) for path in missing) + ) + self._normalize_paths(partial_outputs.values()) + for public, partial in partial_outputs.items(): + os.replace(partial, public) + + def _discard_partial_outputs( + self, partial_outputs: Optional[dict[Path, Path]] = None + ) -> None: + """Remove unpublished CNS artifacts after a failed or retried job.""" + pending = partial_outputs or self._pending_partial_outputs + for partial in pending.values(): + partial.unlink(missing_ok=True) + self._pending_partial_outputs = {} + + def _output_path(self, output_file: Path) -> Path: + """Resolve job outputs relative to the directory that created the job.""" + if output_file.is_absolute(): + return output_file + return self.work_dir / output_file + + def canonical_mapping(self): + """Return the canonical CNS input representation for this job.""" + from haddock.libs.libcnscanonical import canonical_mapping_for_job + + return canonical_mapping_for_job(self) + @staticmethod def contains_cns_stdout_error(out: bytes) -> bool: # Decode end of STDOUT diff --git a/src/haddock/modules/analysis/caprieval/__init__.py b/src/haddock/modules/analysis/caprieval/__init__.py index 5d9c5a4ea1..8d8933b280 100644 --- a/src/haddock/modules/analysis/caprieval/__init__.py +++ b/src/haddock/modules/analysis/caprieval/__init__.py @@ -124,7 +124,14 @@ def _run(self) -> None: # Get reference file if ff == "martini2": references = [ - Path(martinize(ref_aa, self.path.resolve().parent, False)) + Path( + martinize( + ref_aa, + self.path.resolve().parent, + False, + seed=0, + ) + ) for ref_aa in self.get_reference(models) ] else: diff --git a/src/haddock/modules/analysis/caprifilter/__init__.py b/src/haddock/modules/analysis/caprifilter/__init__.py index b628463174..e46650ccc8 100644 --- a/src/haddock/modules/analysis/caprifilter/__init__.py +++ b/src/haddock/modules/analysis/caprifilter/__init__.py @@ -135,7 +135,14 @@ def _run(self) -> None: ff = find_ff(models) if ff == "martini2": references = [ - Path(martinize(ref, self.path.resolve().parent, False)) + Path( + martinize( + ref, + self.path.resolve().parent, + False, + seed=0, + ) + ) for ref in references ] diff --git a/src/haddock/modules/base_cns_module.py b/src/haddock/modules/base_cns_module.py index d1d86c89e5..e3e4a53702 100644 --- a/src/haddock/modules/base_cns_module.py +++ b/src/haddock/modules/base_cns_module.py @@ -1,6 +1,7 @@ """Functionalities related to CNS modules.""" import os +import re import shutil import tarfile from pathlib import Path @@ -8,12 +9,16 @@ from haddock import log from haddock import toppar_path as global_toppar from haddock.core.defaults import cns_exec as global_cns_exec -from haddock.core.typing import Any, FilePath, Optional, Union +from haddock.core.typing import Any, FilePath, Optional, ParamDict, Union from haddock.gear.expandable_parameters import populate_mol_parameters_in_module from haddock.libs.libio import working_directory from haddock.modules import BaseHaddockModule +_CNS_VARIABLE_PATTERN = re.compile(r"\$([A-Za-z0-9_]+)") +_CNS_SPLICED_VARIABLE_PREFIX_PATTERN = re.compile(r"\$([A-Za-z][A-Za-z0-9_]*?)(?=\$)") + + class BaseCNSModule(BaseHaddockModule): """ Operation module for CNS. @@ -21,6 +26,10 @@ class BaseCNSModule(BaseHaddockModule): Contains additional functionalities excusive for CNS modules. """ + # ``ncs_*`` values are consumed by CNS' built-in NCS data structure rather + # than referenced directly by the module recipe tree. + CNS_PARAM_INCLUDE_PREFIXES = ("mol_", "fle_", "ncs_") + def __init__( self, order: int, path: Path, initial_params: FilePath, cns_script: FilePath ) -> None: @@ -38,6 +47,28 @@ def __init__( self.cns_protocol_path = Path(cns_script) self.toppar_path = global_toppar self.recipe_str = self.cns_protocol_path.read_text() + recipe_texts = tuple( + recipe.read_text(encoding="utf-8") + for recipe in self.cns_folder_path.rglob("*.cns") + ) + self._cns_recipe_variables = frozenset( + variable + for recipe_text in recipe_texts + for variable in _CNS_VARIABLE_PATTERN.findall(recipe_text) + ) + self._cns_spliced_variable_prefixes = tuple( + sorted( + { + prefix + for recipe_text in recipe_texts + for prefix in _CNS_SPLICED_VARIABLE_PREFIX_PATTERN.findall( + recipe_text + ) + }, + key=len, + reverse=True, + ) + ) def run(self, **params: Any) -> None: """Execute the module.""" @@ -74,6 +105,30 @@ def default_envvars(self) -> dict[str, str]: return default_envvars + def cns_params(self, params: Optional[ParamDict] = None) -> ParamDict: + """Return only parameters that a CNS recipe can consume. + + The inclusion rule is deliberately derived from the module's CNS recipe + tree rather than maintained as a deny-list of Python orchestration + settings. New scheduler or module-control parameters therefore cannot + silently become part of a CNS job identity. + """ + source = self.params if params is None else params + return { + key: value + for key, value in source.items() + if key.rstrip() in self._cns_recipe_variables + or key.startswith(self.CNS_PARAM_INCLUDE_PREFIXES) + or self._matches_cns_spliced_variable(key.rstrip()) + } + + def _matches_cns_spliced_variable(self, parameter: str) -> bool: + """Return whether a parameter can be assembled by a CNS symbol splice.""" + return any( + re.fullmatch(rf"{re.escape(prefix)}\d+(?:_\d+)*", parameter) + for prefix in self._cns_spliced_variable_prefixes + ) + def save_envvars(self, filename: FilePath = "envvars") -> None: """Save envvars needed for CNS to a file in the module's folder.""" # there are so few variables, best to handle them by hand diff --git a/src/haddock/modules/refinement/cgtoaa/__init__.py b/src/haddock/modules/refinement/cgtoaa/__init__.py index 1aad278ad9..d79286418a 100644 --- a/src/haddock/modules/refinement/cgtoaa/__init__.py +++ b/src/haddock/modules/refinement/cgtoaa/__init__.py @@ -16,7 +16,6 @@ from haddock.core.typing import FilePath from haddock.gear.haddockmodel import HaddockModel from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb -from haddock.libs.libontology import PDBFile from haddock.libs.libsubprocess import CNSJob from haddock.modules import get_engine from haddock.modules.base_cns_module import BaseCNSModule @@ -30,7 +29,6 @@ class HaddockModule(BaseCNSModule): """HADDOCK3 module energy minimization refinement.""" name = RECIPE_PATH.name - def __init__( self, order: int, path: Path, initial_params: FilePath = DEFAULT_CONFIG ) -> None: @@ -69,11 +67,9 @@ def _run(self) -> None: # Pool of jobs to be executed by the CNS engine jobs: list[CNSJob] = [] + cns_params = self.cns_params() idx = 1 for model in models_to_refine: - if isinstance(model, PDBFile): - if not model.seed: - model.seed = self.params["iniseed"] for s_ind in range(sampling_factor): # Prepare CNS input cgtoaa_input = prepare_cns_input( @@ -81,19 +77,15 @@ def _run(self) -> None: model, self.path, self.recipe_str, - self.params, + cns_params, self.name, native_segid=True, debug=self.params["debug"], - seed=(model.seed + s_ind) if isinstance(model, PDBFile) else None, cgtoaa=True, ) # Build CNS Job out_file = f"{self.name}_{idx}.out" err_fname = f"{self.name}_{idx}.cnserr" - job = CNSJob(cgtoaa_input, out_file, err_fname, envvars=self.envvars) - jobs.append(job) - # create the expected PDBobject expected_pdb = prepare_expected_pdb(model, idx, ".", self.name) expected_pdb.topology = expected_pdb.aa_topology @@ -103,6 +95,15 @@ def _run(self) -> None: expected_pdb.ori_name = None self.output_models.append(expected_pdb) + job = CNSJob( + cgtoaa_input, + out_file, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) + jobs.append(job) + idx += 1 # Run CNS Jobs diff --git a/src/haddock/modules/refinement/emref/__init__.py b/src/haddock/modules/refinement/emref/__init__.py index dd15d68e5c..59adaaed4b 100644 --- a/src/haddock/modules/refinement/emref/__init__.py +++ b/src/haddock/modules/refinement/emref/__init__.py @@ -15,8 +15,12 @@ from haddock.core.defaults import MODULE_DEFAULT_YAML from haddock.core.typing import FilePath from haddock.gear.haddockmodel import HaddockModel -from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb -from haddock.libs.libontology import PDBFile +from haddock.libs.libcns import ( + derive_seed, + prepare_cns_input, + prepare_expected_pdb, + refinement_schedule, +) from haddock.libs.libsubprocess import CNSJob from haddock.modules import get_engine from haddock.modules.base_cns_module import BaseCNSModule @@ -30,7 +34,6 @@ class HaddockModule(BaseCNSModule): """HADDOCK3 module energy minimization refinement.""" name = RECIPE_PATH.name - def __init__( self, order: int, path: Path, initial_params: FilePath = DEFAULT_CONFIG ) -> None: @@ -81,47 +84,57 @@ def _run(self) -> None: prev_ambig_fnames = [None for model in models_to_refine] ambig_fnames = self.get_ambig_fnames(prev_ambig_fnames) - - model_idx = 0 - idx = 1 - for model in models_to_refine: + cns_params = self.cns_params() + + # Replicas are emitted in rounds -- every model gets its first replica + # before any model gets its second -- so raising `sampling_factor` + # appends model numbers instead of renumbering the models of every + # input after the first. + schedule = refinement_schedule(len(models_to_refine), sampling_factor) + for idx, (model_idx, replica) in enumerate(schedule, start=1): + model = models_to_refine[model_idx] # assign ambig_fname if ambig_fnames: ambig_fname = ambig_fnames[model_idx] else: ambig_fname = self.params["ambig_fname"] - model_idx += 1 - - for _ in range(self.params["sampling_factor"]): - emref_input = prepare_cns_input( - idx, - model, - self.path, - self.recipe_str, - self.params, - "emref", - ambig_fname=ambig_fname, - native_segid=True, - debug=self.params["debug"], - seed=model.seed if isinstance(model, PDBFile) else None, - ) - out_file = f"emref_{idx}.out" - err_fname = f"emref_{idx}.cnserr" - - # create the expected PDBobject - expected_pdb = prepare_expected_pdb(model, idx, ".", "emref") - expected_pdb.restr_fname = ambig_fname - try: - expected_pdb.ori_name = model.file_name - except AttributeError: - expected_pdb.ori_name = None - self.output_models.append(expected_pdb) - - job = CNSJob(emref_input, out_file, err_fname, envvars=self.envvars) - - jobs.append(job) - - idx += 1 + + seed = derive_seed(self.params["iniseed"], model, replica) + emref_input = prepare_cns_input( + idx, + model, + self.path, + self.recipe_str, + cns_params, + "emref", + ambig_fname=ambig_fname, + native_segid=True, + debug=self.params["debug"], + seed=seed, + ) + + out_file = f"emref_{idx}.out" + err_fname = f"emref_{idx}.cnserr" + + # create the expected PDBobject + expected_pdb = prepare_expected_pdb(model, idx, ".", "emref") + expected_pdb.seed = seed + expected_pdb.restr_fname = ambig_fname + try: + expected_pdb.ori_name = model.file_name + except AttributeError: + expected_pdb.ori_name = None + self.output_models.append(expected_pdb) + + job = CNSJob( + emref_input, + out_file, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) + + jobs.append(job) # Run CNS Jobs self.log(f"Running CNS Jobs n={len(jobs)}") diff --git a/src/haddock/modules/refinement/emref/defaults.yaml b/src/haddock/modules/refinement/emref/defaults.yaml index 4a0d739dd8..151afc134d 100644 --- a/src/haddock/modules/refinement/emref/defaults.yaml +++ b/src/haddock/modules/refinement/emref/defaults.yaml @@ -1360,7 +1360,7 @@ sampling_factor: max: 500 title: Sampling factor for each starting model short: This parameter control how many times a model will be refined. - long: This parameter control how many times a model will be refined. For EM refinement only it does not make sense to increase it unless random removal of restraints is turned on. If not then the energy minimisation will lead to the same final conformation. + long: This parameter control how many times a model will be refined. For EM refinement only it does not make sense to increase it unless random removal of restraints is turned on. If not then the energy minimisation will lead to the same final conformation. The replicas are emitted in rounds - every model is refined once before any model is refined a second time - so raising this value appends model numbers rather than renumbering them. Each replica carries its own seed. group: 'sampling' explevel: expert max_nmodels: diff --git a/src/haddock/modules/refinement/flexref/__init__.py b/src/haddock/modules/refinement/flexref/__init__.py index f06075966e..408436af47 100644 --- a/src/haddock/modules/refinement/flexref/__init__.py +++ b/src/haddock/modules/refinement/flexref/__init__.py @@ -29,7 +29,12 @@ from haddock.core.defaults import MODULE_DEFAULT_YAML from haddock.core.typing import FilePath from haddock.gear.haddockmodel import HaddockModel -from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb +from haddock.libs.libcns import ( + derive_seed, + prepare_cns_input, + prepare_expected_pdb, + refinement_schedule, +) from haddock.libs.libontology import PDBFile from haddock.libs.libsubprocess import CNSJob from haddock.modules import get_engine @@ -44,7 +49,6 @@ class HaddockModule(BaseCNSModule): """HADDOCK3 module for flexible refinement.""" name = RECIPE_PATH.name - def __init__( self, order: int, path: Path, initial_params: FilePath = DEFAULT_CONFIG ) -> None: @@ -94,47 +98,57 @@ def _run(self) -> None: prev_ambig_fnames = [None for model in models_to_refine] ambig_fnames = self.get_ambig_fnames(prev_ambig_fnames) - - idx = 1 - for model_idx, model in enumerate(models_to_refine): + cns_params = self.cns_params() + + # Replicas are emitted in rounds -- every model gets its first replica + # before any model gets its second -- so raising `sampling_factor` + # appends model numbers instead of renumbering the models of every + # input after the first. + schedule = refinement_schedule(len(models_to_refine), sampling_factor) + for idx, (model_idx, replica) in enumerate(schedule, start=1): + model = models_to_refine[model_idx] # assign ambig_fname if ambig_fnames: ambig_fname = ambig_fnames[model_idx] else: ambig_fname = self.params["ambig_fname"] - for _ in range(sampling_factor): - # prepare cns input - flexref_input = prepare_cns_input( - idx, - model, - self.path, - self.recipe_str, - self.params, - self.name, - ambig_fname=ambig_fname, - native_segid=True, - debug=self.params["debug"], - seed=model.seed if isinstance(model, PDBFile) else None, - ) - - out_file = f"{self.name}_{idx}.out" - err_fname = f"{self.name}_{idx}.cnserr" - - # create the expected PDBobject - expected_pdb = prepare_expected_pdb(model, idx, ".", self.name) - expected_pdb.restr_fname = ambig_fname - try: - expected_pdb.ori_name = model.file_name - except AttributeError: - expected_pdb.ori_name = None - self.output_models.append(expected_pdb) - - job = CNSJob(flexref_input, out_file, err_fname, envvars=self.envvars) - - jobs.append(job) - - idx += 1 + seed = derive_seed(self.params["iniseed"], model, replica) + flexref_input = prepare_cns_input( + idx, + model, + self.path, + self.recipe_str, + cns_params, + self.name, + ambig_fname=ambig_fname, + native_segid=True, + debug=self.params["debug"], + seed=seed, + ) + + out_file = f"flexref_{idx}.out" + err_fname = f"flexref_{idx}.cnserr" + + # create the expected PDBobject + expected_pdb = prepare_expected_pdb(model, idx, ".", self.name) + expected_pdb.seed = seed + expected_pdb.restr_fname = ambig_fname + try: + expected_pdb.ori_name = model.file_name + except AttributeError: + expected_pdb.ori_name = None + self.output_models.append(expected_pdb) + + job = CNSJob( + flexref_input, + out_file, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) + + jobs.append(job) # Run CNS Jobs self.log(f"Running CNS Jobs n={len(jobs)}") diff --git a/src/haddock/modules/refinement/flexref/defaults.yaml b/src/haddock/modules/refinement/flexref/defaults.yaml index 597fb1a43d..5cdf3591fc 100644 --- a/src/haddock/modules/refinement/flexref/defaults.yaml +++ b/src/haddock/modules/refinement/flexref/defaults.yaml @@ -1697,7 +1697,7 @@ sampling_factor: max: 500 title: Sampling factor for each starting model short: This paramater controls how many times a model will be refined. - long: This paramater controls how many times a model will be refined. + long: This paramater controls how many times a model will be refined. The replicas are emitted in rounds - every model is refined once before any model is refined a second time - so raising this value appends model numbers rather than renumbering them. Each replica carries its own seed, so it is additional sampling of the same starting structure rather than a repeat of the same calculation. group: 'sampling' explevel: expert max_nmodels: diff --git a/src/haddock/modules/refinement/mdref/__init__.py b/src/haddock/modules/refinement/mdref/__init__.py index f1e05a2eea..c3fb5d2c72 100644 --- a/src/haddock/modules/refinement/mdref/__init__.py +++ b/src/haddock/modules/refinement/mdref/__init__.py @@ -28,7 +28,12 @@ from haddock.core.defaults import MODULE_DEFAULT_YAML from haddock.core.typing import FilePath from haddock.gear.haddockmodel import HaddockModel -from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb +from haddock.libs.libcns import ( + derive_seed, + prepare_cns_input, + prepare_expected_pdb, + refinement_schedule, +) from haddock.libs.libontology import PDBFile from haddock.libs.libsubprocess import CNSJob from haddock.modules import get_engine @@ -43,7 +48,6 @@ class HaddockModule(BaseCNSModule): """HADDOCK3 module for water refinement.""" name = RECIPE_PATH.name - def __init__( self, order: int, path: Path, initial_params: FilePath = DEFAULT_CONFIG ) -> None: @@ -94,47 +98,57 @@ def _run(self) -> None: prev_ambig_fnames = [None for mod in models_to_refine] ambig_fnames = self.get_ambig_fnames(prev_ambig_fnames) - - model_idx = 0 - idx = 1 - for model in models_to_refine: + cns_params = self.cns_params() + + # Replicas are emitted in rounds -- every model gets its first replica + # before any model gets its second -- so raising `sampling_factor` + # appends model numbers instead of renumbering the models of every + # input after the first. + schedule = refinement_schedule(len(models_to_refine), sampling_factor) + for idx, (model_idx, replica) in enumerate(schedule, start=1): + model = models_to_refine[model_idx] # assign ambig_fname if ambig_fnames: ambig_fname = ambig_fnames[model_idx] else: ambig_fname = self.params["ambig_fname"] - model_idx += 1 - - for _ in range(self.params["sampling_factor"]): - mdref_input = prepare_cns_input( - idx, - model, - self.path, - self.recipe_str, - self.params, - "mdref", - ambig_fname=ambig_fname, - native_segid=True, - debug=self.params["debug"], - seed=model.seed if isinstance(model, PDBFile) else None, - ) - out_file = f"mdref_{idx}.out" - err_fname = f"mdref_{idx}.cnserr" - - # create the expected PDBobject - expected_pdb = prepare_expected_pdb(model, idx, ".", "mdref") - expected_pdb.restr_fname = ambig_fname - try: - expected_pdb.ori_name = model.file_name - except AttributeError: - expected_pdb.ori_name = None - self.output_models.append(expected_pdb) - - job = CNSJob(mdref_input, out_file, err_fname, envvars=self.envvars) - - jobs.append(job) - - idx += 1 + + seed = derive_seed(self.params["iniseed"], model, replica) + mdref_input = prepare_cns_input( + idx, + model, + self.path, + self.recipe_str, + cns_params, + "mdref", + ambig_fname=ambig_fname, + native_segid=True, + debug=self.params["debug"], + seed=seed, + ) + + out_file = f"mdref_{idx}.out" + err_fname = f"mdref_{idx}.cnserr" + + # create the expected PDBobject + expected_pdb = prepare_expected_pdb(model, idx, ".", "mdref") + expected_pdb.seed = seed + expected_pdb.restr_fname = ambig_fname + try: + expected_pdb.ori_name = model.file_name + except AttributeError: + expected_pdb.ori_name = None + self.output_models.append(expected_pdb) + + job = CNSJob( + mdref_input, + out_file, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) + + jobs.append(job) # Run CNS Jobs self.log(f"Running CNS Jobs n={len(jobs)}") diff --git a/src/haddock/modules/refinement/mdref/defaults.yaml b/src/haddock/modules/refinement/mdref/defaults.yaml index 74dad51eb2..a6e5bf5de1 100644 --- a/src/haddock/modules/refinement/mdref/defaults.yaml +++ b/src/haddock/modules/refinement/mdref/defaults.yaml @@ -1388,7 +1388,7 @@ sampling_factor: max: 500 title: Sampling factor for each starting model short: This parameter control how many times a model will be refined. - long: This parameter control how many times a model will be refined. + long: This parameter control how many times a model will be refined. The replicas are emitted in rounds - every model is refined once before any model is refined a second time - so raising this value appends model numbers rather than renumbering them. Each replica carries its own seed, so it is additional sampling of the same starting structure rather than a repeat of the same calculation. group: 'sampling' explevel: expert max_nmodels: diff --git a/src/haddock/modules/sampling/rigidbody/__init__.py b/src/haddock/modules/sampling/rigidbody/__init__.py index 1af1f82899..99e5ae17f7 100644 --- a/src/haddock/modules/sampling/rigidbody/__init__.py +++ b/src/haddock/modules/sampling/rigidbody/__init__.py @@ -37,7 +37,7 @@ from haddock.core.defaults import MODULE_DEFAULT_YAML from haddock.core.typing import FilePath, Optional, Sequence, Union from haddock.gear.haddockmodel import HaddockModel -from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb +from haddock.libs.libcns import derive_seed, prepare_cns_input, prepare_expected_pdb from haddock.libs.libontology import PDBFile from haddock.libs.libparallel import GenericTask from haddock.libs.libpdb import check_combination_chains @@ -50,6 +50,41 @@ DEFAULT_CONFIG = Path(RECIPE_PATH, MODULE_DEFAULT_YAML) +def _chainids_for_sampled_combinations( + combinations: Sequence[list[PDBFile]], + sampled_combinations: Sequence[list[PDBFile]], +) -> list[list[str]]: + """Resolve chain IDs once for each distinct source combination.""" + chainids_by_combination = { + id(combination): check_combination_chains(combination) + for combination in combinations + } + return [ + chainids_by_combination[id(combination)] for combination in sampled_combinations + ] + + +def _repeats_of_sampled_combinations( + sampled_combinations: Sequence[list[PDBFile]], +) -> list[int]: + """How many times each sampled combination has already been scheduled. + + A job's seed is derived from the combination it docks and from which + repeat of that combination it is, never from where it sits in the + schedule, so this is counted per combination rather than taken from the + job index. Counting it from the schedule itself rather than computing + ``index // n_combinations`` keeps the two in step by construction if the + round-robin is ever replaced by another prefix-stable order. + """ + seen: dict[int, int] = {} + repeats = [] + for combination in sampled_combinations: + repeat = seen.get(id(combination), 0) + repeats.append(repeat) + seen[id(combination)] = repeat + 1 + return repeats + + class HaddockModule(BaseCNSModule): """HADDOCK3 module for rigid body sampling.""" @@ -82,96 +117,129 @@ def make_cns_jobs( # Create a model for the expected output model = prepare_expected_pdb(combination, idx, ".", self.name) # Set additional attributes - model.restr_fname=str(ambig_fname) + model.restr_fname = str(ambig_fname) model.seed = seed self.output_models.append(model) - job = CNSJob(cns_input, log_fname, err_fname, envvars=self.envvars) + job = CNSJob( + cns_input, + log_fname, + err_fname, + envvars=self.envvars, + output_pdb_files=[model.file_name], + ) jobs.append(job) return jobs + def _cns_default_params(self) -> dict: + """Return rigidbody CNS defaults that affect per-job computation.""" + return self.cns_params() + + @staticmethod + def _sample_models_to_dock( + models_to_dock: list[list[PDBFile]], + sampling: int, + ) -> list[list[PDBFile]]: + """Return a prefix-stable model-combination schedule.""" + return [ + models_to_dock[model_idx % len(models_to_dock)] + for model_idx in range(sampling) + ] + def prepare_cns_input_sequential( self, models_to_dock: list[list[PDBFile]], - sampling_factor: int, ambig_fnames: Union[list, None], + chainid_lists: Optional[list[list[str]]] = None, ) -> list[tuple[list[PDBFile], Union[Path, str], Union[str, None], int]]: _l = [] + cns_params = self._cns_default_params() + repeats = _repeats_of_sampled_combinations(models_to_dock) idx = 1 - for combination in models_to_dock: - for _ in range(sampling_factor): - # assign ambig_fname - if ambig_fnames: - ambig_fname = ambig_fnames[idx - 1] - else: - ambig_fname = self.params["ambig_fname"] - # prepare cns input - seed = self.params["iniseed"] + idx - rigidbody_input = prepare_cns_input( - idx, - combination, - self.path, - self.recipe_str, - self.params, - self.name, - ambig_fname=ambig_fname, - default_params_path=self.toppar_path, - native_segid=True, - debug=self.params["debug"], - seed=seed, - ) - _l.append((combination, rigidbody_input, ambig_fname, seed)) - - idx += 1 + for combination_index, combination in enumerate(models_to_dock): + # assign ambig_fname + if ambig_fnames: + ambig_fname = ambig_fnames[idx - 1] + else: + ambig_fname = self.params["ambig_fname"] + # prepare cns input + seed = derive_seed( + self.params["iniseed"], + combination, + repeats[combination_index], + ) + rigidbody_input = prepare_cns_input( + idx, + combination, + self.path, + self.recipe_str, + cns_params, + self.name, + ambig_fname=ambig_fname, + default_params_path=self.toppar_path, + native_segid=True, + debug=self.params["debug"], + seed=seed, + chainid_list=( + chainid_lists[combination_index] if chainid_lists else None + ), + ) + _l.append((combination, rigidbody_input, ambig_fname, seed)) + idx += 1 return _l def prepare_cns_input_parallel( self, models_to_dock: list[list[PDBFile]], - sampling_factor: int, ambig_fnames: Union[list, None], + chainid_lists: Optional[list[list[str]]] = None, ) -> list[tuple[list[PDBFile], Union[Path, str], Union[str, None], int]]: prepare_tasks = [] _l = [] + cns_params = self._cns_default_params() + repeats = _repeats_of_sampled_combinations(models_to_dock) idx: int = 1 - for ci, combination in enumerate(models_to_dock): - check_combination_chains(combination) - for _ in range(sampling_factor): - ambig_fname = ( - ambig_fnames[idx - 1] - if ambig_fnames - else self.params["ambig_fname"] - ) - seed = self.params["iniseed"] + idx - task = GenericTask( - function=prepare_cns_input, - model_number=idx, - input_element=combination, - step_path=self.path, - recipe_str=self.recipe_str, - defaults=self.params, - identifier=self.name, - ambig_fname=ambig_fname, - native_segid=True, - default_params_path=self.toppar_path, - debug=self.params["debug"], - seed=seed, - ) - - prepare_tasks.append(task) - _l.append((combination, task, ambig_fname, seed)) - idx += 1 + for combination_index, combination in enumerate(models_to_dock): + ambig_fname = ( + ambig_fnames[idx - 1] if ambig_fnames else self.params["ambig_fname"] + ) + seed = derive_seed( + self.params["iniseed"], + combination, + repeats[combination_index], + ) + task = GenericTask( + function=prepare_cns_input, + model_number=idx, + input_element=combination, + step_path=self.path, + recipe_str=self.recipe_str, + defaults=cns_params, + identifier=self.name, + ambig_fname=ambig_fname, + native_segid=True, + default_params_path=self.toppar_path, + debug=self.params["debug"], + seed=seed, + chainid_list=( + chainid_lists[combination_index] if chainid_lists else None + ), + ) + + prepare_tasks.append(task) + _l.append((combination, task, ambig_fname, seed)) + idx += 1 Engine = get_engine(self.params["mode"], self.params) prepare_engine = Engine(prepare_tasks) prepare_engine.run() # Replace the task with the result of the task - l = [] + prepared_inputs = [] for element, task_result in zip(_l, prepare_engine.results): - l.append((element[0], task_result, element[2], element[3])) + prepared_inputs.append((element[0], task_result, element[2], element[3])) - return l + return prepared_inputs def restraints_guardrail(self, ambig_fnames: Optional[list[str]]) -> None: """Makes sure any restraints are available for the docking.""" @@ -212,16 +280,24 @@ def _run(self) -> None: except Exception as e: self.finish_with_error(e) - # How many times each combination should be sampled, - # cannot be smaller than 1 - sampling_factor = int(self.params["sampling"] / len(models_to_dock)) - if sampling_factor < 1: + # Each model combination should be sampled at least once. The global + # sampling sequence itself is prefix-stable: lowering sampling by one + # keeps all preceding CNS job inputs and seeds unchanged. + if self.params["sampling"] < len(models_to_dock): self.finish_with_error( "Sampling is smaller than the number" " of model combinations " f"#model_combinations={len(models_to_dock)}," f" sampling={self.params['sampling']}." ) + sampled_models_to_dock = self._sample_models_to_dock( + models_to_dock, + self.params["sampling"], + ) + sampled_chainid_lists = _chainids_for_sampled_combinations( + models_to_dock, + sampled_models_to_dock, + ) # get all the different ambig files prev_ambig_fnames = [None for _model in range(self.params["sampling"])] @@ -241,15 +317,15 @@ def _run(self) -> None: if self.params["mode"] != "local": # Note: `batch` and (pseudo)-`mpi` mode uses files to communicate and cannot extract the information from the task object. cns_input = self.prepare_cns_input_sequential( - models_to_dock, - sampling_factor, + sampled_models_to_dock, ambig_fnames, # type: ignore + sampled_chainid_lists, ) else: cns_input = self.prepare_cns_input_parallel( - models_to_dock, - sampling_factor, + sampled_models_to_dock, ambig_fnames, # type: ignore + sampled_chainid_lists, ) self.output_models: list[PDBFile] = [] diff --git a/src/haddock/modules/sampling/rigidbody/cns/rigidbody.cns b/src/haddock/modules/sampling/rigidbody/cns/rigidbody.cns index 0a32a25a80..3ebcb9f6a3 100644 --- a/src/haddock/modules/sampling/rigidbody/cns/rigidbody.cns +++ b/src/haddock/modules/sampling/rigidbody/cns/rigidbody.cns @@ -31,8 +31,6 @@ evaluate ($saprotocol.ntrials=$ntrials) evaluate ($saprotocol.inter_rigid=$inter_rigid) evaluate ($saprotocol.rotate180=$rotate180) -evaluate ($ini_count =1) -evaluate ($structures =$sampling) evaluate ($w_vdw =$w_vdw) evaluate ($w_elec =$w_elec) evaluate ($w_air =$w_air) diff --git a/src/haddock/modules/sampling/rigidbody/defaults.yaml b/src/haddock/modules/sampling/rigidbody/defaults.yaml index d54fc1d6af..4e2b9c072b 100644 --- a/src/haddock/modules/sampling/rigidbody/defaults.yaml +++ b/src/haddock/modules/sampling/rigidbody/defaults.yaml @@ -817,7 +817,7 @@ sampling: max: 50000 title: Number of models to generate short: Number of rigidbody docking models to generate - long: Number of rigidbody docking models to generate + long: Number of rigidbody docking models to generate. Models are assigned round-robin across input combinations; exactly this many jobs are run. group: 'sampling' explevel: easy crossdock: diff --git a/src/haddock/modules/scoring/emscoring/__init__.py b/src/haddock/modules/scoring/emscoring/__init__.py index c996220467..04709de42e 100644 --- a/src/haddock/modules/scoring/emscoring/__init__.py +++ b/src/haddock/modules/scoring/emscoring/__init__.py @@ -54,17 +54,17 @@ def _run(self) -> None: # Itereate over models to prepare CNS inputs self.output_models = [] + cns_params = self.cns_params() for model_num, model in enumerate(models_to_score, start=1): scoring_input = prepare_cns_input( model_num, model, self.path, self.recipe_str, - self.params, + cns_params, self.name, native_segid=True, debug=self.params["debug"], - seed=model.seed if isinstance(model, PDBFile) else None, ) scoring_out = f"{self.name}_{model_num}.out" @@ -79,7 +79,13 @@ def _run(self) -> None: self.output_models.append(expected_pdb) - job = CNSJob(scoring_input, scoring_out, err_fname, envvars=self.envvars) + job = CNSJob( + scoring_input, + scoring_out, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) jobs.append(job) diff --git a/src/haddock/modules/scoring/mdscoring/__init__.py b/src/haddock/modules/scoring/mdscoring/__init__.py index e65b7e492a..c862b9bfc4 100644 --- a/src/haddock/modules/scoring/mdscoring/__init__.py +++ b/src/haddock/modules/scoring/mdscoring/__init__.py @@ -11,9 +11,11 @@ from haddock.core.defaults import MODULE_DEFAULT_YAML from haddock.core.typing import FilePath -from haddock.gear.haddockmodel import HaddockModel -from haddock.libs.libcns import prepare_cns_input, prepare_expected_pdb -from haddock.libs.libontology import PDBFile +from haddock.libs.libcns import ( + derive_seed, + prepare_cns_input, + prepare_expected_pdb, +) from haddock.libs.libsubprocess import CNSJob from haddock.modules import get_engine from haddock.modules.scoring import CNSScoringModule @@ -55,17 +57,19 @@ def _run(self) -> None: # Prepare all CNS runs self.output_models = [] + cns_params = self.cns_params() for model_num, model in enumerate(models_to_score, start=1): + seed = derive_seed(self.params["iniseed"], model) scoring_inpyt = prepare_cns_input( model_num, model, self.path, self.recipe_str, - self.params, + cns_params, self.name, native_segid=True, debug=self.params["debug"], - seed=model.seed if isinstance(model, PDBFile) else None, + seed=seed, ) scoring_out = f"{self.name}_{model_num}.out" @@ -73,6 +77,7 @@ def _run(self) -> None: # create the expected PDBobject expected_pdb = prepare_expected_pdb(model, model_num, ".", self.name) + expected_pdb.seed = seed # fill the ori_name field of expected_pdb expected_pdb.ori_name = model.file_name expected_pdb.md5 = model.md5 @@ -80,7 +85,13 @@ def _run(self) -> None: self.output_models.append(expected_pdb) - job = CNSJob(scoring_inpyt, scoring_out, err_fname, envvars=self.envvars) + job = CNSJob( + scoring_inpyt, + scoring_out, + err_fname, + envvars=self.envvars, + output_pdb_files=[expected_pdb.file_name], + ) jobs.append(job) diff --git a/src/haddock/modules/topology/topoaa/__init__.py b/src/haddock/modules/topology/topoaa/__init__.py index de7b433793..a95d8cb1e0 100644 --- a/src/haddock/modules/topology/topoaa/__init__.py +++ b/src/haddock/modules/topology/topoaa/__init__.py @@ -282,7 +282,7 @@ def _run(self) -> None: overwrite=True, custom_topology=custom_top, ) - _params = self.params + _params = self.cns_params() elif self.params["autotoppar"]: # No `ligand_top_fname` was provided, check if there are any unknown molecules @@ -302,7 +302,7 @@ def _run(self) -> None: ) # Inject the automated toppar into the params for module _params = { - **self.params, + **self.cns_params(), "ligand_top_fname": top_path, "ligand_param_fname": par_path, } @@ -326,10 +326,10 @@ def _run(self) -> None: else: self.log("No unknown atoms found") libpdb.sanitize(model, overwrite=True) - _params = self.params + _params = self.cns_params() else: libpdb.sanitize(model, overwrite=True) - _params = self.params + _params = self.cns_params() # Prepare generation of topologies jobs topoaa_input = generate_topology( @@ -346,12 +346,15 @@ def _run(self) -> None: # Add new job to the pool output_filename = Path(f"{model.stem}.{Format.CNS_OUTPUT}") err_fname = f"{model.stem}.cnserr" + output_pdb = Path(f"{model.stem}_haddock.{Format.PDB}") + output_topology = Path(f"{model.stem}_haddock.{Format.TOPOLOGY}") job = CNSJob( topoaa_input, output_filename, err_fname, envvars=self.envvars, cns_exec=cns_exec, + output_files=[output_pdb, output_topology], ) jobs.append(job) diff --git a/src/haddock/modules/topology/topocg/__init__.py b/src/haddock/modules/topology/topocg/__init__.py index 38dd6e7040..fc282e9414 100644 --- a/src/haddock/modules/topology/topocg/__init__.py +++ b/src/haddock/modules/topology/topocg/__init__.py @@ -70,7 +70,12 @@ def generate_topology( if not shape: # AA to CG - cg_pdb_name = martinize(input_pdb, output_path, False) + cg_pdb_name = martinize( + input_pdb, + output_path, + False, + seed=defaults["iniseed"], + ) output = prepare_output( output_pdb_filename=f"{Path(cg_pdb_name).stem}_{force_field}{input_pdb.suffix}", output_psf_filename=f"{Path(cg_pdb_name).stem}_{force_field}.{Format.TOPOLOGY}", @@ -256,7 +261,7 @@ def _run(self) -> None: model, self.path.resolve().parent, self.recipe_str, - self.params, + self.cns_params(), parameters_for_this_molecule, default_params_path=self.toppar_path, write_to_disk=self.params["debug"], @@ -268,12 +273,28 @@ def _run(self) -> None: # Add new job to the pool output_filename = Path(f"{model.stem}.{Format.CNS_OUTPUT}") err_fname = f"{model.stem}.cnserr" + if shape_dic[i]: + output_pdb = Path(f"{model.stem}.{Format.PDB}") + output_topology = Path(f"{model.stem}.{Format.TOPOLOGY}") + else: + output_pdb = Path( + gen_cg_filename("", model.stem, force_field=force_field) + ) + output_topology = Path( + gen_cg_filename( + "", + model.stem, + force_field=force_field, + ext=Format.TOPOLOGY, + ) + ) job = CNSJob( topocg_input, output_filename, err_fname, envvars=self.envvars, cns_exec=cns_exec, + output_files=[output_pdb, output_topology], ) jobs.append(job) diff --git a/tests/golden_data/cns_canonical/cgtoaa.canonical b/tests/golden_data/cns_canonical/cgtoaa.canonical new file mode 100644 index 0000000000..fb26dff025 --- /dev/null +++ b/tests/golden_data/cns_canonical/cgtoaa.canonical @@ -0,0 +1,334 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-cg-input-1.tbl <- example_ambig_1.tbl f6c145286f183493d628eb40329de09d2935d99eed974a647a5d1396cf75317f +canonical-input-1.pdb <- e2a_haddock_cg.pdb dc6af7dc18d021cbd4a8eb45ccde6286ac55c8474ea536685ee7f5f6dd2c6f0b +canonical-input-1.psf <- e2a_haddock_cg.psf 2d3db89f5ee60eef123ad9a9fbddd8c928b384fc49138d6e96146bb46be7a445 +canonical-input-2.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-2.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +module/calc_free-ene.cns a225c98d84a71b6de78b304cbf50fa61b788f9ce240dd41a851b2eab4ae7db0d +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/flex_segment_back.cns 65c5111559a51085c7c3de8f499f863e633f232ac28af1b903ff002a8b6e490c +module/flexauto-neighbors.cns 4f2b0c8e7d7dedb359d603d2b72f9fd37e35b797c773e2be238684bdf136f830 +module/print_coorheader.cns 1aeaaa5d8a8f8c3c20cbf48619387dbdb90f55f66ddf8f32b1e9eb43311553cc +module/protein-ss-restraints-alpha-beta.cns bf4dafb5de2dccc0efa0bafdb5080092229a876c89181509d2e7cef3ef2bd881 +module/read_param.cns 214a98955ac11285e92292199ad7e08ec5ca34ec3439430908e58161e2808f62 +module/restrain-ions.cns 81be80a9f950b1c21ab6f2944979e8eb03ba1b6fa46e770d3706887766f927b5 +module/scale_cg-to-aa.cns 316f7cc94ade460935423000eb1aba3ed14d7badd50d7a6cc044df142b7267f8 +module/scale_inter_final.cns 58f9dc67fd51a7be7b1b23992fd605a7438d01fa48cd5ed65ba2efae1cb82cd4 +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/refinement/cgtoaa/cns/cgtoaa.cns +3x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +3x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($mol_fix_origin_1=false) +eval ($mol_shape_1=false) +eval ($w_air=0.1) +eval ($w_bsa=0.0) +eval ($w_cdih=0.0) +eval ($w_dani=0.1) +eval ($w_desolv=1.0) +eval ($w_elec=0.2) +eval ($w_lcc=-10000.0) +eval ($w_rg=1.0) +eval ($w_sani=0.1) +eval ($w_sym=0.1) +eval ($w_vdw=1.0) +eval ($w_vean=0.1) +eval ($w_xpcs=0.1) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="cdie") +eval ($epsilon=1.0) +eval ($epsilon_cg=10.0) +eval ($nemsteps=10) +eval ($log_level="canonical-log-level") +eval ($nseg1=-1) +eval ($nseg2=-1) +eval ($nseg3=-1) +eval ($nseg4=-1) +eval ($nseg5=-1) +eval ($nseg6=-1) +eval ($nseg7=-1) +eval ($nseg8=-1) +eval ($nseg9=-1) +eval ($nseg10=-1) +eval ($nseg11=-1) +eval ($nseg12=-1) +eval ($nseg13=-1) +eval ($nseg14=-1) +eval ($nseg15=-1) +eval ($nseg16=-1) +eval ($nseg17=-1) +eval ($nseg18=-1) +eval ($nseg19=-1) +eval ($nseg20=-1) +eval ($nfle=0) +eval ($fle_seg_1="none") +eval ($int_1_1=1.0) +eval ($int_1_2=1.0) +eval ($int_1_3=1.0) +eval ($int_1_4=1.0) +eval ($int_1_5=1.0) +eval ($int_1_6=1.0) +eval ($int_1_7=1.0) +eval ($int_1_8=1.0) +eval ($int_1_9=1.0) +eval ($int_1_10=1.0) +eval ($int_1_11=1.0) +eval ($int_1_12=1.0) +eval ($int_1_13=1.0) +eval ($int_1_14=1.0) +eval ($int_1_15=1.0) +eval ($int_1_16=1.0) +eval ($int_1_17=1.0) +eval ($int_1_18=1.0) +eval ($int_1_19=1.0) +eval ($int_1_20=1.0) +eval ($int_2_2=1.0) +eval ($int_2_3=1.0) +eval ($int_2_4=1.0) +eval ($int_2_5=1.0) +eval ($int_2_6=1.0) +eval ($int_2_7=1.0) +eval ($int_2_8=1.0) +eval ($int_2_9=1.0) +eval ($int_2_10=1.0) +eval ($int_2_11=1.0) +eval ($int_2_12=1.0) +eval ($int_2_13=1.0) +eval ($int_2_14=1.0) +eval ($int_2_15=1.0) +eval ($int_2_16=1.0) +eval ($int_2_17=1.0) +eval ($int_2_18=1.0) +eval ($int_2_19=1.0) +eval ($int_2_20=1.0) +eval ($int_3_3=1.0) +eval ($int_3_4=1.0) +eval ($int_3_5=1.0) +eval ($int_3_6=1.0) +eval ($int_3_7=1.0) +eval ($int_3_8=1.0) +eval ($int_3_9=1.0) +eval ($int_3_10=1.0) +eval ($int_3_11=1.0) +eval ($int_3_12=1.0) +eval ($int_3_13=1.0) +eval ($int_3_14=1.0) +eval ($int_3_15=1.0) +eval ($int_3_16=1.0) +eval ($int_3_17=1.0) +eval ($int_3_18=1.0) +eval ($int_3_19=1.0) +eval ($int_3_20=1.0) +eval ($int_4_4=1.0) +eval ($int_4_5=1.0) +eval ($int_4_6=1.0) +eval ($int_4_7=1.0) +eval ($int_4_8=1.0) +eval ($int_4_9=1.0) +eval ($int_4_10=1.0) +eval ($int_4_11=1.0) +eval ($int_4_12=1.0) +eval ($int_4_13=1.0) +eval ($int_4_14=1.0) +eval ($int_4_15=1.0) +eval ($int_4_16=1.0) +eval ($int_4_17=1.0) +eval ($int_4_18=1.0) +eval ($int_4_19=1.0) +eval ($int_4_20=1.0) +eval ($int_5_5=1.0) +eval ($int_5_6=1.0) +eval ($int_5_7=1.0) +eval ($int_5_8=1.0) +eval ($int_5_9=1.0) +eval ($int_5_10=1.0) +eval ($int_5_11=1.0) +eval ($int_5_12=1.0) +eval ($int_5_13=1.0) +eval ($int_5_14=1.0) +eval ($int_5_15=1.0) +eval ($int_5_16=1.0) +eval ($int_5_17=1.0) +eval ($int_5_18=1.0) +eval ($int_5_19=1.0) +eval ($int_5_20=1.0) +eval ($int_6_6=1.0) +eval ($int_6_7=1.0) +eval ($int_6_8=1.0) +eval ($int_6_9=1.0) +eval ($int_6_10=1.0) +eval ($int_6_11=1.0) +eval ($int_6_12=1.0) +eval ($int_6_13=1.0) +eval ($int_6_14=1.0) +eval ($int_6_15=1.0) +eval ($int_6_16=1.0) +eval ($int_6_17=1.0) +eval ($int_6_18=1.0) +eval ($int_6_19=1.0) +eval ($int_6_20=1.0) +eval ($int_7_7=1.0) +eval ($int_7_8=1.0) +eval ($int_7_9=1.0) +eval ($int_7_10=1.0) +eval ($int_7_11=1.0) +eval ($int_7_12=1.0) +eval ($int_7_13=1.0) +eval ($int_7_14=1.0) +eval ($int_7_15=1.0) +eval ($int_7_16=1.0) +eval ($int_7_17=1.0) +eval ($int_7_18=1.0) +eval ($int_7_19=1.0) +eval ($int_8_8=1.0) +eval ($int_8_9=1.0) +eval ($int_7_20=1.0) +eval ($int_8_10=1.0) +eval ($int_8_11=1.0) +eval ($int_8_12=1.0) +eval ($int_8_13=1.0) +eval ($int_8_14=1.0) +eval ($int_8_15=1.0) +eval ($int_8_16=1.0) +eval ($int_8_17=1.0) +eval ($int_8_18=1.0) +eval ($int_8_19=1.0) +eval ($int_8_20=1.0) +eval ($int_9_11=1.0) +eval ($int_9_12=1.0) +eval ($int_9_13=1.0) +eval ($int_9_14=1.0) +eval ($int_9_15=1.0) +eval ($int_9_16=1.0) +eval ($int_9_17=1.0) +eval ($int_9_18=1.0) +eval ($int_9_19=1.0) +eval ($int_9_20=1.0) +eval ($int_9_9=1.0) +eval ($int_9_10=1.0) +eval ($int_10_10=1.0) +eval ($int_10_11=1.0) +eval ($int_10_12=1.0) +eval ($int_10_13=1.0) +eval ($int_10_14=1.0) +eval ($int_10_15=1.0) +eval ($int_10_16=1.0) +eval ($int_10_17=1.0) +eval ($int_10_18=1.0) +eval ($int_10_19=1.0) +eval ($int_10_20=1.0) +eval ($int_11_11=1.0) +eval ($int_11_12=1.0) +eval ($int_11_13=1.0) +eval ($int_11_14=1.0) +eval ($int_11_15=1.0) +eval ($int_11_16=1.0) +eval ($int_11_17=1.0) +eval ($int_11_18=1.0) +eval ($int_11_19=1.0) +eval ($int_11_20=1.0) +eval ($int_12_12=1.0) +eval ($int_12_13=1.0) +eval ($int_12_14=1.0) +eval ($int_12_15=1.0) +eval ($int_12_16=1.0) +eval ($int_12_17=1.0) +eval ($int_12_18=1.0) +eval ($int_12_19=1.0) +eval ($int_12_20=1.0) +eval ($int_13_13=1.0) +eval ($int_13_14=1.0) +eval ($int_13_15=1.0) +eval ($int_13_16=1.0) +eval ($int_13_17=1.0) +eval ($int_13_18=1.0) +eval ($int_13_19=1.0) +eval ($int_13_20=1.0) +eval ($int_14_14=1.0) +eval ($int_14_15=1.0) +eval ($int_14_16=1.0) +eval ($int_14_17=1.0) +eval ($int_14_18=1.0) +eval ($int_14_19=1.0) +eval ($int_14_20=1.0) +eval ($int_15_15=1.0) +eval ($int_15_16=1.0) +eval ($int_15_17=1.0) +eval ($int_15_18=1.0) +eval ($int_15_19=1.0) +eval ($int_15_20=1.0) +eval ($int_16_16=1.0) +eval ($int_16_17=1.0) +eval ($int_16_18=1.0) +eval ($int_16_19=1.0) +eval ($int_16_20=1.0) +eval ($int_17_17=1.0) +eval ($int_17_18=1.0) +eval ($int_17_19=1.0) +eval ($int_17_20=1.0) +eval ($int_18_18=1.0) +eval ($int_18_19=1.0) +eval ($int_18_20=1.0) +eval ($int_19_19=1.0) +eval ($int_19_20=1.0) +eval ($int_20_20=1.0) +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +eval ($ncomponents=1) +eval ($input_aa_psf_filename_1="canonical-input-2.psf") +eval ($input_aa_pdb_filename_1="canonical-input-2.pdb") +eval ($input_cgtbl_filename_1="canonical-cg-input-1.tbl") + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/emref.canonical b/tests/golden_data/cns_canonical/emref.canonical new file mode 100644 index 0000000000..356268b626 --- /dev/null +++ b/tests/golden_data/cns_canonical/emref.canonical @@ -0,0 +1,373 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/calc_free-ene.cns a225c98d84a71b6de78b304cbf50fa61b788f9ce240dd41a851b2eab4ae7db0d +module/charge-beads-interactions.cns 18635b66a5bb3b0078125265d1ff41f1708641e030d8a76bc93babd5e6b084d3 +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/contactairs.cns 2ed3199b7dd2704027344d49e4713e890f7947c0ada534c29dee05db7c3b9a87 +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/dna-rna_restraints.cns 82496362bbfb02a3f7c7dba37234185529a11fabe49ea5b418a9aac82047f926 +module/flex_segment_back.cns 65c5111559a51085c7c3de8f499f863e633f232ac28af1b903ff002a8b6e490c +module/flexauto-neighbors.cns 4f2b0c8e7d7dedb359d603d2b72f9fd37e35b797c773e2be238684bdf136f830 +module/print_coorheader.cns 01d46e002099e53216c013ee4955d5bd0843ebbdbcac8531870b4de3288b5a6c +module/protein-ss-restraints-all.cns 360ee4fd1a12ace4488fbec415420fb90e712f0a8c91c4f0f1bc8317a7adb3bd +module/protein-ss-restraints-alpha-beta.cns bf4dafb5de2dccc0efa0bafdb5080092229a876c89181509d2e7cef3ef2bd881 +module/protein-ss-restraints-alpha.cns ce9fdf8def4a954930aea37d0ba455a17b40a3aef81848faf4098000bb8bcf8f +module/read_data.cns e6d8a60e330f779fdc4c0f1d8d08ee2f1a0804184e03079f76514e8fe044ea6b +module/read_noes.cns 9cb0030057158db27e58df19cdae6681864592f64d95d2fd7de5227504553752 +module/read_param.cns 24b9bb87e923b9411f7335599c84328fd376fc391c889951b28b26113f9e6552 +module/restrain-ions.cns 6c6a3ab0fb6f764fe9e4d3d7e6413dc2b359c331e34124894fd008f6da0cb1c9 +module/scale_inter_final.cns 58f9dc67fd51a7be7b1b23992fd605a7438d01fa48cd5ed65ba2efae1cb82cd4 +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +module/setflags.cns 05609308453eee2eab0ec02b8d9fc458977aa5efac8f6ef9a61983f4426cd04b +module/symmultimer.cns ccb44eb5abe88c4ca4532dac5b35c4073376f75af230c8e1f7716c4e4ea06313 +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/refinement/emref/cns/emref.cns +2x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +2x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($mol_fix_origin_1=false) +eval ($mol_shape_1=false) +eval ($unambig_fname="") +eval ($hbond_fname="") +eval ($ambig_scale=50) +eval ($unambig_scale=50) +eval ($hbond_scale=50) +eval ($randremoval=true) +eval ($npart=2) +eval ($contactairs=false) +eval ($kcont=1.0) +eval ($dihe_fname="") +eval ($dihedrals_on=false) +eval ($dihedrals_scale=200) +eval ($ssdihed="none") +eval ($error_dih=10) +eval ($dnarest_on=false) +eval ($sym_on=false) +eval ($ksym=10.0) +eval ($symtbl_fname="") +eval ($nc2sym=0) +eval ($nc3sym=0) +eval ($nc4sym=0) +eval ($nc5sym=0) +eval ($nc6sym=0) +eval ($ns3sym=0) +eval ($ncs_on=false) +eval ($kncs=1.0) +eval ($nncs=0) +eval ($w_air=0.1) +eval ($w_bsa=0.0) +eval ($w_cdih=0.0) +eval ($w_dani=0.1) +eval ($w_desolv=1.0) +eval ($w_elec=0.2) +eval ($w_lcc=-10000.0) +eval ($w_rg=1.0) +eval ($w_sani=0.1) +eval ($w_sym=0.1) +eval ($w_vdw=1.0) +eval ($w_vean=0.1) +eval ($w_xpcs=0.1) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="cdie") +eval ($epsilon=1.0) +eval ($epsilon_cg=10.0) +eval ($nemsteps=200) +eval ($log_level="canonical-log-level") +eval ($nseg1=-1) +eval ($nseg2=-1) +eval ($nseg3=-1) +eval ($nseg4=-1) +eval ($nseg5=-1) +eval ($nseg6=-1) +eval ($nseg7=-1) +eval ($nseg8=-1) +eval ($nseg9=-1) +eval ($nseg10=-1) +eval ($nseg11=-1) +eval ($nseg12=-1) +eval ($nseg13=-1) +eval ($nseg14=-1) +eval ($nseg15=-1) +eval ($nseg16=-1) +eval ($nseg17=-1) +eval ($nseg18=-1) +eval ($nseg19=-1) +eval ($nseg20=-1) +eval ($nfle=0) +eval ($fle_seg_1="none") +eval ($int_1_1=1.0) +eval ($int_1_2=1.0) +eval ($int_1_3=1.0) +eval ($int_1_4=1.0) +eval ($int_1_5=1.0) +eval ($int_1_6=1.0) +eval ($int_1_7=1.0) +eval ($int_1_8=1.0) +eval ($int_1_9=1.0) +eval ($int_1_10=1.0) +eval ($int_1_11=1.0) +eval ($int_1_12=1.0) +eval ($int_1_13=1.0) +eval ($int_1_14=1.0) +eval ($int_1_15=1.0) +eval ($int_1_16=1.0) +eval ($int_1_17=1.0) +eval ($int_1_18=1.0) +eval ($int_1_19=1.0) +eval ($int_1_20=1.0) +eval ($int_2_2=1.0) +eval ($int_2_3=1.0) +eval ($int_2_4=1.0) +eval ($int_2_5=1.0) +eval ($int_2_6=1.0) +eval ($int_2_7=1.0) +eval ($int_2_8=1.0) +eval ($int_2_9=1.0) +eval ($int_2_10=1.0) +eval ($int_2_11=1.0) +eval ($int_2_12=1.0) +eval ($int_2_13=1.0) +eval ($int_2_14=1.0) +eval ($int_2_15=1.0) +eval ($int_2_16=1.0) +eval ($int_2_17=1.0) +eval ($int_2_18=1.0) +eval ($int_2_19=1.0) +eval ($int_2_20=1.0) +eval ($int_3_3=1.0) +eval ($int_3_4=1.0) +eval ($int_3_5=1.0) +eval ($int_3_6=1.0) +eval ($int_3_7=1.0) +eval ($int_3_8=1.0) +eval ($int_3_9=1.0) +eval ($int_3_10=1.0) +eval ($int_3_11=1.0) +eval ($int_3_12=1.0) +eval ($int_3_13=1.0) +eval ($int_3_14=1.0) +eval ($int_3_15=1.0) +eval ($int_3_16=1.0) +eval ($int_3_17=1.0) +eval ($int_3_18=1.0) +eval ($int_3_19=1.0) +eval ($int_3_20=1.0) +eval ($int_4_4=1.0) +eval ($int_4_5=1.0) +eval ($int_4_6=1.0) +eval ($int_4_7=1.0) +eval ($int_4_8=1.0) +eval ($int_4_9=1.0) +eval ($int_4_10=1.0) +eval ($int_4_11=1.0) +eval ($int_4_12=1.0) +eval ($int_4_13=1.0) +eval ($int_4_14=1.0) +eval ($int_4_15=1.0) +eval ($int_4_16=1.0) +eval ($int_4_17=1.0) +eval ($int_4_18=1.0) +eval ($int_4_19=1.0) +eval ($int_4_20=1.0) +eval ($int_5_5=1.0) +eval ($int_5_6=1.0) +eval ($int_5_7=1.0) +eval ($int_5_8=1.0) +eval ($int_5_9=1.0) +eval ($int_5_10=1.0) +eval ($int_5_11=1.0) +eval ($int_5_12=1.0) +eval ($int_5_13=1.0) +eval ($int_5_14=1.0) +eval ($int_5_15=1.0) +eval ($int_5_16=1.0) +eval ($int_5_17=1.0) +eval ($int_5_18=1.0) +eval ($int_5_19=1.0) +eval ($int_5_20=1.0) +eval ($int_6_6=1.0) +eval ($int_6_7=1.0) +eval ($int_6_8=1.0) +eval ($int_6_9=1.0) +eval ($int_6_10=1.0) +eval ($int_6_11=1.0) +eval ($int_6_12=1.0) +eval ($int_6_13=1.0) +eval ($int_6_14=1.0) +eval ($int_6_15=1.0) +eval ($int_6_16=1.0) +eval ($int_6_17=1.0) +eval ($int_6_18=1.0) +eval ($int_6_19=1.0) +eval ($int_6_20=1.0) +eval ($int_7_7=1.0) +eval ($int_7_8=1.0) +eval ($int_7_9=1.0) +eval ($int_7_10=1.0) +eval ($int_7_11=1.0) +eval ($int_7_12=1.0) +eval ($int_7_13=1.0) +eval ($int_7_14=1.0) +eval ($int_7_15=1.0) +eval ($int_7_16=1.0) +eval ($int_7_17=1.0) +eval ($int_7_18=1.0) +eval ($int_7_19=1.0) +eval ($int_8_8=1.0) +eval ($int_8_9=1.0) +eval ($int_7_20=1.0) +eval ($int_8_10=1.0) +eval ($int_8_11=1.0) +eval ($int_8_12=1.0) +eval ($int_8_13=1.0) +eval ($int_8_14=1.0) +eval ($int_8_15=1.0) +eval ($int_8_16=1.0) +eval ($int_8_17=1.0) +eval ($int_8_18=1.0) +eval ($int_8_19=1.0) +eval ($int_8_20=1.0) +eval ($int_9_11=1.0) +eval ($int_9_12=1.0) +eval ($int_9_13=1.0) +eval ($int_9_14=1.0) +eval ($int_9_15=1.0) +eval ($int_9_16=1.0) +eval ($int_9_17=1.0) +eval ($int_9_18=1.0) +eval ($int_9_19=1.0) +eval ($int_9_20=1.0) +eval ($int_9_9=1.0) +eval ($int_9_10=1.0) +eval ($int_10_10=1.0) +eval ($int_10_11=1.0) +eval ($int_10_12=1.0) +eval ($int_10_13=1.0) +eval ($int_10_14=1.0) +eval ($int_10_15=1.0) +eval ($int_10_16=1.0) +eval ($int_10_17=1.0) +eval ($int_10_18=1.0) +eval ($int_10_19=1.0) +eval ($int_10_20=1.0) +eval ($int_11_11=1.0) +eval ($int_11_12=1.0) +eval ($int_11_13=1.0) +eval ($int_11_14=1.0) +eval ($int_11_15=1.0) +eval ($int_11_16=1.0) +eval ($int_11_17=1.0) +eval ($int_11_18=1.0) +eval ($int_11_19=1.0) +eval ($int_11_20=1.0) +eval ($int_12_12=1.0) +eval ($int_12_13=1.0) +eval ($int_12_14=1.0) +eval ($int_12_15=1.0) +eval ($int_12_16=1.0) +eval ($int_12_17=1.0) +eval ($int_12_18=1.0) +eval ($int_12_19=1.0) +eval ($int_12_20=1.0) +eval ($int_13_13=1.0) +eval ($int_13_14=1.0) +eval ($int_13_15=1.0) +eval ($int_13_16=1.0) +eval ($int_13_17=1.0) +eval ($int_13_18=1.0) +eval ($int_13_19=1.0) +eval ($int_13_20=1.0) +eval ($int_14_14=1.0) +eval ($int_14_15=1.0) +eval ($int_14_16=1.0) +eval ($int_14_17=1.0) +eval ($int_14_18=1.0) +eval ($int_14_19=1.0) +eval ($int_14_20=1.0) +eval ($int_15_15=1.0) +eval ($int_15_16=1.0) +eval ($int_15_17=1.0) +eval ($int_15_18=1.0) +eval ($int_15_19=1.0) +eval ($int_15_20=1.0) +eval ($int_16_16=1.0) +eval ($int_16_17=1.0) +eval ($int_16_18=1.0) +eval ($int_16_19=1.0) +eval ($int_16_20=1.0) +eval ($int_17_17=1.0) +eval ($int_17_18=1.0) +eval ($int_17_19=1.0) +eval ($int_17_20=1.0) +eval ($int_18_18=1.0) +eval ($int_18_19=1.0) +eval ($int_18_20=1.0) +eval ($int_19_19=1.0) +eval ($int_19_20=1.0) +eval ($int_20_20=1.0) +eval ($mol_fix_origin_2=false) +eval ($mol_shape_2=false) +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) +eval ($seed=1694244009) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/emscoring.canonical b/tests/golden_data/cns_canonical/emscoring.canonical new file mode 100644 index 0000000000..8996111331 --- /dev/null +++ b/tests/golden_data/cns_canonical/emscoring.canonical @@ -0,0 +1,99 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/calc_free-ene.cns 226672c92df44e110e27dcbf07540b3acec02402cc279c87ebdf519863785d91 +module/charge-beads-interactions.cns 18635b66a5bb3b0078125265d1ff41f1708641e030d8a76bc93babd5e6b084d3 +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/print_coorheader.cns ba75e9d147deebf46f075d31aa56633c00d02364d869f1aec7cba33c01dd0fa7 +module/read_param.cns 24b9bb87e923b9411f7335599c84328fd376fc391c889951b28b26113f9e6552 +module/restrain-ions.cns 6c6a3ab0fb6f764fe9e4d3d7e6413dc2b359c331e34124894fd008f6da0cb1c9 +module/scale_inter_final.cns b55e7bcc5175bc7b74b5ec5c608b0dab0e71c95451f1fb7a6a120fb9a1a7c769 +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/scoring/emscoring/cns/emscoring.cns +1x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +1x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($per_interface_scoring=false) +eval ($w_air=0.0) +eval ($w_bsa=0.0) +eval ($w_cdih=0.0) +eval ($w_dani=0.0) +eval ($w_desolv=1.0) +eval ($w_elec=0.2) +eval ($w_lcc=-10000.0) +eval ($w_rg=0.0) +eval ($w_sani=0.0) +eval ($w_sym=0.0) +eval ($w_vdw=1.0) +eval ($w_vean=0.0) +eval ($w_xpcs=0.0) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="cdie") +eval ($epsilon=1.0) +eval ($dihedflag=true) +eval ($nemsteps=50) +eval ($log_level="canonical-log-level") +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/flexref.canonical b/tests/golden_data/cns_canonical/flexref.canonical new file mode 100644 index 0000000000..73b686c396 --- /dev/null +++ b/tests/golden_data/cns_canonical/flexref.canonical @@ -0,0 +1,491 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/calc_free-ene.cns a225c98d84a71b6de78b304cbf50fa61b788f9ce240dd41a851b2eab4ae7db0d +module/charge-beads-interactions.cns 18635b66a5bb3b0078125265d1ff41f1708641e030d8a76bc93babd5e6b084d3 +module/check-homomers.cns b74acf4d25d4e3810fc19f90e80bfbdd0c845e3698be3cc01d79bb146210666b +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/cm-restraints.cns df5651f0cfa49798b856825c8a180c0994f18f0be56b7c9f8bb8165d8814489e +module/contactairs.cns 2ed3199b7dd2704027344d49e4713e890f7947c0ada534c29dee05db7c3b9a87 +module/covalions.cns 99ac3e09d68c32421277a5a78a6e09e22a9a8fc092f3d53b2483db04637d7b7a +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/dna-rna_restraints.cns 82496362bbfb02a3f7c7dba37234185529a11fabe49ea5b418a9aac82047f926 +module/flex_segment.cns 90c03922640cd678d0437c53953e15c6a2f6159c88c1494567bcca251d1cc9c8 +module/flex_segment_back.cns 65c5111559a51085c7c3de8f499f863e633f232ac28af1b903ff002a8b6e490c +module/flex_segment_side.cns b4e7490635936fed7b34cd0b0b12972c2f623486ab79a1171cd024ac67c7ab87 +module/flexauto-neighbors.cns 4f2b0c8e7d7dedb359d603d2b72f9fd37e35b797c773e2be238684bdf136f830 +module/numtrees.cns bc9dd356abd1b86cd102704ae6be85085a203b122c6ef6decf2935c737cef261 +module/print_coorheader.cns 01d46e002099e53216c013ee4955d5bd0843ebbdbcac8531870b4de3288b5a6c +module/protein-ss-restraints-all.cns 360ee4fd1a12ace4488fbec415420fb90e712f0a8c91c4f0f1bc8317a7adb3bd +module/protein-ss-restraints-alpha-beta.cns bf4dafb5de2dccc0efa0bafdb5080092229a876c89181509d2e7cef3ef2bd881 +module/protein-ss-restraints-alpha.cns ce9fdf8def4a954930aea37d0ba455a17b40a3aef81848faf4098000bb8bcf8f +module/random_rotations.cns b8395c968a9e5e08cb790cc898afb85c0c66efd411c0ba0d9f9c7b506d36b03b +module/read_data.cns e6d8a60e330f779fdc4c0f1d8d08ee2f1a0804184e03079f76514e8fe044ea6b +module/read_noes.cns 9cb0030057158db27e58df19cdae6681864592f64d95d2fd7de5227504553752 +module/read_param.cns 24b9bb87e923b9411f7335599c84328fd376fc391c889951b28b26113f9e6552 +module/restrain-ions.cns 6c6a3ab0fb6f764fe9e4d3d7e6413dc2b359c331e34124894fd008f6da0cb1c9 +module/sa_ltad_cool1.cns fd6847aedb405255941c6e0a48449f4483e472f0319bb0a9666f423808735b40 +module/sa_ltad_cool2.cns 1cf36f819ee7d331e33f3f387f324821ecb32ff55e954f3ff52a005683e3204d +module/sa_ltad_cool3.cns 9f7e4c5518e96b494a487b4573e28af4f6c25ceafab5d4b9196ac50aaafd849d +module/sa_ltad_hightemp.cns f644798e2f1e862e1e21a7d3821f14f059e736cb9ab94d367687593a5c445c63 +module/scale_inter.cns 98a209ed1eb9b41f43baa00c08e7c00cf5b6bd96396859c5be8f979318f1cbfc +module/scale_inter_final.cns 58f9dc67fd51a7be7b1b23992fd605a7438d01fa48cd5ed65ba2efae1cb82cd4 +module/scale_intra_only.cns ec810447a3a895aa8fddd5948a48de0fea9b583b06415551a01b33bef9639081 +module/separate.cns a6976973db8b294d20f1a86234604bb5ac8249d93d61b0b9047c4057ca63e7f7 +module/setflags.cns 825960406b8f3eab421f29887ee188cef4bc320950cc070fcd944d151e8ef732 +module/symmultimer.cns ccb44eb5abe88c4ca4532dac5b35c4073376f75af230c8e1f7716c4e4ea06313 +module/torsiontop.cns e468af55024ea5457a785a9863f08c2bfc84a83310a786cb4fbb47410c053ace +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/initial_positions/trans_vector_0 4b00414fb1f1375952e3fecbbfe0ba36e967dd12f308fd72f56a30e035679976 +toppar/initial_positions/trans_vector_1 aed8ddb81104ccf0521d0d20074fd78472c678c229d5755045a1ab75d98a45a0 +toppar/initial_positions/trans_vector_10 df6a73d152ece2cdf516bbf10742b8e9694df6eed4ef6b6ac7d2bedd0bc4edba +toppar/initial_positions/trans_vector_11 295260bf796ee666b40dd29296133b6faa4cb4cc56819d95df143ee28fe74c42 +toppar/initial_positions/trans_vector_12 8975e3c5fefe842da1afe7f5fa7b8b2900113a2c295354e505f9a45b4a65bb71 +toppar/initial_positions/trans_vector_13 67a677d6e86cbceac862e59fef40f8a8c0a57d6fc0d7a6af54cf058cdc3918f4 +toppar/initial_positions/trans_vector_14 308c3b733396f87880d28ff3e24879ad6c5cd1d86cce9a8d71cb39800fccfcef +toppar/initial_positions/trans_vector_15 e98e93d404358cbbbb76ca4f5034a8ee1183a59ff1bb787262705438d8794dc4 +toppar/initial_positions/trans_vector_16 205598c73b68f7214dc3f8e2e2f1c5e5db72543f16c33ff3aad724eecc641f25 +toppar/initial_positions/trans_vector_17 11ac7294b42c486a107790c0665f93d6a309eceea4fd670616e37073c6cdf89c +toppar/initial_positions/trans_vector_18 62109ac574a920da1652f8ca247b41cac6e3d12caa33241cef674f4b423deca2 +toppar/initial_positions/trans_vector_19 31fcfc2742bea9a41342ffdde89fca7471a840c423001638faa4d68c3f9ff035 +toppar/initial_positions/trans_vector_2 18fa2716b78aeab0cfb97ae271246e1292338015d235150d42633ecf15174dcc +toppar/initial_positions/trans_vector_20 38304ba4792223b07e05e626791c812fbd7cd87697df2987d0451f3e4db2bfb5 +toppar/initial_positions/trans_vector_21 2ec9c6dee2287441126b9754d1c7b734b49cf97762be26e9a71ada4b97067d36 +toppar/initial_positions/trans_vector_22 e97d9926b3b6bfbe19f367f5cfe0d83bd8a0f9309b3a713f64bf5dac2d188f7f +toppar/initial_positions/trans_vector_23 26f681dbeb37eaa6e3d238e34181549f1f10abb53d0241a2a425c4f2171cf786 +toppar/initial_positions/trans_vector_24 4d9b1e38820e501575def4c751ac29a8eb00f99069591904a215458393726d01 +toppar/initial_positions/trans_vector_25 8dbbfad7f0bd481ad189e79ecce24bea8e15758c843d154c227b87ee2f2bd7b7 +toppar/initial_positions/trans_vector_26 48d47c82520cb38171ae59e5bb342bba746b89259805f24cf34af9a6815177bd +toppar/initial_positions/trans_vector_27 1ce8112537b4da4d675001544c846e44199294ef6fea4f8bca34f2b5ace37dcf +toppar/initial_positions/trans_vector_28 7025d1601179ee6ac1be4ea7cacd18115c2266be6bf1123ac58d07b4b591c831 +toppar/initial_positions/trans_vector_29 f29935840ebeeac96d5a81c00109d3e816d12a620a758f8bf9ea74b62374f957 +toppar/initial_positions/trans_vector_3 a5b7915f9fef6d216e00644f90a176cf8a1c8cf52fb7d1a25dc12826bc69800c +toppar/initial_positions/trans_vector_30 f73e2ce3d732b38f59559d82971da218708faa64d8771ce32e45298a79cf42f0 +toppar/initial_positions/trans_vector_31 08da963b02cef780e97e97c704908c405dbf3c1b7aadd31eb3819356e460a780 +toppar/initial_positions/trans_vector_32 7f7464ca317d71b7e93860bf04b34ea91ac6b4b933b8b439b8cf56561fd798db +toppar/initial_positions/trans_vector_33 be9fe2ed1b4d0280aba7fdea677746a7aab295581627a2477b048e7b9651add9 +toppar/initial_positions/trans_vector_34 56e78fc220725025a7efcd94056f7efd1f81d222c419dd65cbddd0cb434217d3 +toppar/initial_positions/trans_vector_35 1d8e4d3586ca0d3bf9a517914cc85af9d121c558cf4477c0ae6a2b917011cd52 +toppar/initial_positions/trans_vector_36 a92507a9ce0d5d63453a3ba73188417677b49ca0ee50129bb76ae2daf92e1831 +toppar/initial_positions/trans_vector_37 71a6e11c2da06d4015fae6b805d8611d0453fd501f94e94e24c27b9df89d9884 +toppar/initial_positions/trans_vector_38 1beffc6fd6186b9a3a7576e036b6e8ad530a05d5b030333dcea3a23102c7089a +toppar/initial_positions/trans_vector_39 41f42e6860a0ef8e8c33bb16ace894161be418258aec2a9d3c6312c4436808a2 +toppar/initial_positions/trans_vector_4 2905f4c16d40780bee7f9dc7b0d07319742443b6d90cbd5c16bf9251b0318447 +toppar/initial_positions/trans_vector_40 33476b8202ff60cd382f1520374fa83d715f2ac19584ebec2cbfa8b225e19503 +toppar/initial_positions/trans_vector_41 e8c361ec2a6fa311d8d7ee1b1655daeb1e1bd863b28066eca2fc41794db7c78d +toppar/initial_positions/trans_vector_42 c8f5d5f4ad342e25b0e9d879a16b08d4a16ad9e146a6a7607394ed34e6321440 +toppar/initial_positions/trans_vector_43 e3885e782ff92ca0b6404e2de7e8c21f1c0c9c1024a20de9f26cebe4f74d88b7 +toppar/initial_positions/trans_vector_44 b7caee87d1c2afe3cfb3f9f7a2d47cdd37cad5a4033b7c12ddd5f4919586aad7 +toppar/initial_positions/trans_vector_45 0d0883486cc6c94c23a03999e6c43d1a0289230a7a39135ce5ad625ea3bd3d29 +toppar/initial_positions/trans_vector_46 738a431c01c5783982b092bab906bdf19dcbeb31c6338263154bdd8cc7e593ac +toppar/initial_positions/trans_vector_47 737de2d33c98e0f58d491d1521492dbef4016cb5be7733b00fd85529a455db00 +toppar/initial_positions/trans_vector_48 ad7ab4b47d97ead8b85e08332d78ee3356e02b54873060c4fe1592691c294bcb +toppar/initial_positions/trans_vector_49 418c3629030836682102893e3a18b1587742dac84388a7aaf420c2600bbdf1e8 +toppar/initial_positions/trans_vector_5 2d801d04878066986047d1559f0035ac76884d274bc3ae961caa25962038ce52 +toppar/initial_positions/trans_vector_50 7fd573a063954ebd3643a249fcb78d6586f9a24971e5975d1691660cd3ffcc0f +toppar/initial_positions/trans_vector_6 336318b9748243e0e2d421607070edbfb2c9ec7116dbf0de5c722ef5146c4a06 +toppar/initial_positions/trans_vector_7 aea642f694ba2388f2c1c201efe90772fa4f376044abe78b255bc58ad35c7192 +toppar/initial_positions/trans_vector_8 fb9671a5bc309b6145d5d2579f9edef168a2d1a4710acf16cde583ccd5c023b1 +toppar/initial_positions/trans_vector_9 324adecac5cde19a97f87217d20846fa3bb79fac33cba35fb25f5d492f83fb5d +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/refinement/flexref/cns/flexref.cns +2x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +2x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($mol_fix_origin_1=false) +eval ($mol_shape_1=false) +eval ($unambig_fname="") +eval ($hbond_fname="") +eval ($ambig_hot=10) +eval ($ambig_cool1=10) +eval ($ambig_cool2=50) +eval ($ambig_cool3=50) +eval ($unambig_hot=10) +eval ($unambig_cool1=10) +eval ($unambig_cool2=50) +eval ($unambig_cool3=50) +eval ($hbond_hot=10) +eval ($hbond_cool1=10) +eval ($hbond_cool2=50) +eval ($hbond_cool3=50) +eval ($randremoval=true) +eval ($npart=2) +eval ($contactairs=false) +eval ($kcont=1.0) +eval ($cmrest=false) +eval ($cmtight=true) +eval ($kcm=1.0) +eval ($dihe_fname="") +eval ($dihedrals_on=false) +eval ($dihedrals_hot=5) +eval ($dihedrals_cool1=5) +eval ($dihedrals_cool2=50) +eval ($dihedrals_cool3=200) +eval ($ssdihed="none") +eval ($error_dih=10) +eval ($dnarest_on=false) +eval ($mrswi_hot=0.5) +eval ($mrswi_cool1=0.5) +eval ($mrswi_cool2=0.5) +eval ($mrswi_cool3=0.5) +eval ($rswi_hot=0.5) +eval ($rswi_cool1=0.5) +eval ($rswi_cool2=0.5) +eval ($rswi_cool3=0.5) +eval ($masy_hot=-1.0) +eval ($masy_cool1=-1.0) +eval ($masy_cool2=-0.1) +eval ($masy_cool3=-0.1) +eval ($asy_hot=1.0) +eval ($asy_cool1=1.0) +eval ($asy_cool2=0.1) +eval ($asy_cool3=0.1) +eval ($sym_on=false) +eval ($ksym=10.0) +eval ($symtbl_fname="") +eval ($nc2sym=0) +eval ($nc3sym=0) +eval ($nc4sym=0) +eval ($nc5sym=0) +eval ($nc6sym=0) +eval ($ns3sym=0) +eval ($ncs_on=false) +eval ($kncs=1.0) +eval ($nncs=0) +eval ($w_air=0.1) +eval ($w_bsa=-0.01) +eval ($w_cdih=0.0) +eval ($w_dani=0.1) +eval ($w_desolv=1.0) +eval ($w_elec=1.0) +eval ($w_lcc=-10000.0) +eval ($w_rg=1.0) +eval ($w_sani=0.1) +eval ($w_sym=0.1) +eval ($w_vdw=1.0) +eval ($w_vean=0.1) +eval ($w_xpcs=0.1) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="rdie") +eval ($epsilon=1.0) +eval ($epsilon_cg=10.0) +eval ($dihedflag=true) +eval ($nemsteps=200) +eval ($mdsteps_rigid=500) +eval ($mdsteps_cool1=500) +eval ($mdsteps_cool2=1000) +eval ($mdsteps_cool3=1000) +eval ($separate=false) +eval ($randrot=false) +eval ($timestep=0.002) +eval ($tadfactor=8) +eval ($temp_high=2000) +eval ($temp_cool1_init=2000) +eval ($temp_cool1_final=500) +eval ($temp_cool2_init=1000) +eval ($temp_cool2_final=50) +eval ($temp_cool3_init=1000) +eval ($temp_cool3_final=50) +eval ($sinter_rigid_init=0.001) +eval ($sinter_rigid_final=0.001) +eval ($sinter_cool2_init=0.001) +eval ($sinter_cool2_final=1.0) +eval ($sinter_cool3_init=0.05) +eval ($sinter_cool3_final=1.0) +eval ($log_level="canonical-log-level") +eval ($nseg1=-1) +eval ($nseg2=-1) +eval ($nseg3=-1) +eval ($nseg4=-1) +eval ($nseg5=-1) +eval ($nseg6=-1) +eval ($nseg7=-1) +eval ($nseg8=-1) +eval ($nseg9=-1) +eval ($nseg10=-1) +eval ($nseg11=-1) +eval ($nseg12=-1) +eval ($nseg13=-1) +eval ($nseg14=-1) +eval ($nseg15=-1) +eval ($nseg16=-1) +eval ($nseg17=-1) +eval ($nseg18=-1) +eval ($nseg19=-1) +eval ($nseg20=-1) +eval ($nfle=0) +eval ($fle_seg_1="none") +eval ($int_1_1=1.0) +eval ($int_1_2=1.0) +eval ($int_1_3=1.0) +eval ($int_1_4=1.0) +eval ($int_1_5=1.0) +eval ($int_1_6=1.0) +eval ($int_1_7=1.0) +eval ($int_1_8=1.0) +eval ($int_1_9=1.0) +eval ($int_1_10=1.0) +eval ($int_1_11=1.0) +eval ($int_1_12=1.0) +eval ($int_1_13=1.0) +eval ($int_1_14=1.0) +eval ($int_1_15=1.0) +eval ($int_1_16=1.0) +eval ($int_1_17=1.0) +eval ($int_1_18=1.0) +eval ($int_1_19=1.0) +eval ($int_1_20=1.0) +eval ($int_2_2=1.0) +eval ($int_2_3=1.0) +eval ($int_2_4=1.0) +eval ($int_2_5=1.0) +eval ($int_2_6=1.0) +eval ($int_2_7=1.0) +eval ($int_2_8=1.0) +eval ($int_2_9=1.0) +eval ($int_2_10=1.0) +eval ($int_2_11=1.0) +eval ($int_2_12=1.0) +eval ($int_2_13=1.0) +eval ($int_2_14=1.0) +eval ($int_2_15=1.0) +eval ($int_2_16=1.0) +eval ($int_2_17=1.0) +eval ($int_2_18=1.0) +eval ($int_2_19=1.0) +eval ($int_2_20=1.0) +eval ($int_3_3=1.0) +eval ($int_3_4=1.0) +eval ($int_3_5=1.0) +eval ($int_3_6=1.0) +eval ($int_3_7=1.0) +eval ($int_3_8=1.0) +eval ($int_3_9=1.0) +eval ($int_3_10=1.0) +eval ($int_3_11=1.0) +eval ($int_3_12=1.0) +eval ($int_3_13=1.0) +eval ($int_3_14=1.0) +eval ($int_3_15=1.0) +eval ($int_3_16=1.0) +eval ($int_3_17=1.0) +eval ($int_3_18=1.0) +eval ($int_3_19=1.0) +eval ($int_3_20=1.0) +eval ($int_4_4=1.0) +eval ($int_4_5=1.0) +eval ($int_4_6=1.0) +eval ($int_4_7=1.0) +eval ($int_4_8=1.0) +eval ($int_4_9=1.0) +eval ($int_4_10=1.0) +eval ($int_4_11=1.0) +eval ($int_4_12=1.0) +eval ($int_4_13=1.0) +eval ($int_4_14=1.0) +eval ($int_4_15=1.0) +eval ($int_4_16=1.0) +eval ($int_4_17=1.0) +eval ($int_4_18=1.0) +eval ($int_4_19=1.0) +eval ($int_4_20=1.0) +eval ($int_5_5=1.0) +eval ($int_5_6=1.0) +eval ($int_5_7=1.0) +eval ($int_5_8=1.0) +eval ($int_5_9=1.0) +eval ($int_5_10=1.0) +eval ($int_5_11=1.0) +eval ($int_5_12=1.0) +eval ($int_5_13=1.0) +eval ($int_5_14=1.0) +eval ($int_5_15=1.0) +eval ($int_5_16=1.0) +eval ($int_5_17=1.0) +eval ($int_5_18=1.0) +eval ($int_5_19=1.0) +eval ($int_5_20=1.0) +eval ($int_6_6=1.0) +eval ($int_6_7=1.0) +eval ($int_6_8=1.0) +eval ($int_6_9=1.0) +eval ($int_6_10=1.0) +eval ($int_6_11=1.0) +eval ($int_6_12=1.0) +eval ($int_6_13=1.0) +eval ($int_6_14=1.0) +eval ($int_6_15=1.0) +eval ($int_6_16=1.0) +eval ($int_6_17=1.0) +eval ($int_6_18=1.0) +eval ($int_6_19=1.0) +eval ($int_6_20=1.0) +eval ($int_7_7=1.0) +eval ($int_7_8=1.0) +eval ($int_7_9=1.0) +eval ($int_7_10=1.0) +eval ($int_7_11=1.0) +eval ($int_7_12=1.0) +eval ($int_7_13=1.0) +eval ($int_7_14=1.0) +eval ($int_7_15=1.0) +eval ($int_7_16=1.0) +eval ($int_7_17=1.0) +eval ($int_7_18=1.0) +eval ($int_7_19=1.0) +eval ($int_8_8=1.0) +eval ($int_8_9=1.0) +eval ($int_7_20=1.0) +eval ($int_8_10=1.0) +eval ($int_8_11=1.0) +eval ($int_8_12=1.0) +eval ($int_8_13=1.0) +eval ($int_8_14=1.0) +eval ($int_8_15=1.0) +eval ($int_8_16=1.0) +eval ($int_8_17=1.0) +eval ($int_8_18=1.0) +eval ($int_8_19=1.0) +eval ($int_8_20=1.0) +eval ($int_9_11=1.0) +eval ($int_9_12=1.0) +eval ($int_9_13=1.0) +eval ($int_9_14=1.0) +eval ($int_9_15=1.0) +eval ($int_9_16=1.0) +eval ($int_9_17=1.0) +eval ($int_9_18=1.0) +eval ($int_9_19=1.0) +eval ($int_9_20=1.0) +eval ($int_9_9=1.0) +eval ($int_9_10=1.0) +eval ($int_10_10=1.0) +eval ($int_10_11=1.0) +eval ($int_10_12=1.0) +eval ($int_10_13=1.0) +eval ($int_10_14=1.0) +eval ($int_10_15=1.0) +eval ($int_10_16=1.0) +eval ($int_10_17=1.0) +eval ($int_10_18=1.0) +eval ($int_10_19=1.0) +eval ($int_10_20=1.0) +eval ($int_11_11=1.0) +eval ($int_11_12=1.0) +eval ($int_11_13=1.0) +eval ($int_11_14=1.0) +eval ($int_11_15=1.0) +eval ($int_11_16=1.0) +eval ($int_11_17=1.0) +eval ($int_11_18=1.0) +eval ($int_11_19=1.0) +eval ($int_11_20=1.0) +eval ($int_12_12=1.0) +eval ($int_12_13=1.0) +eval ($int_12_14=1.0) +eval ($int_12_15=1.0) +eval ($int_12_16=1.0) +eval ($int_12_17=1.0) +eval ($int_12_18=1.0) +eval ($int_12_19=1.0) +eval ($int_12_20=1.0) +eval ($int_13_13=1.0) +eval ($int_13_14=1.0) +eval ($int_13_15=1.0) +eval ($int_13_16=1.0) +eval ($int_13_17=1.0) +eval ($int_13_18=1.0) +eval ($int_13_19=1.0) +eval ($int_13_20=1.0) +eval ($int_14_14=1.0) +eval ($int_14_15=1.0) +eval ($int_14_16=1.0) +eval ($int_14_17=1.0) +eval ($int_14_18=1.0) +eval ($int_14_19=1.0) +eval ($int_14_20=1.0) +eval ($int_15_15=1.0) +eval ($int_15_16=1.0) +eval ($int_15_17=1.0) +eval ($int_15_18=1.0) +eval ($int_15_19=1.0) +eval ($int_15_20=1.0) +eval ($int_16_16=1.0) +eval ($int_16_17=1.0) +eval ($int_16_18=1.0) +eval ($int_16_19=1.0) +eval ($int_16_20=1.0) +eval ($int_17_17=1.0) +eval ($int_17_18=1.0) +eval ($int_17_19=1.0) +eval ($int_17_20=1.0) +eval ($int_18_18=1.0) +eval ($int_18_19=1.0) +eval ($int_18_20=1.0) +eval ($int_19_19=1.0) +eval ($int_19_20=1.0) +eval ($int_20_20=1.0) +eval ($mol_fix_origin_2=false) +eval ($mol_shape_2=false) +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) +eval ($seed=1694244009) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/mdref.canonical b/tests/golden_data/cns_canonical/mdref.canonical new file mode 100644 index 0000000000..305d1f2689 --- /dev/null +++ b/tests/golden_data/cns_canonical/mdref.canonical @@ -0,0 +1,386 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/calc_free-ene.cns a225c98d84a71b6de78b304cbf50fa61b788f9ce240dd41a851b2eab4ae7db0d +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/cm-restraints.cns 5ea236c8b87c59a81c11e63754039bb943eb76001e56fc86bf8afeb9d1437c7d +module/contactairs.cns 2ed3199b7dd2704027344d49e4713e890f7947c0ada534c29dee05db7c3b9a87 +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/dna-rna_restraints.cns 82496362bbfb02a3f7c7dba37234185529a11fabe49ea5b418a9aac82047f926 +module/flex_segment_back.cns 8b2b2bd06cf5ea48f5231bcf0c203b5645b5ac0999bffa87a09227d483421469 +module/flex_segment_side.cns 2cbbf5c5440efa7e73edf4876fd7ece6ba3e063f09b152c7ce56d24c1c70af77 +module/flexauto-neighbors.cns 60c1708d11fb381531f0c8e9da44a32a52cb944afca391b3f4440bd9e24c670b +module/generate_dmso.cns fb0b8a64e3618c34be1239ec2d099cb8bcdfea68d64eab3f1d40733f0eabe3d6 +module/generate_water.cns 18eea7c93ee95d9b4728113c9a1f4c4fcabc75b9ed4311c45ccc5e07f128dc8d +module/print_coorheader.cns 01d46e002099e53216c013ee4955d5bd0843ebbdbcac8531870b4de3288b5a6c +module/protein-ss-restraints-all.cns 360ee4fd1a12ace4488fbec415420fb90e712f0a8c91c4f0f1bc8317a7adb3bd +module/protein-ss-restraints-alpha-beta.cns bf4dafb5de2dccc0efa0bafdb5080092229a876c89181509d2e7cef3ef2bd881 +module/protein-ss-restraints-alpha.cns ce9fdf8def4a954930aea37d0ba455a17b40a3aef81848faf4098000bb8bcf8f +module/read_data.cns e6d8a60e330f779fdc4c0f1d8d08ee2f1a0804184e03079f76514e8fe044ea6b +module/read_noes.cns 9cb0030057158db27e58df19cdae6681864592f64d95d2fd7de5227504553752 +module/read_param.cns 0e6fe81a7f5a20920b3ec1486b546f029db5c3901e7a45cbc492dd87bbbd0576 +module/restrain-ions.cns 6c6a3ab0fb6f764fe9e4d3d7e6413dc2b359c331e34124894fd008f6da0cb1c9 +module/scale_inter_final.cns 58f9dc67fd51a7be7b1b23992fd605a7438d01fa48cd5ed65ba2efae1cb82cd4 +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +module/setflags.cns 05609308453eee2eab0ec02b8d9fc458977aa5efac8f6ef9a61983f4426cd04b +module/symmultimer.cns ccb44eb5abe88c4ca4532dac5b35c4073376f75af230c8e1f7716c4e4ea06313 +toppar/boxtyp20.pdb b113ae0e268cfaa2c4da4b6cce33fd65259c905d8d6f44b6fe429587cd524700 +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dmso.pdb 8a7d6f535e88c312182b4d05cf1423c5e107b506d303d29bab42d9d0ace1bc43 +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c +toppar/solvent-allhdg5-4.top 4b3546f0a8ab278eb45874b4612680b96ab1460ee48afb47aebd44ac4723bb8e + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/refinement/mdref/cns/mdref.cns +2x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +2x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($mol_fix_origin_1=false) +eval ($mol_shape_1=false) +eval ($unambig_fname="") +eval ($hbond_fname="") +eval ($ambig_scale=50) +eval ($unambig_scale=50) +eval ($hbond_scale=50) +eval ($randremoval=true) +eval ($npart=2) +eval ($contactairs=false) +eval ($kcont=1.0) +eval ($cmrest=false) +eval ($cmtight=true) +eval ($kcm=1.0) +eval ($dihe_fname="") +eval ($dihedrals_on=false) +eval ($dihedrals_scale=200) +eval ($ssdihed="none") +eval ($error_dih=10) +eval ($dnarest_on=false) +eval ($sym_on=false) +eval ($ksym=10.0) +eval ($symtbl_fname="") +eval ($nc2sym=0) +eval ($nc3sym=0) +eval ($nc4sym=0) +eval ($nc5sym=0) +eval ($nc6sym=0) +eval ($ns3sym=0) +eval ($ncs_on=false) +eval ($kncs=1.0) +eval ($nncs=0) +eval ($w_air=0.1) +eval ($w_bsa=0.0) +eval ($w_cdih=0.0) +eval ($w_dani=0.1) +eval ($w_desolv=1.0) +eval ($w_elec=0.2) +eval ($w_lcc=-10000.0) +eval ($w_rg=1.0) +eval ($w_sani=0.1) +eval ($w_sym=0.1) +eval ($w_vdw=1.0) +eval ($w_vean=0.1) +eval ($w_xpcs=0.1) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="cdie") +eval ($epsilon=1.0) +eval ($dihedflag=true) +eval ($solvent="water") +eval ($nemsteps=200) +eval ($timestep=0.002) +eval ($waterheatsteps=100) +eval ($watersteps=1250) +eval ($watercoolsteps=500) +eval ($keepwater=false) +eval ($log_level="canonical-log-level") +eval ($nseg1=-1) +eval ($nseg2=-1) +eval ($nseg3=-1) +eval ($nseg4=-1) +eval ($nseg5=-1) +eval ($nseg6=-1) +eval ($nseg7=-1) +eval ($nseg8=-1) +eval ($nseg9=-1) +eval ($nseg10=-1) +eval ($nseg11=-1) +eval ($nseg12=-1) +eval ($nseg13=-1) +eval ($nseg14=-1) +eval ($nseg15=-1) +eval ($nseg16=-1) +eval ($nseg17=-1) +eval ($nseg18=-1) +eval ($nseg19=-1) +eval ($nseg20=-1) +eval ($nfle=0) +eval ($fle_seg_1="none") +eval ($int_1_1=1.0) +eval ($int_1_2=1.0) +eval ($int_1_3=1.0) +eval ($int_1_4=1.0) +eval ($int_1_5=1.0) +eval ($int_1_6=1.0) +eval ($int_1_7=1.0) +eval ($int_1_8=1.0) +eval ($int_1_9=1.0) +eval ($int_1_10=1.0) +eval ($int_1_11=1.0) +eval ($int_1_12=1.0) +eval ($int_1_13=1.0) +eval ($int_1_14=1.0) +eval ($int_1_15=1.0) +eval ($int_1_16=1.0) +eval ($int_1_17=1.0) +eval ($int_1_18=1.0) +eval ($int_1_19=1.0) +eval ($int_1_20=1.0) +eval ($int_2_2=1.0) +eval ($int_2_3=1.0) +eval ($int_2_4=1.0) +eval ($int_2_5=1.0) +eval ($int_2_6=1.0) +eval ($int_2_7=1.0) +eval ($int_2_8=1.0) +eval ($int_2_9=1.0) +eval ($int_2_10=1.0) +eval ($int_2_11=1.0) +eval ($int_2_12=1.0) +eval ($int_2_13=1.0) +eval ($int_2_14=1.0) +eval ($int_2_15=1.0) +eval ($int_2_16=1.0) +eval ($int_2_17=1.0) +eval ($int_2_18=1.0) +eval ($int_2_19=1.0) +eval ($int_2_20=1.0) +eval ($int_3_3=1.0) +eval ($int_3_4=1.0) +eval ($int_3_5=1.0) +eval ($int_3_6=1.0) +eval ($int_3_7=1.0) +eval ($int_3_8=1.0) +eval ($int_3_9=1.0) +eval ($int_3_10=1.0) +eval ($int_3_11=1.0) +eval ($int_3_12=1.0) +eval ($int_3_13=1.0) +eval ($int_3_14=1.0) +eval ($int_3_15=1.0) +eval ($int_3_16=1.0) +eval ($int_3_17=1.0) +eval ($int_3_18=1.0) +eval ($int_3_19=1.0) +eval ($int_3_20=1.0) +eval ($int_4_4=1.0) +eval ($int_4_5=1.0) +eval ($int_4_6=1.0) +eval ($int_4_7=1.0) +eval ($int_4_8=1.0) +eval ($int_4_9=1.0) +eval ($int_4_10=1.0) +eval ($int_4_11=1.0) +eval ($int_4_12=1.0) +eval ($int_4_13=1.0) +eval ($int_4_14=1.0) +eval ($int_4_15=1.0) +eval ($int_4_16=1.0) +eval ($int_4_17=1.0) +eval ($int_4_18=1.0) +eval ($int_4_19=1.0) +eval ($int_4_20=1.0) +eval ($int_5_5=1.0) +eval ($int_5_6=1.0) +eval ($int_5_7=1.0) +eval ($int_5_8=1.0) +eval ($int_5_9=1.0) +eval ($int_5_10=1.0) +eval ($int_5_11=1.0) +eval ($int_5_12=1.0) +eval ($int_5_13=1.0) +eval ($int_5_14=1.0) +eval ($int_5_15=1.0) +eval ($int_5_16=1.0) +eval ($int_5_17=1.0) +eval ($int_5_18=1.0) +eval ($int_5_19=1.0) +eval ($int_5_20=1.0) +eval ($int_6_6=1.0) +eval ($int_6_7=1.0) +eval ($int_6_8=1.0) +eval ($int_6_9=1.0) +eval ($int_6_10=1.0) +eval ($int_6_11=1.0) +eval ($int_6_12=1.0) +eval ($int_6_13=1.0) +eval ($int_6_14=1.0) +eval ($int_6_15=1.0) +eval ($int_6_16=1.0) +eval ($int_6_17=1.0) +eval ($int_6_18=1.0) +eval ($int_6_19=1.0) +eval ($int_6_20=1.0) +eval ($int_7_7=1.0) +eval ($int_7_8=1.0) +eval ($int_7_9=1.0) +eval ($int_7_10=1.0) +eval ($int_7_11=1.0) +eval ($int_7_12=1.0) +eval ($int_7_13=1.0) +eval ($int_7_14=1.0) +eval ($int_7_15=1.0) +eval ($int_7_16=1.0) +eval ($int_7_17=1.0) +eval ($int_7_18=1.0) +eval ($int_7_19=1.0) +eval ($int_8_8=1.0) +eval ($int_8_9=1.0) +eval ($int_7_20=1.0) +eval ($int_8_10=1.0) +eval ($int_8_11=1.0) +eval ($int_8_12=1.0) +eval ($int_8_13=1.0) +eval ($int_8_14=1.0) +eval ($int_8_15=1.0) +eval ($int_8_16=1.0) +eval ($int_8_17=1.0) +eval ($int_8_18=1.0) +eval ($int_8_19=1.0) +eval ($int_8_20=1.0) +eval ($int_9_11=1.0) +eval ($int_9_12=1.0) +eval ($int_9_13=1.0) +eval ($int_9_14=1.0) +eval ($int_9_15=1.0) +eval ($int_9_16=1.0) +eval ($int_9_17=1.0) +eval ($int_9_18=1.0) +eval ($int_9_19=1.0) +eval ($int_9_20=1.0) +eval ($int_9_9=1.0) +eval ($int_9_10=1.0) +eval ($int_10_10=1.0) +eval ($int_10_11=1.0) +eval ($int_10_12=1.0) +eval ($int_10_13=1.0) +eval ($int_10_14=1.0) +eval ($int_10_15=1.0) +eval ($int_10_16=1.0) +eval ($int_10_17=1.0) +eval ($int_10_18=1.0) +eval ($int_10_19=1.0) +eval ($int_10_20=1.0) +eval ($int_11_11=1.0) +eval ($int_11_12=1.0) +eval ($int_11_13=1.0) +eval ($int_11_14=1.0) +eval ($int_11_15=1.0) +eval ($int_11_16=1.0) +eval ($int_11_17=1.0) +eval ($int_11_18=1.0) +eval ($int_11_19=1.0) +eval ($int_11_20=1.0) +eval ($int_12_12=1.0) +eval ($int_12_13=1.0) +eval ($int_12_14=1.0) +eval ($int_12_15=1.0) +eval ($int_12_16=1.0) +eval ($int_12_17=1.0) +eval ($int_12_18=1.0) +eval ($int_12_19=1.0) +eval ($int_12_20=1.0) +eval ($int_13_13=1.0) +eval ($int_13_14=1.0) +eval ($int_13_15=1.0) +eval ($int_13_16=1.0) +eval ($int_13_17=1.0) +eval ($int_13_18=1.0) +eval ($int_13_19=1.0) +eval ($int_13_20=1.0) +eval ($int_14_14=1.0) +eval ($int_14_15=1.0) +eval ($int_14_16=1.0) +eval ($int_14_17=1.0) +eval ($int_14_18=1.0) +eval ($int_14_19=1.0) +eval ($int_14_20=1.0) +eval ($int_15_15=1.0) +eval ($int_15_16=1.0) +eval ($int_15_17=1.0) +eval ($int_15_18=1.0) +eval ($int_15_19=1.0) +eval ($int_15_20=1.0) +eval ($int_16_16=1.0) +eval ($int_16_17=1.0) +eval ($int_16_18=1.0) +eval ($int_16_19=1.0) +eval ($int_16_20=1.0) +eval ($int_17_17=1.0) +eval ($int_17_18=1.0) +eval ($int_17_19=1.0) +eval ($int_17_20=1.0) +eval ($int_18_18=1.0) +eval ($int_18_19=1.0) +eval ($int_18_20=1.0) +eval ($int_19_19=1.0) +eval ($int_19_20=1.0) +eval ($int_20_20=1.0) +eval ($mol_fix_origin_2=false) +eval ($mol_shape_2=false) +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) +eval ($seed=1694244009) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/mdscoring.canonical b/tests/golden_data/cns_canonical/mdscoring.canonical new file mode 100644 index 0000000000..52cadb99df --- /dev/null +++ b/tests/golden_data/cns_canonical/mdscoring.canonical @@ -0,0 +1,123 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/calc_free-ene.cns a225c98d84a71b6de78b304cbf50fa61b788f9ce240dd41a851b2eab4ae7db0d +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/contactairs.cns 2ed3199b7dd2704027344d49e4713e890f7947c0ada534c29dee05db7c3b9a87 +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/dna-rna_restraints.cns 82496362bbfb02a3f7c7dba37234185529a11fabe49ea5b418a9aac82047f926 +module/flex_segment_back.cns c4e8ab72177f6950c770ead58966d4c09c80de8b667bf539b6f7565105e3f8f2 +module/flex_segment_side.cns c5ce9a0a9809205e34ef03fb945e007e88b48b8ec98e684d437401f61066f4dd +module/flexauto-neighbors.cns 60c1708d11fb381531f0c8e9da44a32a52cb944afca391b3f4440bd9e24c670b +module/generate_dmso.cns fb0b8a64e3618c34be1239ec2d099cb8bcdfea68d64eab3f1d40733f0eabe3d6 +module/generate_water.cns 18eea7c93ee95d9b4728113c9a1f4c4fcabc75b9ed4311c45ccc5e07f128dc8d +module/print_coorheader.cns ba75e9d147deebf46f075d31aa56633c00d02364d869f1aec7cba33c01dd0fa7 +module/protein-ss-restraints-all.cns 360ee4fd1a12ace4488fbec415420fb90e712f0a8c91c4f0f1bc8317a7adb3bd +module/protein-ss-restraints-alpha-beta.cns bf4dafb5de2dccc0efa0bafdb5080092229a876c89181509d2e7cef3ef2bd881 +module/protein-ss-restraints-alpha.cns ce9fdf8def4a954930aea37d0ba455a17b40a3aef81848faf4098000bb8bcf8f +module/read_param.cns 0e6fe81a7f5a20920b3ec1486b546f029db5c3901e7a45cbc492dd87bbbd0576 +module/restrain-ions.cns 6c6a3ab0fb6f764fe9e4d3d7e6413dc2b359c331e34124894fd008f6da0cb1c9 +module/scale_inter_final.cns b55e7bcc5175bc7b74b5ec5c608b0dab0e71c95451f1fb7a6a120fb9a1a7c769 +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +module/setflags.cns 05609308453eee2eab0ec02b8d9fc458977aa5efac8f6ef9a61983f4426cd04b +toppar/boxtyp20.pdb b113ae0e268cfaa2c4da4b6cce33fd65259c905d8d6f44b6fe429587cd524700 +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dmso.pdb 8a7d6f535e88c312182b4d05cf1423c5e107b506d303d29bab42d9d0ace1bc43 +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c +toppar/solvent-allhdg5-4.top 4b3546f0a8ab278eb45874b4612680b96ab1460ee48afb47aebd44ac4723bb8e + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/scoring/mdscoring/cns/mdscoring.cns +2x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +2x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($per_interface_scoring=false) +eval ($w_air=0.0) +eval ($w_bsa=0.0) +eval ($w_cdih=0.0) +eval ($w_dani=0.0) +eval ($w_desolv=1.0) +eval ($w_elec=0.2) +eval ($w_lcc=-10000.0) +eval ($w_rg=0.0) +eval ($w_sani=0.0) +eval ($w_sym=0.0) +eval ($w_vdw=1.0) +eval ($w_vean=0.0) +eval ($w_xpcs=0.0) +eval ($contactairs=false) +eval ($kcont=1.0) +eval ($ssdihed="none") +eval ($error_dih=10) +eval ($dnarest_on=false) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="cdie") +eval ($epsilon=1.0) +eval ($dihedflag=true) +eval ($solvent="water") +eval ($nemsteps=200) +eval ($timestep=0.002) +eval ($waterheatsteps=100) +eval ($watersteps=1250) +eval ($watercoolsteps=500) +eval ($iniseed=917) +eval ($keepwater=false) +eval ($log_level="canonical-log-level") +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) +eval ($seed=1694244009) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) diff --git a/tests/golden_data/cns_canonical/rigidbody.canonical b/tests/golden_data/cns_canonical/rigidbody.canonical new file mode 100644 index 0000000000..fbee1f6675 --- /dev/null +++ b/tests/golden_data/cns_canonical/rigidbody.canonical @@ -0,0 +1,431 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +canonical-input-1.psf <- e2aP_1F3G_haddock.psf f74d4a039db9a47975bb295f9e73042d6328d78f5baa0cf6b0a883cb25b42fb0 +canonical-input-2.pdb <- hpr_ensemble_1_haddock.pdb 6f839075ff31b639994fe20bf42809c3a552c3a83230d16294171ce4a1a2d587 +canonical-input-2.psf <- hpr_ensemble_1_haddock.psf 936207d36b8b0987ddcd29008d0960fd173b9a95e2300685cecd1b2903bc7e08 +module/bestener.cns 116d8427578c84ac7a0602dcd8ad435111c891a8d871062e5f2e13774f579d7c +module/bsa.cns 046849150a441e8feda6c5aed6aaa4e07e3e4c6cabdbb9b6af8f45e5a8d652c7 +module/charge-beads-interactions.cns e1c75de660ab9acd54466427ad4e40166476fd4e9d9f745b0ce1456a95e3527c +module/check-homomers.cns b74acf4d25d4e3810fc19f90e80bfbdd0c845e3698be3cc01d79bb146210666b +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/cm-restraints.cns df5651f0cfa49798b856825c8a180c0994f18f0be56b7c9f8bb8165d8814489e +module/def_solv_param.cns 851c8b5f06e79bbce80a2762fc47b3b6af8dbc18d28962097f1a3d597711ccfa +module/print_coorheader.cns 01d46e002099e53216c013ee4955d5bd0843ebbdbcac8531870b4de3288b5a6c +module/random_rotations.cns b8395c968a9e5e08cb790cc898afb85c0c66efd411c0ba0d9f9c7b506d36b03b +module/randomairs.cns 06fe538edc0240cc300cb447a0d73c6d0347f8ac42c243f2cd019044f9d321ac +module/read_noes.cns 9cb0030057158db27e58df19cdae6681864592f64d95d2fd7de5227504553752 +module/read_param.cns 24b9bb87e923b9411f7335599c84328fd376fc391c889951b28b26113f9e6552 +module/rotation180.cns 2745175e18e3ebbfa12889c8e789fccb7425b7c19e393d64e556933cba7a33eb +module/scale_inter_final.cns 58f9dc67fd51a7be7b1b23992fd605a7438d01fa48cd5ed65ba2efae1cb82cd4 +module/scale_inter_mini.cns 4307c45377c1c41a9339e07904351b45f6bd16b1af89f9eb6945452d122f7c2c +module/scale_intra_only.cns 04803d9391136284acc188995d39d5684e60e8dee4d134c88611be026a2ddd0f +module/separate.cns a6976973db8b294d20f1a86234604bb5ac8249d93d61b0b9047c4057ca63e7f7 +module/surf-restraints.cns 5ec87daca5de0786c410bb0643fd3ac5808b1801980f592387587c3265d13cd5 +module/symmultimer.cns 1ba366a7df115a0d5d9f972d4651d672e63c15c26373b2075844159ab97779b7 +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/fragment_probes.param 5a48c187fef139db509e70b988c4f714074c0be84f023ce1c79b226e5dc2c3d4 +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/initial_positions/trans_vector_0 4b00414fb1f1375952e3fecbbfe0ba36e967dd12f308fd72f56a30e035679976 +toppar/initial_positions/trans_vector_1 aed8ddb81104ccf0521d0d20074fd78472c678c229d5755045a1ab75d98a45a0 +toppar/initial_positions/trans_vector_10 df6a73d152ece2cdf516bbf10742b8e9694df6eed4ef6b6ac7d2bedd0bc4edba +toppar/initial_positions/trans_vector_11 295260bf796ee666b40dd29296133b6faa4cb4cc56819d95df143ee28fe74c42 +toppar/initial_positions/trans_vector_12 8975e3c5fefe842da1afe7f5fa7b8b2900113a2c295354e505f9a45b4a65bb71 +toppar/initial_positions/trans_vector_13 67a677d6e86cbceac862e59fef40f8a8c0a57d6fc0d7a6af54cf058cdc3918f4 +toppar/initial_positions/trans_vector_14 308c3b733396f87880d28ff3e24879ad6c5cd1d86cce9a8d71cb39800fccfcef +toppar/initial_positions/trans_vector_15 e98e93d404358cbbbb76ca4f5034a8ee1183a59ff1bb787262705438d8794dc4 +toppar/initial_positions/trans_vector_16 205598c73b68f7214dc3f8e2e2f1c5e5db72543f16c33ff3aad724eecc641f25 +toppar/initial_positions/trans_vector_17 11ac7294b42c486a107790c0665f93d6a309eceea4fd670616e37073c6cdf89c +toppar/initial_positions/trans_vector_18 62109ac574a920da1652f8ca247b41cac6e3d12caa33241cef674f4b423deca2 +toppar/initial_positions/trans_vector_19 31fcfc2742bea9a41342ffdde89fca7471a840c423001638faa4d68c3f9ff035 +toppar/initial_positions/trans_vector_2 18fa2716b78aeab0cfb97ae271246e1292338015d235150d42633ecf15174dcc +toppar/initial_positions/trans_vector_20 38304ba4792223b07e05e626791c812fbd7cd87697df2987d0451f3e4db2bfb5 +toppar/initial_positions/trans_vector_21 2ec9c6dee2287441126b9754d1c7b734b49cf97762be26e9a71ada4b97067d36 +toppar/initial_positions/trans_vector_22 e97d9926b3b6bfbe19f367f5cfe0d83bd8a0f9309b3a713f64bf5dac2d188f7f +toppar/initial_positions/trans_vector_23 26f681dbeb37eaa6e3d238e34181549f1f10abb53d0241a2a425c4f2171cf786 +toppar/initial_positions/trans_vector_24 4d9b1e38820e501575def4c751ac29a8eb00f99069591904a215458393726d01 +toppar/initial_positions/trans_vector_25 8dbbfad7f0bd481ad189e79ecce24bea8e15758c843d154c227b87ee2f2bd7b7 +toppar/initial_positions/trans_vector_26 48d47c82520cb38171ae59e5bb342bba746b89259805f24cf34af9a6815177bd +toppar/initial_positions/trans_vector_27 1ce8112537b4da4d675001544c846e44199294ef6fea4f8bca34f2b5ace37dcf +toppar/initial_positions/trans_vector_28 7025d1601179ee6ac1be4ea7cacd18115c2266be6bf1123ac58d07b4b591c831 +toppar/initial_positions/trans_vector_29 f29935840ebeeac96d5a81c00109d3e816d12a620a758f8bf9ea74b62374f957 +toppar/initial_positions/trans_vector_3 a5b7915f9fef6d216e00644f90a176cf8a1c8cf52fb7d1a25dc12826bc69800c +toppar/initial_positions/trans_vector_30 f73e2ce3d732b38f59559d82971da218708faa64d8771ce32e45298a79cf42f0 +toppar/initial_positions/trans_vector_31 08da963b02cef780e97e97c704908c405dbf3c1b7aadd31eb3819356e460a780 +toppar/initial_positions/trans_vector_32 7f7464ca317d71b7e93860bf04b34ea91ac6b4b933b8b439b8cf56561fd798db +toppar/initial_positions/trans_vector_33 be9fe2ed1b4d0280aba7fdea677746a7aab295581627a2477b048e7b9651add9 +toppar/initial_positions/trans_vector_34 56e78fc220725025a7efcd94056f7efd1f81d222c419dd65cbddd0cb434217d3 +toppar/initial_positions/trans_vector_35 1d8e4d3586ca0d3bf9a517914cc85af9d121c558cf4477c0ae6a2b917011cd52 +toppar/initial_positions/trans_vector_36 a92507a9ce0d5d63453a3ba73188417677b49ca0ee50129bb76ae2daf92e1831 +toppar/initial_positions/trans_vector_37 71a6e11c2da06d4015fae6b805d8611d0453fd501f94e94e24c27b9df89d9884 +toppar/initial_positions/trans_vector_38 1beffc6fd6186b9a3a7576e036b6e8ad530a05d5b030333dcea3a23102c7089a +toppar/initial_positions/trans_vector_39 41f42e6860a0ef8e8c33bb16ace894161be418258aec2a9d3c6312c4436808a2 +toppar/initial_positions/trans_vector_4 2905f4c16d40780bee7f9dc7b0d07319742443b6d90cbd5c16bf9251b0318447 +toppar/initial_positions/trans_vector_40 33476b8202ff60cd382f1520374fa83d715f2ac19584ebec2cbfa8b225e19503 +toppar/initial_positions/trans_vector_41 e8c361ec2a6fa311d8d7ee1b1655daeb1e1bd863b28066eca2fc41794db7c78d +toppar/initial_positions/trans_vector_42 c8f5d5f4ad342e25b0e9d879a16b08d4a16ad9e146a6a7607394ed34e6321440 +toppar/initial_positions/trans_vector_43 e3885e782ff92ca0b6404e2de7e8c21f1c0c9c1024a20de9f26cebe4f74d88b7 +toppar/initial_positions/trans_vector_44 b7caee87d1c2afe3cfb3f9f7a2d47cdd37cad5a4033b7c12ddd5f4919586aad7 +toppar/initial_positions/trans_vector_45 0d0883486cc6c94c23a03999e6c43d1a0289230a7a39135ce5ad625ea3bd3d29 +toppar/initial_positions/trans_vector_46 738a431c01c5783982b092bab906bdf19dcbeb31c6338263154bdd8cc7e593ac +toppar/initial_positions/trans_vector_47 737de2d33c98e0f58d491d1521492dbef4016cb5be7733b00fd85529a455db00 +toppar/initial_positions/trans_vector_48 ad7ab4b47d97ead8b85e08332d78ee3356e02b54873060c4fe1592691c294bcb +toppar/initial_positions/trans_vector_49 418c3629030836682102893e3a18b1587742dac84388a7aaf420c2600bbdf1e8 +toppar/initial_positions/trans_vector_5 2d801d04878066986047d1559f0035ac76884d274bc3ae961caa25962038ce52 +toppar/initial_positions/trans_vector_50 7fd573a063954ebd3643a249fcb78d6586f9a24971e5975d1691660cd3ffcc0f +toppar/initial_positions/trans_vector_6 336318b9748243e0e2d421607070edbfb2c9ec7116dbf0de5c722ef5146c4a06 +toppar/initial_positions/trans_vector_7 aea642f694ba2388f2c1c201efe90772fa4f376044abe78b255bc58ad35c7192 +toppar/initial_positions/trans_vector_8 fb9671a5bc309b6145d5d2579f9edef168a2d1a4710acf16cde583ccd5c023b1 +toppar/initial_positions/trans_vector_9 324adecac5cde19a97f87217d20846fa3bb79fac33cba35fb25f5d492f83fb5d +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c + +[outputs] pdb +canonical-output.pdb + +[recipe rewrites] haddock/modules/sampling/rigidbody/cns/rigidbody.cns +1x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +1x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then +2x - elseif ( $log_level = "normal" ) then + + elseif ( $log_level = "canonical-log-level" ) then +2x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($mol_fix_origin_1=false) +eval ($mol_shape_1=false) +eval ($unambig_fname="") +eval ($hbond_fname="") +eval ($ambig_scale=50) +eval ($unambig_scale=50) +eval ($hbond_scale=50) +eval ($randremoval=true) +eval ($npart=2) +eval ($cmrest=false) +eval ($cmtight=true) +eval ($kcm=1.0) +eval ($ranair=false) +eval ($nrair_1=0) +eval ($nrair_2=0) +eval ($nrair_3=0) +eval ($nrair_4=0) +eval ($nrair_5=0) +eval ($nrair_6=0) +eval ($nrair_7=0) +eval ($nrair_8=0) +eval ($nrair_9=0) +eval ($nrair_10=0) +eval ($nrair_11=0) +eval ($nrair_12=0) +eval ($nrair_13=0) +eval ($nrair_14=0) +eval ($nrair_15=0) +eval ($nrair_16=0) +eval ($nrair_17=0) +eval ($nrair_18=0) +eval ($nrair_19=0) +eval ($nrair_20=0) +eval ($surfrest=false) +eval ($ksurf=1.0) +eval ($crossdock=true) +eval ($ntrials=5) +eval ($rotate180=true) +eval ($separate=true) +eval ($randrot=true) +eval ($rigidtrans=true) +eval ($inter_rigid=1.0) +eval ($log_level="canonical-log-level") +eval ($sym_on=false) +eval ($ksym=10.0) +eval ($symtbl_fname="") +eval ($nc2sym=0) +eval ($nc3sym=0) +eval ($nc4sym=0) +eval ($nc5sym=0) +eval ($nc6sym=0) +eval ($ns3sym=0) +eval ($w_air=0.01) +eval ($w_bsa=-0.01) +eval ($w_cdih=0.0) +eval ($w_dani=0.01) +eval ($w_deint=0.0) +eval ($w_desolv=1.0) +eval ($w_elec=1.0) +eval ($w_lcc=-400.0) +eval ($w_rg=0.1) +eval ($w_sani=0.1) +eval ($w_sym=0.1) +eval ($w_vdw=0.01) +eval ($w_vean=0.1) +eval ($w_xpcs=0.1) +eval ($w_xrdc=0.1) +eval ($w_zres=0.1) +eval ($ligand_param_fname="") +eval ($elecflag=true) +eval ($dielec="rdie") +eval ($epsilon=10.0) +eval ($epsilon_cg=10.0) +eval ($int_1_1=1.0) +eval ($int_1_2=1.0) +eval ($int_1_3=1.0) +eval ($int_1_4=1.0) +eval ($int_1_5=1.0) +eval ($int_1_6=1.0) +eval ($int_1_7=1.0) +eval ($int_1_8=1.0) +eval ($int_1_9=1.0) +eval ($int_1_10=1.0) +eval ($int_1_11=1.0) +eval ($int_1_12=1.0) +eval ($int_1_13=1.0) +eval ($int_1_14=1.0) +eval ($int_1_15=1.0) +eval ($int_1_16=1.0) +eval ($int_1_17=1.0) +eval ($int_1_18=1.0) +eval ($int_1_19=1.0) +eval ($int_1_20=1.0) +eval ($int_2_2=1.0) +eval ($int_2_3=1.0) +eval ($int_2_4=1.0) +eval ($int_2_5=1.0) +eval ($int_2_6=1.0) +eval ($int_2_7=1.0) +eval ($int_2_8=1.0) +eval ($int_2_9=1.0) +eval ($int_2_10=1.0) +eval ($int_2_11=1.0) +eval ($int_2_12=1.0) +eval ($int_2_13=1.0) +eval ($int_2_14=1.0) +eval ($int_2_15=1.0) +eval ($int_2_16=1.0) +eval ($int_2_17=1.0) +eval ($int_2_18=1.0) +eval ($int_2_19=1.0) +eval ($int_2_20=1.0) +eval ($int_3_3=1.0) +eval ($int_3_4=1.0) +eval ($int_3_5=1.0) +eval ($int_3_6=1.0) +eval ($int_3_7=1.0) +eval ($int_3_8=1.0) +eval ($int_3_9=1.0) +eval ($int_3_10=1.0) +eval ($int_3_11=1.0) +eval ($int_3_12=1.0) +eval ($int_3_13=1.0) +eval ($int_3_14=1.0) +eval ($int_3_15=1.0) +eval ($int_3_16=1.0) +eval ($int_3_17=1.0) +eval ($int_3_18=1.0) +eval ($int_3_19=1.0) +eval ($int_3_20=1.0) +eval ($int_4_4=1.0) +eval ($int_4_5=1.0) +eval ($int_4_6=1.0) +eval ($int_4_7=1.0) +eval ($int_4_8=1.0) +eval ($int_4_9=1.0) +eval ($int_4_10=1.0) +eval ($int_4_11=1.0) +eval ($int_4_12=1.0) +eval ($int_4_13=1.0) +eval ($int_4_14=1.0) +eval ($int_4_15=1.0) +eval ($int_4_16=1.0) +eval ($int_4_17=1.0) +eval ($int_4_18=1.0) +eval ($int_4_19=1.0) +eval ($int_4_20=1.0) +eval ($int_5_5=1.0) +eval ($int_5_6=1.0) +eval ($int_5_7=1.0) +eval ($int_5_8=1.0) +eval ($int_5_9=1.0) +eval ($int_5_10=1.0) +eval ($int_5_11=1.0) +eval ($int_5_12=1.0) +eval ($int_5_13=1.0) +eval ($int_5_14=1.0) +eval ($int_5_15=1.0) +eval ($int_5_16=1.0) +eval ($int_5_17=1.0) +eval ($int_5_18=1.0) +eval ($int_5_19=1.0) +eval ($int_5_20=1.0) +eval ($int_6_6=1.0) +eval ($int_6_7=1.0) +eval ($int_6_8=1.0) +eval ($int_6_9=1.0) +eval ($int_6_10=1.0) +eval ($int_6_11=1.0) +eval ($int_6_12=1.0) +eval ($int_6_13=1.0) +eval ($int_6_14=1.0) +eval ($int_6_15=1.0) +eval ($int_6_16=1.0) +eval ($int_6_17=1.0) +eval ($int_6_18=1.0) +eval ($int_6_19=1.0) +eval ($int_6_20=1.0) +eval ($int_7_7=1.0) +eval ($int_7_8=1.0) +eval ($int_7_9=1.0) +eval ($int_7_10=1.0) +eval ($int_7_11=1.0) +eval ($int_7_12=1.0) +eval ($int_7_13=1.0) +eval ($int_7_14=1.0) +eval ($int_7_15=1.0) +eval ($int_7_16=1.0) +eval ($int_7_17=1.0) +eval ($int_7_18=1.0) +eval ($int_7_19=1.0) +eval ($int_8_8=1.0) +eval ($int_8_9=1.0) +eval ($int_7_20=1.0) +eval ($int_8_10=1.0) +eval ($int_8_11=1.0) +eval ($int_8_12=1.0) +eval ($int_8_13=1.0) +eval ($int_8_14=1.0) +eval ($int_8_15=1.0) +eval ($int_8_16=1.0) +eval ($int_8_17=1.0) +eval ($int_8_18=1.0) +eval ($int_8_19=1.0) +eval ($int_8_20=1.0) +eval ($int_9_11=1.0) +eval ($int_9_12=1.0) +eval ($int_9_13=1.0) +eval ($int_9_14=1.0) +eval ($int_9_15=1.0) +eval ($int_9_16=1.0) +eval ($int_9_17=1.0) +eval ($int_9_18=1.0) +eval ($int_9_19=1.0) +eval ($int_9_20=1.0) +eval ($int_9_9=1.0) +eval ($int_9_10=1.0) +eval ($int_10_10=1.0) +eval ($int_10_11=1.0) +eval ($int_10_12=1.0) +eval ($int_10_13=1.0) +eval ($int_10_14=1.0) +eval ($int_10_15=1.0) +eval ($int_10_16=1.0) +eval ($int_10_17=1.0) +eval ($int_10_18=1.0) +eval ($int_10_19=1.0) +eval ($int_10_20=1.0) +eval ($int_11_11=1.0) +eval ($int_11_12=1.0) +eval ($int_11_13=1.0) +eval ($int_11_14=1.0) +eval ($int_11_15=1.0) +eval ($int_11_16=1.0) +eval ($int_11_17=1.0) +eval ($int_11_18=1.0) +eval ($int_11_19=1.0) +eval ($int_11_20=1.0) +eval ($int_12_12=1.0) +eval ($int_12_13=1.0) +eval ($int_12_14=1.0) +eval ($int_12_15=1.0) +eval ($int_12_16=1.0) +eval ($int_12_17=1.0) +eval ($int_12_18=1.0) +eval ($int_12_19=1.0) +eval ($int_12_20=1.0) +eval ($int_13_13=1.0) +eval ($int_13_14=1.0) +eval ($int_13_15=1.0) +eval ($int_13_16=1.0) +eval ($int_13_17=1.0) +eval ($int_13_18=1.0) +eval ($int_13_19=1.0) +eval ($int_13_20=1.0) +eval ($int_14_14=1.0) +eval ($int_14_15=1.0) +eval ($int_14_16=1.0) +eval ($int_14_17=1.0) +eval ($int_14_18=1.0) +eval ($int_14_19=1.0) +eval ($int_14_20=1.0) +eval ($int_15_15=1.0) +eval ($int_15_16=1.0) +eval ($int_15_17=1.0) +eval ($int_15_18=1.0) +eval ($int_15_19=1.0) +eval ($int_15_20=1.0) +eval ($int_16_16=1.0) +eval ($int_16_17=1.0) +eval ($int_16_18=1.0) +eval ($int_16_19=1.0) +eval ($int_16_20=1.0) +eval ($int_17_17=1.0) +eval ($int_17_18=1.0) +eval ($int_17_19=1.0) +eval ($int_17_20=1.0) +eval ($int_18_18=1.0) +eval ($int_18_19=1.0) +eval ($int_18_20=1.0) +eval ($int_19_19=1.0) +eval ($int_19_20=1.0) +eval ($int_20_20=1.0) +eval ($mol_fix_origin_2=false) +eval ($mol_shape_2=false) +eval ($ambig_fname="") + +! Input structure +structure + @@canonical-input-1.psf +end +structure + @@canonical-input-2.psf +end +coor @@canonical-input-1.pdb +eval ($input_pdb_filename_1="canonical-input-1.pdb") +coor @@canonical-input-2.pdb +eval ($input_pdb_filename_2="canonical-input-2.pdb") +eval ($ncomponents=2) +eval ($seed=1694244009) + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($count=canonical-count) +eval ($prot_segid_1="A") +eval ($prot_segid_2="B") diff --git a/tests/golden_data/cns_canonical/topoaa.canonical b/tests/golden_data/cns_canonical/topoaa.canonical new file mode 100644 index 0000000000..a8ae22b2ce --- /dev/null +++ b/tests/golden_data/cns_canonical/topoaa.canonical @@ -0,0 +1,162 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +module/auto-his.cns b1dcea30172f407301d6d32d6e443f64a3a65a8a59b55be6d2936a38a28825d6 +module/bondglycans.cns 0a691f27b8a660c4c52839a425cddb694fb5b78150d656c8a06daaf5bd0a6099 +module/build-missing.cns 06699840d0d32c51b00241c97aba2355bfe59f02ba0baed20b327f01a8bd3434 +module/check_coordinates.cns 7622d4673dd7c4c31bd6da3039a79694646248d87bd9d801e685fab55e7bb458 +module/coval-ace-cys.cns 303bacf49c68b75c1527ec003a609f4faa82cd7fdd0c49bacbe1501bbede576e +module/covalheme.cns 3ed3e92284c11455187a0e7e990e2cd3c579ed27af60513c8e0e051624d807f3 +module/dna_break.cns 6017cc86641520583f6d947b205a7a2b2085d2e6e10684372525d6fbceec8451 +module/prot_break.cns 0d9caaf320c3df27e72f481f586098bfa84430d709f6d72b655df857ed8c6dd7 +toppar/carbohydrate.param a2796cd04f3957c4f3ba078daf7a6e5afb0e14f1c820ae5adf6dfa8d4ba00007 +toppar/carbohydrate.top 7b5b0d01b79de88e53248da90beb2c0675107bd6c96f485f5fe3beedbf36f46a +toppar/cofactors.param cec99dbdf652fcce524294f6c4e3da0add37997a8ce18d14320ef192d9f1d7c5 +toppar/cofactors.top 5b17d7b98dbd1195467d1959a86e996f37792ef33315f92bd944120fed805de9 +toppar/dna-rna-1.3.link 16e80159790c62b73a9b4c26a6daf4d5b28b08f0bc2a64b98e6b750488f7ca55 +toppar/dna-rna-allatom-hj-opls-1.3.param e56e4f1c0f6f245bebcdb0d6ae70a5f0bd9af08869f4f4f784d1194f29044cdd +toppar/dna-rna-allatom-hj-opls-1.3.top 04a0026d1e5d9a8df207fa2bff32ccffb03a7ab20b81d8b0c92747251a4cdb10 +toppar/dna_break.top 8cd5a4f7c3088a2b9502ff83ee76ec0b3e154496f2c5703b98e79f39897427bf +toppar/hemes-allhdg.param b5678c12c376ec118d9bb4b121f3494462e453702c739d4b5098527053ca2aa5 +toppar/hemes-allhdg.top dcd467d6bc003a74bd31e56b436ff6f02eb297796b691efe80bd8fa4a3c1aad5 +toppar/ion.param 86fd006434fdfb5a0d6bc9903759d127a0aa92de603f0d57deefd0c2715593ee +toppar/ion.top b5458992669f361e2af6d8839eb7c674080c4fef3b449f065509f37d597bfdfc +toppar/protein-allhdg5-4-noter.link d6d88ae174352edd8ee62a5d33ac36841410cde0e9efe27e95e3e191d3cdacb0 +toppar/protein-allhdg5-4.param 0f8cc2a24cb44363c23b53f4107a881929eefa2a1afda62fad35c9816455b175 +toppar/protein-allhdg5-4.top d5cdfb0574bab46009088309cb6472d8771efa3eda05f46c0d33e7cddc571651 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/shape.top 7b8577dbf235defb4d3c67492e2856821fa558dcc548a296374267b1c81d64d4 +toppar/solvent-allhdg5-4.param 05a6a9b961b4a9495342352f6200794198db12b9d154723edc7b898b8783434c +toppar/solvent-allhdg5-4.top 4b3546f0a8ab278eb45874b4612680b96ab1460ee48afb47aebd44ac4723bb8e + +[outputs] pdb+psf +canonical-output.pdb +canonical-output.psf + +[recipe rewrites] haddock/modules/topology/topoaa/cns/generate-topology.cns +1x - elseif ( $log_level = "normal") then + + elseif ( $log_level = "canonical-log-level") then +1x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($autohis=true) +eval ($delenph=true) +eval ($hydrogen_build="all") +eval ($log_level="canonical-log-level") +eval ($iniseed=917) +eval ($ligand_param_fname="") +eval ($ligand_top_fname="") +eval ($set_bfactor=true) +eval ($cyclicpept_dist=2.0) +eval ($disulphide_dist=2.2) +eval ($acecys_dist=4.0) +eval ($cyclicpept=false) +eval ($nhisd=0) +eval ($nhise=0) +eval ($prot_link_infile="TOPPAR:protein-allhdg5-4-noter.link") +eval ($nucl_link_infile="TOPPAR:dna-rna-1.3.link") + +! Input structure +eval ($file="canonical-input-1.pdb") +eval ($ncomponents=1) +eval ($prot_segid_1="A") + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($output_psf_filename="canonical-output.psf") + +! Translation vectors +eval ($trans_vector_0="TOPPAR:initial_positions/trans_vector_0") +eval ($trans_vector_1="TOPPAR:initial_positions/trans_vector_1") +eval ($trans_vector_2="TOPPAR:initial_positions/trans_vector_2") +eval ($trans_vector_3="TOPPAR:initial_positions/trans_vector_3") +eval ($trans_vector_4="TOPPAR:initial_positions/trans_vector_4") +eval ($trans_vector_5="TOPPAR:initial_positions/trans_vector_5") +eval ($trans_vector_6="TOPPAR:initial_positions/trans_vector_6") +eval ($trans_vector_7="TOPPAR:initial_positions/trans_vector_7") +eval ($trans_vector_8="TOPPAR:initial_positions/trans_vector_8") +eval ($trans_vector_9="TOPPAR:initial_positions/trans_vector_9") +eval ($trans_vector_10="TOPPAR:initial_positions/trans_vector_10") +eval ($trans_vector_11="TOPPAR:initial_positions/trans_vector_11") +eval ($trans_vector_12="TOPPAR:initial_positions/trans_vector_12") +eval ($trans_vector_13="TOPPAR:initial_positions/trans_vector_13") +eval ($trans_vector_14="TOPPAR:initial_positions/trans_vector_14") +eval ($trans_vector_15="TOPPAR:initial_positions/trans_vector_15") +eval ($trans_vector_16="TOPPAR:initial_positions/trans_vector_16") +eval ($trans_vector_17="TOPPAR:initial_positions/trans_vector_17") +eval ($trans_vector_18="TOPPAR:initial_positions/trans_vector_18") +eval ($trans_vector_19="TOPPAR:initial_positions/trans_vector_19") +eval ($trans_vector_20="TOPPAR:initial_positions/trans_vector_20") +eval ($trans_vector_21="TOPPAR:initial_positions/trans_vector_21") +eval ($trans_vector_22="TOPPAR:initial_positions/trans_vector_22") +eval ($trans_vector_23="TOPPAR:initial_positions/trans_vector_23") +eval ($trans_vector_24="TOPPAR:initial_positions/trans_vector_24") +eval ($trans_vector_25="TOPPAR:initial_positions/trans_vector_25") +eval ($trans_vector_26="TOPPAR:initial_positions/trans_vector_26") +eval ($trans_vector_27="TOPPAR:initial_positions/trans_vector_27") +eval ($trans_vector_28="TOPPAR:initial_positions/trans_vector_28") +eval ($trans_vector_29="TOPPAR:initial_positions/trans_vector_29") +eval ($trans_vector_30="TOPPAR:initial_positions/trans_vector_30") +eval ($trans_vector_31="TOPPAR:initial_positions/trans_vector_31") +eval ($trans_vector_32="TOPPAR:initial_positions/trans_vector_32") +eval ($trans_vector_33="TOPPAR:initial_positions/trans_vector_33") +eval ($trans_vector_34="TOPPAR:initial_positions/trans_vector_34") +eval ($trans_vector_35="TOPPAR:initial_positions/trans_vector_35") +eval ($trans_vector_36="TOPPAR:initial_positions/trans_vector_36") +eval ($trans_vector_37="TOPPAR:initial_positions/trans_vector_37") +eval ($trans_vector_38="TOPPAR:initial_positions/trans_vector_38") +eval ($trans_vector_39="TOPPAR:initial_positions/trans_vector_39") +eval ($trans_vector_40="TOPPAR:initial_positions/trans_vector_40") +eval ($trans_vector_41="TOPPAR:initial_positions/trans_vector_41") +eval ($trans_vector_42="TOPPAR:initial_positions/trans_vector_42") +eval ($trans_vector_43="TOPPAR:initial_positions/trans_vector_43") +eval ($trans_vector_44="TOPPAR:initial_positions/trans_vector_44") +eval ($trans_vector_45="TOPPAR:initial_positions/trans_vector_45") +eval ($trans_vector_46="TOPPAR:initial_positions/trans_vector_46") +eval ($trans_vector_47="TOPPAR:initial_positions/trans_vector_47") +eval ($trans_vector_48="TOPPAR:initial_positions/trans_vector_48") +eval ($trans_vector_49="TOPPAR:initial_positions/trans_vector_49") +eval ($trans_vector_50="TOPPAR:initial_positions/trans_vector_50") + +! Tensors +eval ($tensor_psf="TOPPAR:tensor.psf") +eval ($tensor_pdb="TOPPAR:tensor.pdb") +eval ($tensor_para_psf="TOPPAR:tensor_para.psf") +eval ($tensor_para_pdb="TOPPAR:tensor_para.pdb") +eval ($tensor_dani_psf="TOPPAR:tensor_dani.psf") +eval ($tensor_dani_pdb="TOPPAR:tensor_dani.pdb") + +! Scatter lib +eval ($scatter_lib="TOPPAR:scatter.lib") + +! Axis +eval ($top_axis="TOPPAR:top_axis.pro") +eval ($par_axis="TOPPAR:par_axis.pro") +eval ($top_axis_dani="TOPPAR:top_axis_dani.pro") + +! Water box +eval ($boxtyp20="TOPPAR:boxtyp20.pdb") diff --git a/tests/golden_data/cns_canonical/topocg.canonical b/tests/golden_data/cns_canonical/topocg.canonical new file mode 100644 index 0000000000..c568a22e86 --- /dev/null +++ b/tests/golden_data/cns_canonical/topocg.canonical @@ -0,0 +1,143 @@ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. + +[pins] +canonical-input-1.pdb <- e2aP_1F3G_haddock.pdb d2c5082f8d461803a84c3ecc9b02f25948fff8dddf25d98690f93b023313c240 +module/patch-bb-cg.cns 2e713b6a19d28b46fd62dbe78a4d6b1ceebbca83eaa79204b675f0ed19748a20 +module/patch-breaks-cg-dna-rna.cns 754f47a661d895a4375f609855a552130ebda0b8d0d4eea4ccb85671fabb9a2f +module/patch-types-cg-hbond-dna-rna.cns c88c98b97bca3e988f9add39aa81d22c5e2aa13c54fe7a5d0d7f4140607c34a5 +module/patch-types-cg.cns b5a02024108c08e82365558f9dbc5f388e8ddbb3e664275651eb39271d830b50 +module/prot_break.cns 0d9caaf320c3df27e72f481f586098bfa84430d709f6d72b655df857ed8c6dd7 +toppar/dna-rna-CG-MARTINI-2-1p.link 485ffd0c35108152263bfc210a3a29f24226c38a310266a2a73b6267635e07df +toppar/dna-rna-CG-MARTINI-2-1p.param 95a46c3d71a64ebff2244896cfd7faa705fa6780445d6b56e0f04c00d877854b +toppar/dna-rna-CG-MARTINI-2-1p.top 8b48e573090ddd1d6684dee37372292eec78b66562b5532163cdd9d2191a10f3 +toppar/protein-CG-Martini-2-2.link 47e246c6e8a301b8e9ab92f68a48e5da09d0fefaf1fd12f9299bdbf336bbf054 +toppar/protein-CG-Martini-2-2.param 4ec92ffa79d3a5a1a7315485fdaa3ff18d4efe1dba6e2368b2ffe760955a48cf +toppar/protein-CG-Martini-2-2.top 3ed2bd919489da8f4682cff556f845d0c0b4e95c661be40fcef3efefb35b4946 +toppar/shape.param dcdb35ed2e1247a5aa5cbaba854d7ec9bd520d59782f58b8a397a3b001cb8185 +toppar/shape.top 7b8577dbf235defb4d3c67492e2856821fa558dcc548a296374267b1c81d64d4 + +[outputs] pdb+psf +canonical-output.pdb +canonical-output.psf + +[recipe rewrites] haddock/modules/topology/topocg/cns/generate-topology.cns +1x - elseif ( $log_level = "normal") then + + elseif ( $log_level = "canonical-log-level") then +1x - if ( $log_level = "verbose" ) then + + if ( $log_level = "canonical-log-level" ) then + +[canonical parameter header] + +! Parameters +eval ($cgffversion="martini2") +eval ($log_level="canonical-log-level") +eval ($iniseed=917) +eval ($ligand_param_fname="") +eval ($ligand_top_fname="") +eval ($cyclicpept=false) +eval ($nhisd=0) +eval ($nhise=0) + +! Input structure +eval ($file="canonical-input-1.pdb") +eval ($ncomponents=1) +eval ($prot_segid_1="A") + +! Output structure +eval ($output_pdb_filename="canonical-output.pdb") +eval ($output_psf_filename="canonical-output.psf") + +! Link file +eval ($prot_link_infile="TOPPAR:protein-allhdg5-4-noter.link") + +! Translation vectors +eval ($trans_vector_0="TOPPAR:initial_positions/trans_vector_0") +eval ($trans_vector_1="TOPPAR:initial_positions/trans_vector_1") +eval ($trans_vector_2="TOPPAR:initial_positions/trans_vector_2") +eval ($trans_vector_3="TOPPAR:initial_positions/trans_vector_3") +eval ($trans_vector_4="TOPPAR:initial_positions/trans_vector_4") +eval ($trans_vector_5="TOPPAR:initial_positions/trans_vector_5") +eval ($trans_vector_6="TOPPAR:initial_positions/trans_vector_6") +eval ($trans_vector_7="TOPPAR:initial_positions/trans_vector_7") +eval ($trans_vector_8="TOPPAR:initial_positions/trans_vector_8") +eval ($trans_vector_9="TOPPAR:initial_positions/trans_vector_9") +eval ($trans_vector_10="TOPPAR:initial_positions/trans_vector_10") +eval ($trans_vector_11="TOPPAR:initial_positions/trans_vector_11") +eval ($trans_vector_12="TOPPAR:initial_positions/trans_vector_12") +eval ($trans_vector_13="TOPPAR:initial_positions/trans_vector_13") +eval ($trans_vector_14="TOPPAR:initial_positions/trans_vector_14") +eval ($trans_vector_15="TOPPAR:initial_positions/trans_vector_15") +eval ($trans_vector_16="TOPPAR:initial_positions/trans_vector_16") +eval ($trans_vector_17="TOPPAR:initial_positions/trans_vector_17") +eval ($trans_vector_18="TOPPAR:initial_positions/trans_vector_18") +eval ($trans_vector_19="TOPPAR:initial_positions/trans_vector_19") +eval ($trans_vector_20="TOPPAR:initial_positions/trans_vector_20") +eval ($trans_vector_21="TOPPAR:initial_positions/trans_vector_21") +eval ($trans_vector_22="TOPPAR:initial_positions/trans_vector_22") +eval ($trans_vector_23="TOPPAR:initial_positions/trans_vector_23") +eval ($trans_vector_24="TOPPAR:initial_positions/trans_vector_24") +eval ($trans_vector_25="TOPPAR:initial_positions/trans_vector_25") +eval ($trans_vector_26="TOPPAR:initial_positions/trans_vector_26") +eval ($trans_vector_27="TOPPAR:initial_positions/trans_vector_27") +eval ($trans_vector_28="TOPPAR:initial_positions/trans_vector_28") +eval ($trans_vector_29="TOPPAR:initial_positions/trans_vector_29") +eval ($trans_vector_30="TOPPAR:initial_positions/trans_vector_30") +eval ($trans_vector_31="TOPPAR:initial_positions/trans_vector_31") +eval ($trans_vector_32="TOPPAR:initial_positions/trans_vector_32") +eval ($trans_vector_33="TOPPAR:initial_positions/trans_vector_33") +eval ($trans_vector_34="TOPPAR:initial_positions/trans_vector_34") +eval ($trans_vector_35="TOPPAR:initial_positions/trans_vector_35") +eval ($trans_vector_36="TOPPAR:initial_positions/trans_vector_36") +eval ($trans_vector_37="TOPPAR:initial_positions/trans_vector_37") +eval ($trans_vector_38="TOPPAR:initial_positions/trans_vector_38") +eval ($trans_vector_39="TOPPAR:initial_positions/trans_vector_39") +eval ($trans_vector_40="TOPPAR:initial_positions/trans_vector_40") +eval ($trans_vector_41="TOPPAR:initial_positions/trans_vector_41") +eval ($trans_vector_42="TOPPAR:initial_positions/trans_vector_42") +eval ($trans_vector_43="TOPPAR:initial_positions/trans_vector_43") +eval ($trans_vector_44="TOPPAR:initial_positions/trans_vector_44") +eval ($trans_vector_45="TOPPAR:initial_positions/trans_vector_45") +eval ($trans_vector_46="TOPPAR:initial_positions/trans_vector_46") +eval ($trans_vector_47="TOPPAR:initial_positions/trans_vector_47") +eval ($trans_vector_48="TOPPAR:initial_positions/trans_vector_48") +eval ($trans_vector_49="TOPPAR:initial_positions/trans_vector_49") +eval ($trans_vector_50="TOPPAR:initial_positions/trans_vector_50") + +! Tensors +eval ($tensor_psf="TOPPAR:tensor.psf") +eval ($tensor_pdb="TOPPAR:tensor.pdb") +eval ($tensor_para_psf="TOPPAR:tensor_para.psf") +eval ($tensor_para_pdb="TOPPAR:tensor_para.pdb") +eval ($tensor_dani_psf="TOPPAR:tensor_dani.psf") +eval ($tensor_dani_pdb="TOPPAR:tensor_dani.pdb") + +! Scatter lib +eval ($scatter_lib="TOPPAR:scatter.lib") + +! Axis +eval ($top_axis="TOPPAR:top_axis.pro") +eval ($par_axis="TOPPAR:par_axis.pro") +eval ($top_axis_dani="TOPPAR:top_axis_dani.pro") + +! Water box +eval ($boxtyp20="TOPPAR:boxtyp20.pdb") diff --git a/tests/test_generated_cns_canonical.py b/tests/test_generated_cns_canonical.py new file mode 100644 index 0000000000..63fd9654bc --- /dev/null +++ b/tests/test_generated_cns_canonical.py @@ -0,0 +1,390 @@ +"""Golden canonical forms built from real HADDOCK CNS inputs.""" + +import os +from collections import Counter +from copy import deepcopy +from difflib import SequenceMatcher +from pathlib import Path + +import pytest + +import haddock +from haddock.libs.libcns import ( + derive_seed, + find_desired_linkfiles, + prepare_cns_input, +) +from haddock.libs.libontology import Format, PDBFile, Persistent +from haddock.libs.libsubprocess import CNSJob +from haddock.modules.refinement.cgtoaa import HaddockModule as Cgtoaa +from haddock.modules.refinement.emref import HaddockModule as Emref +from haddock.modules.refinement.flexref import HaddockModule as Flexref +from haddock.modules.refinement.mdref import HaddockModule as Mdref +from haddock.modules.sampling.rigidbody import HaddockModule as Rigidbody +from haddock.modules.scoring.emscoring import HaddockModule as Emscoring +from haddock.modules.scoring.mdscoring import HaddockModule as Mdscoring +from haddock.modules.topology.topoaa import HaddockModule as Topoaa +from haddock.modules.topology.topoaa import generate_topology as generate_topoaa +from haddock.modules.topology.topocg import HaddockModule as Topocg +from haddock.modules.topology.topocg import generate_topology as generate_topocg + +from . import golden_data + + +SOURCE_GOLDEN_DATA = Path(golden_data).resolve() +GOLDEN_DIR = SOURCE_GOLDEN_DATA / "cns_canonical" +#: The shapes whose recipes read ``$seed``. +SEEDED_SHAPES = frozenset( + {"rigidbody", "flexref", "emref", "mdref", "mdscoring"} +) +GENERIC_MODULES = { + "rigidbody": Rigidbody, + "flexref": Flexref, + "emref": Emref, + "mdref": Mdref, + "emscoring": Emscoring, + "mdscoring": Mdscoring, +} + + +@pytest.mark.parametrize( + ("module_class", "constructed_parameters"), + ( + ( + Rigidbody, + { + "int_1_2", + "nrair_1", + "rair_sta_1_1", + "rair_end_1_1", + "c6sym_seg6_1", + }, + ), + ( + Flexref, + { + "int_1_2", + "seg_sta_1_1", + "seg_end_1_1", + "nseg1", + "ncs_sta1_1", + "c6sym_seg6_1", + }, + ), + ( + Emref, + { + "int_1_2", + "seg_sta_1_1", + "seg_end_1_1", + "nseg1", + "ncs_sta1_1", + "c6sym_seg6_1", + }, + ), + ( + Mdref, + { + "int_1_2", + "seg_sta_1_1", + "seg_end_1_1", + "nseg1", + "ncs_sta1_1", + "c6sym_seg6_1", + }, + ), + ( + Cgtoaa, + {"int_1_2", "seg_sta_1_1", "seg_end_1_1", "nseg1"}, + ), + ), +) +def test_constructed_cns_parameter_families_are_included( + module_class, + constructed_parameters, + tmp_path, +): + """Every affected module binds representative spliced CNS symbols.""" + module = module_class(0, tmp_path) + + assert constructed_parameters <= module.cns_params().keys() + + +@pytest.mark.parametrize( + "shape", + ( + "topoaa", + "topocg", + "rigidbody", + "flexref", + "emref", + "mdref", + "emscoring", + "mdscoring", + "cgtoaa", + ), +) +def test_generated_cns_input_matches_canonical_golden(shape, tmp_path, monkeypatch): + """Canonicalize a generated production input and compare its canonical form.""" + input_path = _stage_inputs(tmp_path) + work_path = tmp_path / "step" + work_path.mkdir() + monkeypatch.chdir(work_path) + mapping, script, module = _generated_mapping(shape, work_path, input_path) + observed = _canonical_golden_text(mapping, script, module) + golden = GOLDEN_DIR / f"{shape}.canonical" + if os.environ.get("HADDOCK_UPDATE_CNS_GOLDENS") == "1": + golden.parent.mkdir(parents=True, exist_ok=True) + golden.write_text(observed, encoding="utf-8") + + assert observed == golden.read_text(encoding="utf-8") + + +#: What a golden form is, for whoever reads one in a review. +GOLDEN_PREAMBLE = """\ +# The canonical form of one real generated CNS job, in reviewable pieces. +# +# [pins] binds each canonical name to the file that occupies it, by basename +# and content checksum. Both halves are identity: which pin an input occupies +# and what that input contains. A change to a file a recipe includes moves a +# checksum here, which is the only place it would otherwise be invisible. +# +# [outputs] is the declared output shape. +# +# [recipe rewrites] is everything canonicalization does to the module's CNS +# recipe, as before/after pairs with the number of places each occurs. The +# recipe itself is not copied here -- it is in the tree already, and copying it +# would mirror every edit to it into a file that has no opinion about the +# change. Editing a recipe therefore leaves this section alone unless it +# changes what canonicalization has to do. +# +# [canonical parameter header] is the generated part of the input -- the +# parameter set, the seed, the pin references, the erasures -- which exists +# nowhere else, so it is kept verbatim. +# +# Regenerate with HADDOCK_UPDATE_CNS_GOLDENS=1; the diff is the review. +""" + + +def _canonical_golden_text(mapping, script: str, module) -> str: + """Render the canonical form of one generated job as a reviewable file.""" + canonical_lines = mapping.canonical_script.splitlines(keepends=True) + generated_lines = script.splitlines(keepends=True) + recipe_lines = module.recipe_str.splitlines(keepends=True) + + # A module recipe is spliced into the generated input unchanged, as its + # tail. That is what makes it separable from the generated part at all, + # so it is asserted rather than assumed. + header_length = len(generated_lines) - len(recipe_lines) + assert generated_lines[header_length:] == recipe_lines + + boundary, rewrites = _recipe_rewrites( + generated_lines, canonical_lines, header_length + ) + + sections = [GOLDEN_PREAMBLE, "\n[pins]\n"] + for dependency in sorted( + mapping.dependencies, key=lambda dependency: dependency.canonical_name + ): + # An install-tree pin is named after the file already; only a job + # input needs to say which file it stands for. + name = dependency.original_path.name + binding = dependency.canonical_name + if not binding.endswith(name): + binding = f"{binding} <- {name}" + sections.append(f"{binding} {dependency.checksum}\n") + sections.append(f"\n[outputs] {mapping.output_shape}\n") + sections.extend(f"{name}\n" for name in mapping.canonical_output_names) + sections.append(f"\n[recipe rewrites] {_recipe_label(module)}\n") + if not rewrites: + sections.append("(none)\n") + for (before, after), count in sorted(rewrites.items()): + source, canonical = before.rstrip("\n"), after.rstrip("\n") + sections.append(f"{count}x - {source}\n") + sections.append(f" + {canonical}\n") + sections.append("\n[canonical parameter header]\n") + sections.append("".join(canonical_lines[:boundary])) + return "".join(sections) + + +def _recipe_rewrites(generated_lines, canonical_lines, header_length): + """Find the recipe in the canonical script and count what changed in it. + + Canonicalization drops whole assignments out of the generated header, so + the two scripts do not line up by index and are aligned instead. Within + the recipe it only rewrites lines in place; a rule that started adding or + removing lines there would misalign these sections silently, so that is + asserted too. + """ + matcher = SequenceMatcher(None, generated_lines, canonical_lines, autojunk=False) + boundary = None + rewrites: Counter = Counter() + for tag, source, source_end, canonical, canonical_end in matcher.get_opcodes(): + if source_end <= header_length: + continue + if source < header_length: + assert tag == "equal", "the recipe boundary is inside a rewritten block" + boundary = canonical + (header_length - source) + continue + if boundary is None: + boundary = canonical + if tag == "equal": + continue + assert tag == "replace" and source_end - source == canonical_end - canonical, ( + "canonicalization added or removed a line inside the recipe" + ) + rewrites.update( + zip( + generated_lines[source:source_end], + canonical_lines[canonical:canonical_end], + ) + ) + assert boundary is not None + return boundary, rewrites + + +def _recipe_label(module) -> str: + """Name a module recipe by its place in the package, not on this machine.""" + package_root = Path(haddock.__file__).resolve().parent.parent + return str(Path(module.cns_protocol_path).resolve().relative_to(package_root)) + + +def _stage_inputs(tmp_path: Path) -> Path: + """Create the sibling input directory used by production step paths.""" + input_path = tmp_path / "inputs" + input_path.mkdir() + sources = { + name: SOURCE_GOLDEN_DATA / name + for name in ( + "e2aP_1F3G_haddock.pdb", + "e2aP_1F3G_haddock.psf", + "hpr_ensemble_1_haddock.pdb", + "hpr_ensemble_1_haddock.psf", + "example_ambig_1.tbl", + ) + } + integration_data = Path(__file__).parents[1] / "integration_tests" / "golden_data" + sources.update( + { + name: integration_data / name + for name in ("e2a_haddock_cg.pdb", "e2a_haddock_cg.psf") + } + ) + for name, source in sources.items(): + (input_path / name).symlink_to(source.resolve()) + return input_path + + +def _generated_mapping(shape: str, work_path: Path, input_path: Path): + if shape == "topoaa": + module = Topoaa(0, work_path) + input_pdb = input_path / "e2aP_1F3G_haddock.pdb" + mol_params = deepcopy(module.params["mol1"]) + charged_nter = mol_params.pop("charged_nter") + charged_cter = mol_params.pop("charged_cter") + phosphate_5 = mol_params.pop("5_phosphate") + mol_params.update( + find_desired_linkfiles( + charged_nter, + charged_cter, + phosphate_5, + module.toppar_path, + ) + ) + script = generate_topoaa( + input_pdb, + module.recipe_str, + module.cns_params(), + mol_params, + default_params_path=module.toppar_path, + write_to_disk=False, + ) + output_stem = f"{input_pdb.stem}_haddock" + outputs = [Path(f"{output_stem}.pdb"), Path(f"{output_stem}.psf")] + elif shape == "topocg": + module = Topocg(0, work_path) + input_pdb = input_path / "e2aP_1F3G_haddock.pdb" + script = generate_topocg( + input_pdb, + str(work_path), + module.recipe_str, + module.cns_params(), + module.params["mol1"], + default_params_path=module.toppar_path, + write_to_disk=False, + shape=True, + ) + outputs = [Path(input_pdb.name), Path(f"{input_pdb.stem}.psf")] + else: + module, input_element, cgtoaa = _generic_input( + shape, + work_path, + input_path, + ) + # The seeded shapes are given the seed production derives for them, + # so the golden form pins the derivation as well as the layout. + seed = ( + derive_seed(module.params["iniseed"], input_element) + if shape in SEEDED_SHAPES + else None + ) + script = prepare_cns_input( + 1, + input_element, + work_path, + module.recipe_str, + module.cns_params(), + shape, + default_params_path=module.toppar_path, + native_segid=shape == "rigidbody", + cgtoaa=cgtoaa, + seed=seed, + ) + outputs = [Path(f"{shape}_1.pdb")] + + job = CNSJob( + script, + envvars=module.default_envvars(), + output_files=outputs, + ) + return job.canonical_mapping(), script, module + + +def _generic_input(shape: str, work_path: Path, input_path: Path): + if shape == "cgtoaa": + module = Cgtoaa(0, work_path) + model = PDBFile( + file_name="e2a_haddock_cg.pdb", + path=input_path, + topology=Persistent( + file_name="e2a_haddock_cg.psf", + path=input_path, + file_type=Format.TOPOLOGY, + ), + ) + model.aa_topology = Persistent( + file_name="e2aP_1F3G_haddock.psf", + path=input_path, + file_type=Format.TOPOLOGY, + ) + model.cgtoaa_tbl = (input_path / "example_ambig_1.tbl").resolve() + return module, model, True + + module = GENERIC_MODULES[shape](0, work_path) + inputs = [ + _model("e2aP_1F3G_haddock", input_path), + _model("hpr_ensemble_1_haddock", input_path), + ] + return module, inputs, False + + +def _model(stem: str, input_path: Path) -> PDBFile: + return PDBFile( + file_name=f"{stem}.pdb", + path=input_path, + topology=Persistent( + file_name=f"{stem}.psf", + path=input_path, + file_type=Format.TOPOLOGY, + ), + ) diff --git a/tests/test_libcns.py b/tests/test_libcns.py index 640d84045d..a41112e8fa 100644 --- a/tests/test_libcns.py +++ b/tests/test_libcns.py @@ -7,9 +7,11 @@ import pytest +import haddock.modules from haddock import EmptyPath from haddock.libs import libcns from haddock.libs.libcns import ( + _add_cg_backmapping_arguments, prepare_cns_input, prepare_expected_pdb, prepare_multiple_input, @@ -151,6 +153,73 @@ def test_prepare_cns_input(pdbfile): assert observed_cns_input == expected_cns_input +def test_prepare_cns_input_omits_seed_when_one_is_not_declared(pdbfile): + observed_cns_input = prepare_cns_input( + model_number=1, + input_element=pdbfile, + step_path=Path("."), + recipe_str="", + defaults={}, + identifier="model", + seed=None, + ) + + assert "$seed" not in observed_cns_input + + +def test_prepare_cns_input_replaces_archive_restraint_assignment(pdbfile): + observed_cns_input = prepare_cns_input( + model_number=1, + input_element=pdbfile, + step_path=Path("."), + recipe_str="", + defaults={"ambig_fname": Path("ambig.tbl.tgz")}, + identifier="model", + ambig_fname=Path("ambig_1.tbl"), + ) + + assert "ambig.tbl.tgz" not in observed_cns_input + assert observed_cns_input.count("$ambig_fname") == 1 + assert 'eval ($ambig_fname="ambig_1.tbl")' in observed_cns_input + + +def test_prepare_cns_input_expands_molecule_defaults_for_direct_call(pdbfile, mocker): + """Direct callers receive one molecule-family value per component.""" + mocker.patch( + "haddock.libs.libpdb.identify_chainseg", + return_value=(["A"], []), + ) + + observed = prepare_cns_input( + model_number=1, + input_element=[pdbfile, pdbfile], + step_path=Path("."), + recipe_str="", + defaults={"mol_shape_1": False}, + identifier="model", + ) + + assert "eval ($mol_shape_1=false)" in observed + assert "eval ($mol_shape_2=false)" in observed + + +def test_cg_backmapping_keeps_psf_tbl_pairs_aligned_with_shapes(tmp_path): + """A leading shape component must not shift CG-to-AA file pairings.""" + model = PDBFile(file_name="complex.pdb", path=tmp_path) + model.aa_topology = [ + Persistent("shape.psf", Format.TOPOLOGY, tmp_path), + Persistent("protein.psf", Format.TOPOLOGY, tmp_path), + ] + model.cgtoaa_tbl = [None, Path("protein.tbl")] + model.shape = [True, False] + + observed = _add_cg_backmapping_arguments(model) + + assert "shape.psf" not in observed + assert f'input_aa_psf_filename_1="{model.aa_topology[1].rel_path}"' in observed + assert 'input_cgtbl_filename_1="protein.tbl"' in observed + + def test_prepare_multiple_input(mocker): mocker.patch("haddock.libs.libpdb.identify_chainseg", return_value="A") @@ -223,8 +292,135 @@ def test_prepare_cns_input_with_ligand_files(): seed=pdb.seed, ) - # Check that ligand files are included in the CNS input - assert "ligand.top" in cns_input + # The per-model ligand *parameter* file is included: every recipe + # reached through `prepare_cns_input` reads it. assert "ligand.param" in cns_input - assert 'eval ($ligand_top_fname="ligand.top")' in cns_input assert 'eval ($ligand_param_fname="ligand.param")' in cns_input + + # The ligand *topology* file is not. Only the two topology recipes + # read it, and neither of them comes through here, so emitting it + # would leave a step-folder path in the generated input that CNS never + # opens. The topology it describes is already in the PSF by now. + assert "ligand.top" not in cns_input + assert "$ligand_top_fname" not in cns_input + + +def _seeded_model(directory: Path, name: str, content: str) -> PDBFile: + """A model whose bytes are on disk, so its seed can be derived.""" + path = Path(directory, name) + path.write_text(content, encoding="utf-8") + return PDBFile(file_name=name, path=directory) + + +def test_derive_seed_ignores_the_job_position_and_name(tmp_path, monkeypatch): + """The same structure yields the same seed wherever it is scheduled.""" + monkeypatch.chdir(tmp_path) + first = _seeded_model(tmp_path, "rigidbody_1.pdb", "ATOM one\n") + renamed = _seeded_model(tmp_path, "rank_37.pdb", "ATOM one\n") + + assert libcns.derive_seed(917, first) == libcns.derive_seed(917, renamed) + + +def test_derive_seed_follows_the_content_it_reads(tmp_path, monkeypatch): + """A changed structure is a different computation and gets a new seed.""" + monkeypatch.chdir(tmp_path) + original = _seeded_model(tmp_path, "model.pdb", "ATOM one\n") + changed = _seeded_model(tmp_path, "other.pdb", "ATOM two\n") + + assert libcns.derive_seed(917, original) != libcns.derive_seed(917, changed) + + +def test_derive_seed_separates_repeats_of_one_job(tmp_path, monkeypatch): + """Repeats of one job are additional sampling, not duplicates of it.""" + monkeypatch.chdir(tmp_path) + model = _seeded_model(tmp_path, "model.pdb", "ATOM one\n") + + seeds = [libcns.derive_seed(917, model, repeat) for repeat in range(8)] + + assert len(set(seeds)) == len(seeds) + + +def test_derive_seed_still_follows_iniseed(tmp_path, monkeypatch): + """``iniseed`` keeps its meaning: changing it changes every seed.""" + monkeypatch.chdir(tmp_path) + model = _seeded_model(tmp_path, "model.pdb", "ATOM one\n") + + assert libcns.derive_seed(917, model) != libcns.derive_seed(4242, model) + + +def test_derive_seed_binds_the_inputs_in_order(tmp_path, monkeypatch): + """Two molecules in the other order is a different docking job.""" + monkeypatch.chdir(tmp_path) + receptor = _seeded_model(tmp_path, "receptor.pdb", "ATOM receptor\n") + ligand = _seeded_model(tmp_path, "ligand.pdb", "ATOM ligand\n") + + assert libcns.derive_seed(917, [receptor, ligand]) != libcns.derive_seed( + 917, [ligand, receptor] + ) + + +def test_derive_seed_stays_in_the_range_cns_represents_exactly(tmp_path, monkeypatch): + """CNS holds numbers as doubles; a derived seed must be an exact integer.""" + monkeypatch.chdir(tmp_path) + + for index in range(200): + model = _seeded_model(tmp_path, f"model_{index}.pdb", f"ATOM {index}\n") + seed = libcns.derive_seed(9999999999999999, model, index) + assert 1 <= seed < libcns.SEED_CEILING + assert float(seed) == seed + + +def test_derive_seed_reads_a_compressed_model(tmp_path, monkeypatch): + """A cleaned step holds ``model.pdb.gz``; that is the same content.""" + import gzip + + monkeypatch.chdir(tmp_path) + plain = _seeded_model(tmp_path, "model.pdb", "ATOM one\n") + expected = libcns.derive_seed(917, plain) + + Path(tmp_path, "model.pdb.gz").write_bytes( + gzip.compress(b"ATOM one\n", mtime=0) + ) + Path(tmp_path, "model.pdb").unlink() + + assert libcns.derive_seed(917, plain) == expected + + +def test_refinement_schedule_emits_replicas_in_rounds(): + """Every model gets its first replica before any model gets its second.""" + schedule = libcns.refinement_schedule(3, 2) + + assert schedule == [(0, 0), (1, 0), (2, 0), (0, 1), (1, 1), (2, 1)] + + +def test_refinement_schedule_is_prefix_stable_in_sampling_factor(): + """Raising ``sampling_factor`` appends jobs and renumbers none of them.""" + for factor in range(1, 6): + schedule = libcns.refinement_schedule(4, factor) + assert len(schedule) == 4 * factor + for smaller in range(1, factor + 1): + assert schedule[: 4 * smaller] == libcns.refinement_schedule(4, smaller) + + +def test_no_prepare_cns_input_module_reads_the_ligand_topology_variable(): + """`prepare_cns_input` omits `$ligand_top_fname`; this pins why that is safe. + + The per-model assignment is a step-folder path. Being unread it is not a + declared dependency, so nothing rewrites it and it would leak a locator + into the job's canonical form. Only the two topology recipes read it, and + neither reaches CNS through `prepare_cns_input`. If that ever changes, + this fails rather than the omission silently dropping a real input. + """ + modules_dir = Path(haddock.modules.__file__).parent + offenders = sorted( + module_dir.name + for module_dir in modules_dir.glob("*/*") + if (module_dir / "cns").is_dir() + and "prepare_cns_input" + in (module_dir / "__init__.py").read_text(encoding="utf-8") + and any( + "$ligand_top_fname" in recipe.read_text(encoding="utf-8", errors="ignore") + for recipe in (module_dir / "cns").rglob("*.cns") + ) + ) + assert offenders == [] diff --git a/tests/test_libcnscanonical.py b/tests/test_libcnscanonical.py new file mode 100644 index 0000000000..43e3d7bbbb --- /dev/null +++ b/tests/test_libcnscanonical.py @@ -0,0 +1,589 @@ +"""Tests for CNS canonical input representations.""" + +import gzip +import re +from pathlib import Path + +import pytest + +from haddock.libs.libcnscanonical import ( + build_canonical_mapping, + compression_transparent_checksum, +) +from haddock.libs.libsubprocess import CNSJob + + +def test_canonical_mapping_is_independent_of_run_step_and_install_names(tmp_path): + first, _ = _mapping(tmp_path, "one", "2_rigidbody", "install-a") + second, _ = _mapping(tmp_path, "two", "02_rigidbody", "install-b") + + assert first.canonical_script == second.canonical_script + assert first.checksums == second.checksums + assert first.invariant_dependencies == ( + "canonical-cns", + "module/protocol.cns", + "toppar/protein.top", + ) + + +def test_canonical_mapping_is_independent_of_the_cns_executable(tmp_path): + """The executable evaluates the computation; it is not part of it. + + Two installations hold different binaries, always -- nobody compiles or + downloads the same bytes twice -- so an identity that read the + executable's content could never be shared between them. The pin is + still there and still occupies its position; it is bound to a policy + constant rather than to bytes. + """ + first, _ = _mapping(tmp_path, "one", "2_rigidbody", "install-a") + second, _ = _mapping(tmp_path, "two", "2_rigidbody", "install-b") + + assert compression_transparent_checksum( + first.cns_exec + ) != compression_transparent_checksum(second.cns_exec) + assert first.checksums["canonical-cns"] == second.checksums["canonical-cns"] + assert "canonical-cns" in first.invariant_dependencies + + +def test_canonical_mapping_is_independent_of_input_and_output_filenames(tmp_path): + first = _named_model_mapping(tmp_path, "run-a", "rank_1.pdb", "flexref_1.pdb") + second = _named_model_mapping(tmp_path, "run-b", "rank_9.pdb", "flexref_9.pdb") + + assert first.canonical_script == second.canonical_script + assert first.checksums == second.checksums + + +def test_canonical_mapping_keeps_model_content_in_identity(tmp_path): + first = _named_model_mapping(tmp_path, "run-a", "rank_1.pdb", "flexref_1.pdb") + second = _named_model_mapping( + tmp_path, + "run-b", + "rank_9.pdb", + "flexref_9.pdb", + model_content="ATOM changed\n", + ) + + assert first.canonical_script == second.canonical_script + assert first.checksums["canonical-input-1.pdb"] != ( + second.checksums["canonical-input-1.pdb"] + ) + + +def test_canonical_mapping_erases_count_but_keeps_seed(tmp_path): + first = _indexed_model_mapping(tmp_path, "run-a", index=1, seed=1001) + second = _indexed_model_mapping(tmp_path, "run-b", index=9, seed=1001) + different_seed = _indexed_model_mapping(tmp_path, "run-c", index=1, seed=1002) + + assert first.canonical_script == second.canonical_script + assert "canonical-count" in first.canonical_script + assert "1001" in first.canonical_script + assert first.canonical_script != different_seed.canonical_script + + +def test_canonical_mapping_resolves_count_suffix_restraint_file(tmp_path): + mapping = _counted_restraint_mapping(tmp_path, count=3, suffixed_exists=True) + + assert mapping.dependency_paths[ + (tmp_path / "run" / "03_flexref" / "ambig.tbl_3").resolve() + ] == "canonical-ambig.tbl" + + +def test_canonical_mapping_erases_count_suffix_base_alias(tmp_path): + first = _renamed_counted_restraint_mapping( + tmp_path, + "run-a", + base_name="z_ambig.tbl", + count=3, + ) + second = _renamed_counted_restraint_mapping( + tmp_path, + "run-b", + base_name="a_ambig.tbl", + count=9, + ) + + assert first.canonical_script == second.canonical_script + assert first.checksums == second.checksums + assert "z_ambig.tbl" not in first.canonical_script + assert "a_ambig.tbl" not in second.canonical_script + + +def test_canonical_mapping_falls_back_to_base_restraint_file(tmp_path): + mapping = _counted_restraint_mapping(tmp_path, count=3, suffixed_exists=False) + + assert mapping.dependency_paths[ + (tmp_path / "run" / "03_flexref" / "ambig.tbl").resolve() + ] == "canonical-ambig.tbl" + + +def test_canonical_mapping_replaces_relative_sibling_dependency_path(tmp_path): + work_dir = tmp_path / "run" / "01_rigidbody" + input_pdb = tmp_path / "run" / "data" / "00_topoaa" / "structure_1.pdb" + module, toppar, cns = _install(tmp_path, "install") + work_dir.mkdir(parents=True) + input_pdb.parent.mkdir(parents=True) + input_pdb.write_text("ATOM\n", encoding="utf-8") + + mapping = build_canonical_mapping( + 'evaluate ($input_pdb = "../data/00_topoaa/structure_1.pdb")\n' + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[work_dir / "result.pdb"], + work_dir=work_dir, + ) + + assert "canonical-input-1.pdb" in mapping.canonical_script + assert "00_topoaa" not in mapping.canonical_script + + +def test_canonical_mapping_accepts_dependency_named_after_its_role(tmp_path): + work_dir = tmp_path / "run" / "01_rigidbody" + module, toppar, cns = _install(tmp_path, "install") + restraints = tmp_path / "run" / "data" / "ambig.tbl" + work_dir.mkdir(parents=True) + restraints.parent.mkdir(parents=True, exist_ok=True) + restraints.write_text("assign\n", encoding="utf-8") + script = work_dir / "job.inp" + script.write_text( + f'evaluate ($ambig_fname = "{restraints}")\n' + "noe @@$ambig_fname end\n" + "inline @@MODULE:protocol.cns\n" + "inline @@TOPPAR:protein.top\n", + encoding="utf-8", + ) + + mapping = build_canonical_mapping( + script, + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[work_dir / "result.pdb"], + work_dir=work_dir, + ) + + assert "canonical-ambig.tbl" in mapping.canonical_script + assert str(restraints) not in mapping.canonical_script + + +def test_canonical_dependency_names_follow_first_reference_order(tmp_path): + first = _two_input_mapping( + tmp_path, + "run-a", + first_name="z_model.pdb", + second_name="a_model.pdb", + ) + second = _two_input_mapping( + tmp_path, + "run-b", + first_name="a_model.pdb", + second_name="z_model.pdb", + ) + + assert first.canonical_script == second.canonical_script + assert first.checksums["canonical-input-1.pdb"] == ( + second.checksums["canonical-input-1.pdb"] + ) + assert first.checksums["canonical-input-2.pdb"] == ( + second.checksums["canonical-input-2.pdb"] + ) + + +def test_canonical_mapping_resolves_module_absolute_suffix_and_toppar_slash(tmp_path): + work_dir = tmp_path / "run" / "03_emref" + module, toppar, cns = _install(tmp_path, "install") + work_dir.mkdir(parents=True) + (module / "protein-ss-restraints-all.cns").write_text( + "{ ss restraints }\n", + encoding="utf-8", + ) + (toppar / "dmso.pdb").write_text("DMSO\n", encoding="utf-8") + + mapping = build_canonical_mapping( + "inline @@MODULE:/protein-ss-restraints-all.cns\n" + "coor @@TOPPAR/dmso.pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[work_dir / "result.pdb"], + work_dir=work_dir, + ) + + assert mapping.dependency_paths[ + (module / "protein-ss-restraints-all.cns").resolve() + ] == "module/protein-ss-restraints-all.cns" + assert mapping.dependency_paths[ + (toppar / "dmso.pdb").resolve() + ] == "toppar/dmso.pdb" + + +def test_canonical_mapping_rejects_unresolved_reads(tmp_path): + work_dir = tmp_path / "run" / "01_rigidbody" + module, toppar, cns = _install(tmp_path, "install") + work_dir.mkdir(parents=True) + + with pytest.raises(ValueError, match="unresolved reads"): + build_canonical_mapping( + "coor @@$missing\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[work_dir / "result.pdb"], + work_dir=work_dir, + ) + + +def test_cnsjob_exposes_canonical_mapping(monkeypatch, tmp_path): + mapping, step = _mapping(tmp_path, "run", "1_rigidbody", "install") + monkeypatch.chdir(step) + job = CNSJob( + step / "job.inp", + envvars={ + "MODULE": str(tmp_path / "install" / "module"), + "TOPPAR": str(tmp_path / "install" / "toppar"), + }, + cns_exec=mapping.cns_exec, + output_files=[step / "result.pdb"], + ) + + assert job.canonical_mapping().checksums == mapping.checksums + + +def test_compression_transparent_checksum(tmp_path): + plain = tmp_path / "input.pdb" + compressed = tmp_path / "input.pdb.gz" + plain.write_bytes(b"ATOM\n") + with gzip.open(compressed, "wb") as handle: + handle.write(plain.read_bytes()) + assert compression_transparent_checksum(plain) == ( + compression_transparent_checksum(compressed) + ) + + +def test_canonical_mapping_keeps_install_reference_spelling(tmp_path): + """``MODULE:``/``TOPPAR:`` references resolve through the environment already.""" + mapping, _ = _mapping(tmp_path, "run", "1_rigidbody", "install") + + assert "@@MODULE:protocol.cns" in mapping.canonical_script + assert "@@TOPPAR:protein.top" in mapping.canonical_script + assert "MODULE:module/" not in mapping.canonical_script + assert "TOPPAR:toppar/" not in mapping.canonical_script + # the pin names are unchanged; only the script text keeps its own spelling + assert "module/protocol.cns" in mapping.checksums + assert "toppar/protein.top" in mapping.checksums + + +def test_canonical_mapping_rewrites_unread_absolute_install_paths(tmp_path): + """An install path reaches the key even when the recipe never reads it.""" + root = tmp_path / "run" / "1_rigidbody" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + (toppar / "boxtyp20.pdb").write_text("BOX\n", encoding="utf-8") + + mapping = build_canonical_mapping( + f'evaluate ($boxtyp20 = "{toppar / "boxtyp20.pdb"}")\n' + "inline @@MODULE:protocol.cns\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + assert 'evaluate ($boxtyp20 = "TOPPAR:boxtyp20.pdb")' in mapping.canonical_script + assert str(toppar) not in mapping.canonical_script + + +def test_canonical_mapping_preserves_cns_variable_names(tmp_path): + """Rewriting a dependency must not rename the symbol that refers to it.""" + root = tmp_path / "run" / "0_topoaa" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + (root / "molA.pdb").write_text("ATOM\n", encoding="utf-8") + + mapping = build_canonical_mapping( + 'eval ($file = "molA.pdb")\n' + "evaluate ($coor_infile = $file)\n" + "coor @@$coor_infile\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + assert 'eval ($file = "canonical-input-1.pdb")' in mapping.canonical_script + assert "evaluate ($coor_infile = $file)" in mapping.canonical_script + assert "eval (canonical-input-1.pdb" not in mapping.canonical_script + + +def test_canonical_mapping_output_survives_input_basename_collision(tmp_path): + """An input and an output may share a basename in different directories.""" + root = tmp_path / "run" / "1_topocg" + upstream = tmp_path / "run" / "0_topoaa" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + upstream.mkdir(parents=True) + (upstream / "shape_haddock.pdb").write_text("ATOM\n", encoding="utf-8") + + mapping = build_canonical_mapping( + 'evaluate ($input_pdb = "../0_topoaa/shape_haddock.pdb")\n' + 'evaluate ($output_pdb_filename = "shape_haddock.pdb")\n' + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "shape_haddock.pdb"], + work_dir=root, + ) + + assert ( + 'evaluate ($output_pdb_filename = "canonical-output.pdb")' + in mapping.canonical_script + ) + assert "canonical-input-1.pdb" in mapping.canonical_script + + +def test_canonical_mapping_rejects_output_bound_to_an_input_pin(tmp_path): + """The guard refuses a script that writes to something other than its output.""" + root = tmp_path / "run" / "1_rigidbody" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + (root / "model.pdb").write_text("ATOM\n", encoding="utf-8") + + with pytest.raises(ValueError, match="output_pdb_filename"): + build_canonical_mapping( + 'evaluate ($input_pdb = "model.pdb")\n' + 'evaluate ($output_pdb_filename = "elsewhere.pdb")\n' + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + +def test_canonical_mapping_normalizes_logging_only_parameter(tmp_path): + root = tmp_path / "run" / "1_rigidbody" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + (root / "input.pdb").write_text("ATOM\n", encoding="utf-8") + + mapping = build_canonical_mapping( + 'eval ($log_level="verbose")\n' + 'eval ($input_pdb="input.pdb")\n' + 'eval ($output_pdb_filename="result.pdb")\n' + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + assert 'eval ($log_level="canonical-log-level")' in mapping.canonical_script + + +def test_canonical_mapping_resolves_cgtoaa_indexed_variable_references(tmp_path): + root = tmp_path / "run" / "2_cgtoaa" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + for index in (1, 2): + (root / f"input_{index}.psf").write_text("PSF\n", encoding="utf-8") + + mapping = build_canonical_mapping( + "eval ($input_aa_psf_filename_1=\"input_1.psf\")\n" + "eval ($input_aa_psf_filename_2=\"input_2.psf\")\n" + "while ($nchain < 2) loop nloop1\n" + " structure @@$input_aa_psf_filename_$nchain end\n" + "end loop nloop1\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + assert [dependency.original_path.name for dependency in mapping.dependencies] == [ + "input_1.psf", + "input_2.psf", + ] + + +def test_logging_and_count_canonicalizations_are_limited_to_known_cns_uses(): + module_root = Path(__file__).parents[1] / "src" / "haddock" / "modules" + recipes = module_root.glob("*/*/cns/*.cns") + log_level_lines = [] + count_lines = [] + for recipe in recipes: + for line in recipe.read_text(encoding="utf-8").splitlines(): + if "$log_level" in line: + log_level_lines.append(line.strip()) + if re.search(r"\$count(?![A-Za-z0-9_])", line): + count_lines.append(line.strip()) + + assert log_level_lines + assert all( + line.startswith(("if ( $log_level", "elseif ( $log_level")) + for line in log_level_lines + ) + assert count_lines + assert all( + "display STRUCTURE NUMBER $count" in line + or "$ambig_fname + \"_\" + encode($count)" in line + for line in count_lines + ) + + +def _mapping(tmp_path: Path, run_name: str, step_name: str, install_name: str): + root = tmp_path / run_name / step_name + module, toppar, cns = _install(tmp_path, install_name) + root.mkdir(parents=True) + (root / "renamed-model.pdb").write_text("ATOM\n", encoding="utf-8") + script = root / "job.inp" + script.write_text( + 'evaluate ($input_pdb = "renamed-model.pdb")\n' + "coor @@$input_pdb\n" + "inline @@MODULE:protocol.cns\n" + "inline @@TOPPAR:protein.top\n", + encoding="utf-8", + ) + return ( + build_canonical_mapping( + script, + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ), + root, + ) + + +def _named_model_mapping( + tmp_path: Path, + run_name: str, + input_name: str, + output_name: str, + model_content: str = "ATOM\n", +): + root = tmp_path / run_name / "03_flexref" + module, toppar, cns = _install(tmp_path, f"{run_name}-install") + root.mkdir(parents=True) + (root / input_name).write_text(model_content, encoding="utf-8") + return build_canonical_mapping( + f'evaluate ($input_pdb = "{input_name}")\n' + f'evaluate ($output_pdb_filename = "{output_name}")\n' + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / output_name], + work_dir=root, + ) + + +def _indexed_model_mapping( + tmp_path: Path, + run_name: str, + index: int, + seed: int, +): + root = tmp_path / run_name / "03_flexref" + module, toppar, cns = _install(tmp_path, f"{run_name}-install") + root.mkdir(parents=True) + input_name = f"rank_{index}.pdb" + output_name = f"flexref_{index}.pdb" + (root / input_name).write_text("ATOM\n", encoding="utf-8") + return build_canonical_mapping( + f'evaluate ($input_pdb = "{input_name}")\n' + f'evaluate ($output_pdb_filename = "{output_name}")\n' + f"evaluate ($count = {index})\n" + f"evaluate ($seed = {seed})\n" + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / output_name], + work_dir=root, + ) + + +def _counted_restraint_mapping( + tmp_path: Path, + count: int, + suffixed_exists: bool, +): + root = tmp_path / "run" / "03_flexref" + module, toppar, cns = _install(tmp_path, "install") + root.mkdir(parents=True) + (root / "model.pdb").write_text("ATOM\n", encoding="utf-8") + (root / "ambig.tbl").write_text("assign base\n", encoding="utf-8") + if suffixed_exists: + (root / f"ambig.tbl_{count}").write_text("assign counted\n", encoding="utf-8") + return build_canonical_mapping( + 'evaluate ($input_pdb = "model.pdb")\n' + 'evaluate ($ambig_fname = "ambig.tbl")\n' + f"evaluate ($count = {count})\n" + 'evaluate ($filenam0 = $ambig_fname + "_" + encode($count))\n' + "noe class ambi @@$filenam0 end\n" + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + +def _renamed_counted_restraint_mapping( + tmp_path: Path, + run_name: str, + base_name: str, + count: int, +): + root = tmp_path / run_name / "03_flexref" + module, toppar, cns = _install(tmp_path, f"{run_name}-install") + root.mkdir(parents=True) + (root / "model.pdb").write_text("ATOM\n", encoding="utf-8") + (root / f"{base_name}_{count}").write_text("assign same\n", encoding="utf-8") + return build_canonical_mapping( + 'evaluate ($input_pdb = "model.pdb")\n' + f'evaluate ($ambig_fname = "{base_name}")\n' + f"evaluate ($count = {count})\n" + 'evaluate ($filenam0 = $ambig_fname + "_" + encode($count))\n' + "noe class ambi @@$filenam0 end\n" + "coor @@$input_pdb\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + +def _two_input_mapping( + tmp_path: Path, + run_name: str, + first_name: str, + second_name: str, +): + root = tmp_path / run_name / "03_flexref" + module, toppar, cns = _install(tmp_path, f"{run_name}-install") + root.mkdir(parents=True) + (root / first_name).write_text("ATOM first\n", encoding="utf-8") + (root / second_name).write_text("ATOM second\n", encoding="utf-8") + return build_canonical_mapping( + f'evaluate ($input_pdb_1 = "{first_name}")\n' + f'evaluate ($input_pdb_2 = "{second_name}")\n' + "coor @@$input_pdb_1\n" + "coor @@$input_pdb_2\n", + envvars={"MODULE": str(module), "TOPPAR": str(toppar)}, + cns_exec=cns, + output_files=[root / "result.pdb"], + work_dir=root, + ) + + +def _install(tmp_path: Path, install_name: str) -> tuple[Path, Path, Path]: + module = tmp_path / install_name / "module" + toppar = tmp_path / install_name / "toppar" + module.mkdir(parents=True, exist_ok=True) + toppar.mkdir(parents=True, exist_ok=True) + (module / "protocol.cns").write_text("{ module }\n", encoding="utf-8") + (toppar / "protein.top").write_text("{ toppar }\n", encoding="utf-8") + cns = tmp_path / install_name / "cns" + # Two installations never hold the same executable bytes. Writing the + # install name into it keeps that true here, so that every test comparing + # two installations is comparing what a real pair of them would look like. + cns.write_text(f"#!/bin/sh\n# {install_name}\n", encoding="utf-8") + cns.chmod(0o755) + return module, toppar, cns diff --git a/tests/test_libcnsoutput.py b/tests/test_libcnsoutput.py new file mode 100644 index 0000000000..a4380750c5 --- /dev/null +++ b/tests/test_libcnsoutput.py @@ -0,0 +1,303 @@ +import gzip +from pathlib import Path +from unittest.mock import MagicMock + +import pytest + +from haddock.core.exceptions import CNSRunningError +from haddock.libs.libcnsoutput import ( + is_normalized_cns_pdb, + normalize_cns_pdb, +) +from haddock.libs.libsubprocess import CNSJob + + +def test_normalize_cns_pdb_removes_run_volatile_remarks(tmp_path): + pdb = tmp_path / "model.pdb" + pdb.write_text( + "\n".join( + [ + "REMARK FILENAME= /tmp/run_1/model.pdb", + "REMARK initial structure 1 - ../run_1/input.pdb", + "REMARK DATE: 27-Jun-2026 12:34:56", + "REMARK HADDOCK stats for rigidbody_1.pdb", + "REMARK score: 1.23", + "ATOM 1 CA ALA A 1 0.000 0.000 0.000", + "END", + ] + ) + + "\n", + encoding="utf-8", + ) + + changed = normalize_cns_pdb(pdb) + + assert changed is True + assert pdb.read_text(encoding="utf-8") == ( + "REMARK score: 1.23\n" + "ATOM 1 CA ALA A 1 0.000 0.000 0.000\n" + "END\n" + ) + + +def test_normalize_cns_pdb_leaves_stable_file_unchanged(tmp_path): + pdb = tmp_path / "model.pdb" + content = "REMARK score: 1.23\nEND\n" + pdb.write_text(content, encoding="utf-8") + + changed = normalize_cns_pdb(pdb) + + assert changed is False + assert pdb.read_text(encoding="utf-8") == content + + +def test_normalize_cns_pdb_handles_gzip_artifacts(tmp_path): + pdb = tmp_path / "model.pdb.gz" + pdb.write_bytes(gzip.compress(b"REMARK DATE: volatile\nATOM\n", mtime=0)) + + assert normalize_cns_pdb(pdb) is True + assert gzip.decompress(pdb.read_bytes()) == b"ATOM\n" + + +def test_normalize_cns_pdb_preserves_non_utf8_stable_bytes(tmp_path): + pdb = tmp_path / "model.pdb" + pdb.write_bytes( + b"REMARK DATE: volatile\n" + b"ATOM 1 X BIN A 1 0.000 0.000 0.000\xff\n" + ) + + assert normalize_cns_pdb(pdb) is True + assert pdb.read_bytes() == ( + b"ATOM 1 X BIN A 1 0.000 0.000 0.000\xff\n" + ) + + +def test_normalize_cns_pdb_does_not_mutate_hardlink_source(tmp_path): + source = tmp_path / "source.pdb" + destination = tmp_path / "destination.pdb" + source.write_text("REMARK DATE: volatile\nATOM\n", encoding="utf-8") + destination.hardlink_to(source) + + assert normalize_cns_pdb(destination) is True + assert source.read_text(encoding="utf-8") == "REMARK DATE: volatile\nATOM\n" + assert destination.read_text(encoding="utf-8") == "ATOM\n" + + +def test_is_normalized_cns_pdb(tmp_path): + pdb = tmp_path / "model.pdb" + pdb.write_text("REMARK DATE: volatile\nATOM\n", encoding="utf-8") + + assert is_normalized_cns_pdb(pdb) is False + normalize_cns_pdb(pdb) + assert is_normalized_cns_pdb(pdb) is True + + +def test_cnsjob_run_normalizes_output_pdb(monkeypatch, tmp_path): + output_pdb = tmp_path / "model.pdb" + output_psf = tmp_path / "model.psf" + output_pdb.write_text( + "REMARK DATE: volatile\n" + "REMARK initial structure 1 - ../run_1/input.pdb\n" + "REMARK score: 1.23\n", + encoding="utf-8", + ) + output_psf.write_text(PSF_WITH_DATE, encoding="utf-8") + job = CNSJob( + input_file="cns input stream", + cns_exec=_executable(tmp_path), + output_files=[output_pdb, output_psf], + ) + _mock_popen(monkeypatch) + + job.run() + + assert output_pdb.read_text(encoding="utf-8") == "REMARK score: 1.23\n" + assert "created by user" not in output_psf.read_text(encoding="utf-8") + + +def test_cnsjob_normalizes_relative_outputs_from_creation_dir(tmp_path, monkeypatch): + work_dir = tmp_path / "run" / "1_rigidbody" + other_dir = tmp_path / "other" + work_dir.mkdir(parents=True) + other_dir.mkdir() + output_pdb = work_dir / "model.pdb" + output_pdb.write_text("REMARK DATE: volatile\nATOM\n", encoding="utf-8") + + monkeypatch.chdir(work_dir) + job = CNSJob( + input_file="cns input stream", + cns_exec=_executable(tmp_path), + output_files=["model.pdb"], + ) + monkeypatch.chdir(other_dir) + + job.normalize_outputs() + + assert output_pdb.read_text(encoding="utf-8") == "ATOM\n" + + +def test_cnsjob_rejects_mismatched_declared_output(tmp_path): + with pytest.raises(ValueError, match="does not match"): + CNSJob( + input_file='eval ($output_pdb_filename="actual.pdb")\n', + cns_exec=_executable(tmp_path), + output_files=[tmp_path / "declared.pdb"], + ) + + +def test_cnsjob_publishes_complete_normalized_staged_outputs(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + public = tmp_path / "model.pdb" + job = CNSJob( + input_file='eval ($output_pdb_filename="model.pdb")\n', + cns_exec=_executable(tmp_path), + output_files=[public], + ) + script, partial_outputs = job._partial_output_script() + partial = partial_outputs[public] + partial.write_text("REMARK DATE: volatile\nATOM\n", encoding="utf-8") + + assert "model.partial.pdb" in script + job._publish_partial_outputs(partial_outputs) + + assert public.read_text(encoding="utf-8") == "ATOM\n" + assert not partial.exists() + + +def test_cnsjob_retains_the_executed_input(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + input_file = tmp_path / "job.inp" + input_file.write_text('eval ($output_pdb_filename="model.pdb")\n') + job = CNSJob( + input_file=input_file, + cns_exec=_executable(tmp_path), + output_files=[tmp_path / "model.pdb"], + ) + + script = job.prepare_execution_input() + + assert input_file.read_text() == script + assert "model.partial.pdb" in script + + +def test_cnsjob_discards_incomplete_staged_outputs(tmp_path, monkeypatch): + monkeypatch.chdir(tmp_path) + job = CNSJob( + input_file='eval ($output_pdb_filename="model.pdb")\n', + cns_exec=_executable(tmp_path), + output_files=[tmp_path / "model.pdb"], + ) + job.prepare_execution_input() + partial = next(iter(job._pending_partial_outputs.values())) + partial.write_text("incomplete\n") + + job._discard_partial_outputs() + + assert not partial.exists() + + +def test_cnsjob_ignores_only_the_benign_denormal_stderr(tmp_path, monkeypatch): + job = CNSJob(input_file="stop\n", cns_exec=_executable(tmp_path)) + popen = _mock_popen(monkeypatch) + popen.return_value.communicate.return_value = ( + b"normal output", + b"Note: The following floating-point exceptions are signalling: " + b"IEEE_DENORMAL\n", + ) + popen.return_value.returncode = 0 + + assert job.run() == b"normal output" + + +@pytest.mark.parametrize( + ("stdout", "stderr", "returncode"), + [ + (b"^^^^^\n", b"", 0), + (b"normal output", b"real stderr\n", 0), + (b"normal output", b"", 2), + ], +) +def test_cnsjob_preserves_hard_failure_detection( + tmp_path, monkeypatch, stdout, stderr, returncode +): + job = CNSJob(input_file="stop\n", cns_exec=_executable(tmp_path)) + popen = _mock_popen(monkeypatch) + popen.return_value.communicate.return_value = (stdout, stderr) + popen.return_value.returncode = returncode + + with pytest.raises(CNSRunningError): + job.run() + + +def _executable(tmp_path: Path) -> Path: + path = tmp_path / "cns" + path.write_text("#!/bin/sh\n", encoding="utf-8") + path.chmod(0o755) + return path + + +def _mock_popen(monkeypatch): + popen = MagicMock() + popen.return_value.communicate.return_value = (b"output", None) + monkeypatch.setattr("subprocess.Popen", popen) + return popen + + +PSF_WITH_DATE = """data_cns_mtf + +_cns_mtf.title +; FILENAME="model_haddock.psf" + disulphide added: from A 6 to A 127 + DATE:31-Aug-2026 01:17:08 created by user: unknown + VERSION:1.3U +; + +_cns_mtf.id 1 +""" + + +def test_psf_normalization_removes_only_the_date_stamp(tmp_path): + from haddock.libs.libcnsoutput import ( + is_normalized_cns_psf, + normalize_cns_psf, + ) + + path = tmp_path / "model_haddock.psf" + path.write_text(PSF_WITH_DATE, encoding="utf-8") + assert not is_normalized_cns_psf(path) + + assert normalize_cns_psf(path) is True + text = path.read_text(encoding="utf-8") + + assert "created by user" not in text + assert "model_haddock.psf" not in text + assert text.splitlines() == [ + "data_cns_mtf", + "", + "_cns_mtf.title", + "; HADDOCK3 normalized topology", + " disulphide added: from A 6 to A 127", + " VERSION:1.3U", + ";", + "", + "_cns_mtf.id 1", + ] + assert is_normalized_cns_psf(path) + assert normalize_cns_psf(path) is False + + +def test_psf_normalization_makes_two_runs_agree(tmp_path): + from haddock.libs.libcnsoutput import normalize_cns_psf_bytes + + monday = PSF_WITH_DATE.encode("utf-8") + tuesday = PSF_WITH_DATE.replace("01:17:08", "09:42:55").encode("utf-8") + + assert monday != tuesday + assert normalize_cns_psf_bytes(monday) == normalize_cns_psf_bytes(tuesday) + + +def test_psf_normalization_does_not_touch_structural_data(tmp_path): + from haddock.libs.libcnsoutput import normalize_cns_psf_bytes + + body = " DATE: 1 2 3 4\n 1 A 6 CYS SG SG 0.000000\n" + assert normalize_cns_psf_bytes(body.encode()) == body.encode() diff --git a/tests/test_libgrid.py b/tests/test_libgrid.py index b2804cb371..cdfafcd807 100644 --- a/tests/test_libgrid.py +++ b/tests/test_libgrid.py @@ -3,6 +3,7 @@ from haddock.libs.libgrid import ( GridJob, CompositeGridJob, + GRIDScheduler, JobStatus, Tag, ping_dirac, @@ -109,6 +110,41 @@ def test_clean_removes_tmpdir(dummy_paths): assert not loc.exists() +def test_retrieve_output_normalizes_cns_artifacts(tmp_path, dummy_paths, monkeypatch): + toppar, module = dummy_paths + job = GridJob(input="input", toppar_path=toppar, module_path=module) + job.id = 42 + job.wd = tmp_path / "work" + job.wd.mkdir() + output_dir = job.loc / str(job.id) + output_dir.mkdir() + (output_dir / "model.pdb").write_text("REMARK DATE: volatile\nATOM\n") + job.expected_outputs = ["model.pdb"] + monkeypatch.setattr( + "subprocess.run", + lambda *args, **kwargs: type("Result", (), {"returncode": 0})(), + ) + + job.retrieve_output() + + assert (job.wd / "model.pdb").read_text() == "ATOM\n" + + +def test_grid_completion_propagates_worker_exceptions(mocker): + """Failures raised while retrieving a grid result must reach the caller.""" + scheduler = object.__new__(GRIDScheduler) + scheduler.ncores = 1 + scheduler.workload = [mocker.Mock(status=JobStatus.RUNNING)] + mocker.patch.object( + scheduler, + "process_job", + side_effect=RuntimeError("incomplete transfer"), + ) + + with pytest.raises(RuntimeError, match="incomplete transfer"): + scheduler.wait_for_completion() + + def test_ping_and_validate_dirac(monkeypatch): monkeypatch.setattr("shutil.which", lambda cmd: "/usr/bin/" + cmd) monkeypatch.setattr( diff --git a/tests/test_libhpc.py b/tests/test_libhpc.py index 5bb1e1827d..ed2a24b84a 100644 --- a/tests/test_libhpc.py +++ b/tests/test_libhpc.py @@ -1,4 +1,5 @@ """Test libhpc.""" + import os import pytest import pytest_mock # noqa : F401 @@ -7,22 +8,24 @@ from subprocess import CompletedProcess from haddock.libs.libhpc import ( + HPCScheduler, HPCWorker, extract_slurm_status, JOB_STATUS_DIC, to_torque_time, - ) +) from haddock.libs.libsubprocess import CNSJob +from haddock.core.exceptions import CNSRunningError def test_to_torque_time(): """Test minutes to HH:MM:SS cast.""" - assert to_torque_time(10) == '00:10:00' - assert to_torque_time(60) == '01:00:00' - assert to_torque_time(70) == '01:10:00' - assert to_torque_time(1510) == '25:10:00' - assert to_torque_time(6070) == '101:10:00' + assert to_torque_time(10) == "00:10:00" + assert to_torque_time(60) == "01:00:00" + assert to_torque_time(70) == "01:10:00" + assert to_torque_time(1510) == "25:10:00" + assert to_torque_time(6070) == "101:10:00" @pytest.fixture @@ -61,18 +64,18 @@ def slurm_scontrol_terminated_jobid(): def test_slurm_status(slurm_scontrol_terminated_jobid): status = extract_slurm_status(slurm_scontrol_terminated_jobid) - assert status == 'RUNNING' - assert JOB_STATUS_DIC[status] == 'running' + assert status == "RUNNING" + assert JOB_STATUS_DIC[status] == "running" @pytest.fixture def slurm_scontrol_wrongjobid(): - return 'slurm_load_jobs error: Invalid job id specified' + return "slurm_load_jobs error: Invalid job id specified" def test_slurm_nojobid(slurm_scontrol_wrongjobid): status = extract_slurm_status(slurm_scontrol_wrongjobid) - assert status == 'FAILED' + assert status == "FAILED" @pytest.fixture @@ -81,25 +84,25 @@ def hpcworker(mocker): mocker.patch( "haddock.libs.libsubprocess.CNSJob.cns_exec", return_value=None, - ) + ) return HPCWorker( tasks=[ CNSJob( - Path('rigidbody.inp'), - Path('rigidbody.out'), + Path("rigidbody.inp"), + Path("rigidbody.out"), envvars={ - 'MODDIR': '.', - 'TOPPAR': 'topology_params', - 'MODULE': 'rigidbody', - }, + "MODDIR": ".", + "TOPPAR": "topology_params", + "MODULE": "rigidbody", + }, cns_exec=None, - ), - ], + ), + ], num=1, job_id=123456789, - workfload_manager='slurm', + workfload_manager="slurm", queue=10, - ) + ) def test_hpcworker_run(hpcworker, mocker): @@ -107,34 +110,145 @@ def test_hpcworker_run(hpcworker, mocker): mocker.patch( "subprocess.run", return_value=CompletedProcess( - args=['sbatch', str(hpcworker.job_fname)], + args=["sbatch", str(hpcworker.job_fname)], returncode=0, - stdout=b'Submitted batch job 42914957', - stderr=b'', - ) - ) + stdout=b"Submitted batch job 42914957", + stderr=b"", + ), + ) hpcworker.run() assert os.path.exists(hpcworker.job_fname) os.remove(hpcworker.job_fname) assert hpcworker.job_id == 42914957 - assert hpcworker.job_status == 'submitted' + assert hpcworker.job_status == "submitted" def test_hpcworker_update_status( hpcworker, slurm_scontrol_terminated_jobid, mocker, - ): +): """Test `update_status` function.""" mocker.patch( "subprocess.run", return_value=CompletedProcess( - args=['scontrol', 'show', 'jobid', '-dd', '42909924'], + args=["scontrol", "show", "jobid", "-dd", "42909924"], returncode=0, - stdout=bytes(slurm_scontrol_terminated_jobid, 'utf-8'), - stderr=b'', - ) - ) + stdout=bytes(slurm_scontrol_terminated_jobid, "utf-8"), + stderr=b"", + ), + ) status = hpcworker.update_status() assert status == hpcworker.job_status - assert status == 'running' + assert status == "running" + + +def test_hpcworker_normalizes_task_outputs(tmp_path, monkeypatch): + """Test that HPC workers normalize CNS output artifacts after execution.""" + work_dir = tmp_path / "1_rigidbody" + work_dir.mkdir() + (work_dir / "rigidbody_1.pdb").write_text("REMARK DATE: volatile\nATOM\n") + cns_exec = tmp_path / "cns" + cns_exec.write_text("#!/bin/sh\n") + cns_exec.chmod(0o755) + + monkeypatch.chdir(work_dir) + job = CNSJob( + Path("rigidbody_1.inp"), + Path("rigidbody_1.out"), + envvars={ + "MODDIR": str(work_dir), + "TOPPAR": "topology_params", + "MODULE": "rigidbody", + }, + cns_exec=cns_exec, + output_files=[Path("rigidbody_1.pdb")], + ) + worker = HPCWorker( + tasks=[job], + num=1, + job_id=123456789, + workfload_manager="slurm", + ) + monkeypatch.chdir(tmp_path) + + worker.normalize_outputs() + + assert (work_dir / "rigidbody_1.pdb").read_text() == "ATOM\n" + + +def test_hpcworker_publishes_tasks_independently(hpcworker, mocker, tmp_path): + """One faulty concatenated task must not discard another task's output.""" + failed_task = hpcworker.tasks[0] + successful_task = mocker.MagicMock() + failed_publish = mocker.patch.object( + failed_task, + "publish_outputs", + side_effect=CNSRunningError("missing output"), + ) + hpcworker.tasks.append(successful_task) + partial_input = tmp_path / "1_1.partial.inp" + partial_input.write_text("input\n") + hpcworker.partial_inputs = [partial_input] + + hpcworker.normalize_outputs() + + failed_publish.assert_called_once_with(check_output_log=True) + successful_task.publish_outputs.assert_called_once_with(check_output_log=True) + assert not partial_input.exists() + + +def test_hpcworker_job_file_continues_after_each_concatenated_task( + tmp_path, + monkeypatch, +): + """A hard failure in one shell command must not skip later CNS tasks.""" + work_dir = tmp_path / "1_rigidbody" + work_dir.mkdir() + executable = tmp_path / "cns" + executable.write_text("#!/bin/sh\n") + executable.chmod(0o755) + monkeypatch.chdir(work_dir) + jobs = [] + for index in (1, 2): + input_file = Path(f"rigidbody_{index}.inp") + input_file.write_text(f'eval ($output_pdb_filename="rigidbody_{index}.pdb")\n') + jobs.append( + CNSJob( + input_file, + Path(f"rigidbody_{index}.out"), + envvars={ + "MODDIR": str(work_dir), + "TOPPAR": "topology_params", + "MODULE": "rigidbody", + }, + cns_exec=executable, + output_files=[Path(f"rigidbody_{index}.pdb")], + ) + ) + worker = HPCWorker(tasks=jobs, num=1, workfload_manager="slurm") + + worker.prepare_job_file() + + job_file = worker.job_fname.read_text() + assert "|| exit" not in job_file + assert f"{executable} < 1_1.partial.inp > rigidbody_1.out" in job_file + assert f"{executable} < 1_2.partial.inp > rigidbody_2.out" in job_file + + +def test_hpcscheduler_normalizes_failed_workers(hpcworker, mocker): + """Terminal worker status must not suppress its successful task outputs.""" + scheduler = HPCScheduler(hpcworker.tasks) + worker = scheduler.worker_list[0] + mocker.patch.object(worker, "run") + + def mark_failed(): + worker.job_status = "failed" + return worker.job_status + + mocker.patch.object(worker, "update_status", side_effect=mark_failed) + normalize = mocker.patch.object(worker, "normalize_outputs") + + scheduler.run() + + normalize.assert_called_once_with() diff --git a/tests/test_libmath.py b/tests/test_libmath.py deleted file mode 100644 index 77a1df8abb..0000000000 --- a/tests/test_libmath.py +++ /dev/null @@ -1,38 +0,0 @@ -"""Test libmath.""" - -import pytest - -from haddock.libs.libmath import RandomNumberGenerator - - -@pytest.mark.parametrize( - "lower,upper", - [ - (0, 9), - (100, 99999), # the range used for CNS seeds in libcns - (5, 5), # degenerate single-value range - (-3, 3), - ], -) -def test_randint_stays_within_bounds(lower, upper): - """randint must never return a value outside [lower, upper].""" - rng = RandomNumberGenerator() - for _ in range(10000): - value = rng.randint(lower, upper) - assert lower <= value <= upper - - -def test_randint_is_reproducible_for_same_seed(): - """Same seed must yield the same sequence.""" - rng_a = RandomNumberGenerator(seed=42) - rng_b = RandomNumberGenerator(seed=42) - seq_a = [rng_a.randint(100, 99999) for _ in range(50)] - seq_b = [rng_b.randint(100, 99999) for _ in range(50)] - assert seq_a == seq_b - - -def test_randint_covers_full_range(): - """Both endpoints must be reachable across enough samples.""" - rng = RandomNumberGenerator() - values = {rng.randint(0, 3) for _ in range(2000)} - assert values == {0, 1, 2, 3} diff --git a/tests/test_libparallel.py b/tests/test_libparallel.py index 405b978f8e..0bb07ce826 100644 --- a/tests/test_libparallel.py +++ b/tests/test_libparallel.py @@ -1,4 +1,5 @@ import uuid +import time from multiprocessing import Queue from pathlib import Path @@ -53,6 +54,18 @@ def run(self): raise ValueError("Test error") +class DelayedTask(Task): + """Task that lets a later worker finish before the first one.""" + + def __init__(self, input, delay): + super().__init__(input) + self.delay = delay + + def run(self) -> int: + time.sleep(self.delay) + return super().run() + + @pytest.fixture def worker(): """Return a worker with 3 tasks.""" @@ -173,6 +186,22 @@ def test_scheduler_with_exception(scheduler_with_exception): assert scheduler_with_exception.results[2] == 4 +def test_scheduler_returns_results_in_submission_order_despite_worker_skew(): + scheduler = Scheduler( + ncores=2, + tasks=[ + DelayedTask(0, 0.1), + DelayedTask(1, 0.1), + DelayedTask(2, 0), + DelayedTask(3, 0), + ], + ) + + scheduler.run() + + assert scheduler.results == [1, 2, 3, 4] + + def test_generic_task_init(): def sample_function(a, b, c=3): return a + b + c diff --git a/tests/test_libsubprocess.py b/tests/test_libsubprocess.py index a6bc30e726..843df42f91 100644 --- a/tests/test_libsubprocess.py +++ b/tests/test_libsubprocess.py @@ -34,7 +34,7 @@ def cnsjob(mocker): Here we create a temporary file and set it as the mock CNS executable. """ - with tempfile.NamedTemporaryFile() as f: + with tempfile.NamedTemporaryFile() as f, tempfile.TemporaryDirectory() as tempdir: f.file.write(b"") f.file.flush() f.file.seek(0) @@ -42,9 +42,11 @@ def cnsjob(mocker): mocker.patch("haddock.libs.libsubprocess.global_cns_exec", f.name) + input_file = Path(tempdir, "input") + input_file.write_text("stop\n", encoding="utf-8") yield CNSJob( - input_file=Path("input"), - output_file=Path("output"), + input_file=input_file, + output_file=Path(tempdir, "output"), cns_exec=Path(f.name), ) diff --git a/tests/test_module_flexref.py b/tests/test_module_flexref.py index 6ef2073eb5..866afefb05 100644 --- a/tests/test_module_flexref.py +++ b/tests/test_module_flexref.py @@ -48,6 +48,32 @@ def test_prev_fnames(flexref, protdna_input_list): obs_ambig_fnames = flexref.get_ambig_fnames(prev_ambig_fnames) +def test_cns_params_excludes_global_and_python_loop_controls(flexref): + """CNS inputs should receive only the module schema parameter view.""" + flexref.params.update( + { + "ncores": 99, + "debug": True, + "sampling_factor": 3, + "nfle ": 1, + "fle_sta_2": 1, + "fle_end_2": 2, + "fle_seg_2": "A", + } + ) + + cns_params = flexref.cns_params() + + assert "ncores" not in cns_params + assert "debug" not in cns_params + assert "sampling_factor" not in cns_params + assert cns_params["ambig_fname"] == flexref.params["ambig_fname"] + assert cns_params["nfle "] == 1 + assert cns_params["fle_sta_2"] == 1 + assert cns_params["fle_end_2"] == 2 + assert cns_params["fle_seg_2"] == "A" + + def test_multiple_ambigs(flexref): """Test usage of multiple ambigs.""" # Copy ambig.tbl.tgz diff --git a/tests/test_module_rigidbody.py b/tests/test_module_rigidbody.py index 9e9ec2449c..d342e3a06e 100644 --- a/tests/test_module_rigidbody.py +++ b/tests/test_module_rigidbody.py @@ -12,6 +12,10 @@ DEFAULT_CONFIG as DEFAULT_RIGIDBODY_PARAMS, ) from haddock.modules.sampling.rigidbody import HaddockModule as RigidbodyModule +from haddock.modules.sampling.rigidbody import ( + _chainids_for_sampled_combinations, + _repeats_of_sampled_combinations, +) @pytest.fixture(name="rigidbody_module") @@ -43,6 +47,67 @@ def test_prev_fnames(monkeypatch): assert diff_ambig_fnames is None +def test_cns_params_include_spliced_parameter_families(rigidbody_module): + """Parameters assembled from loop variables must reach CNS.""" + expected = { + "nrair_1": 3, + "rair_sta_1_1": 10, + "rair_end_1_1": 20, + "int_1_2": 2.5, + "c2sym_sta1_1": 1, + "s3sym_end3_1": 30, + } + rigidbody_module.params.update({**expected, "ncores": 99}) + + cns_params = rigidbody_module.cns_params() + + assert {name: cns_params[name] for name in expected} == expected + assert "ncores" not in cns_params + + +def test_sample_models_to_dock_is_prefix_stable(rigidbody_module): + """Increasing sampling must only append scheduled combinations.""" + combinations = [[object()], [object()], [object()]] + + for sampling in range(1, 9): + scheduled = rigidbody_module._sample_models_to_dock(combinations, sampling) + assert len(scheduled) == sampling + for prefix_length in range(1, sampling + 1): + assert scheduled[:prefix_length] == rigidbody_module._sample_models_to_dock( + combinations, prefix_length + ) + + +def test_repeats_count_per_combination_not_per_job(): + """A job's repeat index counts its own combination, not the schedule. + + This is what keeps a seed out of the schedule's numbering: combination 2 + is on its first repeat at job 2 of a three-combination run and at job 2 + of a four-combination one alike, so both jobs are the same computation + and receive the same seed. + """ + first, second, third = [object()], [object()], [object()] + schedule = [first, second, third, first, second, third, first] + + assert _repeats_of_sampled_combinations(schedule) == [0, 0, 0, 1, 1, 1, 2] + assert _repeats_of_sampled_combinations(schedule[:4]) == [0, 0, 0, 1] + + +def test_chainids_are_checked_once_per_distinct_combination(mocker): + """Repeated samples reuse the source combination's checked chain IDs.""" + combinations = [[object()], [object()]] + sampled = [combinations[0], combinations[1], combinations[0]] + check = mocker.patch( + "haddock.modules.sampling.rigidbody.check_combination_chains", + side_effect=[["A"], ["B"]], + ) + + observed = _chainids_for_sampled_combinations(combinations, sampled) + + assert observed == [["A"], ["B"], ["A"]] + assert check.call_count == len(combinations) + + def test_rigidbody_make_cns_jobs(rigidbody_module): "???" @@ -84,6 +149,7 @@ def test_rigidbody_make_cns_jobs(rigidbody_module): assert rigidbody_module.output_models[0].seed == seed assert rigidbody_module.output_models[0].ligand_top_fname is None assert rigidbody_module.output_models[0].ligand_param_fname is None + assert observed_jobs[0].output_pdb_files == [Path(output_pdb_name)] def test_rigidbody_make_cns_jobs_with_toppar(rigidbody_module): @@ -140,6 +206,19 @@ def test_rigidbody_make_cns_jobs_with_toppar(rigidbody_module): assert rigidbody_module.output_models[0].ligand_param_fname == "param" +def _models_on_disk() -> tuple[PDBFile, PDBFile]: + """Two models whose bytes exist, because a job's seed is read from them.""" + topology = Persistent(file_name="topology.psf", path=".", file_type=Format.TOPOLOGY) + models = [] + for index, restraints in enumerate(("ambig1.tbl", "ambig2.tbl"), start=1): + name = f"model{index}.pdb" + Path(name).write_text(f"ATOM model {index}\n", encoding="utf-8") + models.append( + PDBFile(Path(name), path=".", restr_fname=restraints, topology=topology) + ) + return models[0], models[1] + + def test_prepare_cns_input_sequential(mocker, rigidbody_module): """???""" @@ -148,13 +227,7 @@ def test_prepare_cns_input_sequential(mocker, rigidbody_module): return_value="cns_input", ) - topology = Persistent(file_name="topology.psf", path=".", file_type=Format.TOPOLOGY) - input_pdb_1 = PDBFile( - Path("model1.pdb"), path=".", restr_fname="ambig1.tbl", topology=topology - ) - input_pdb_2 = PDBFile( - Path("model2.pdb"), path=".", restr_fname="ambig2.tbl", topology=topology - ) + input_pdb_1, input_pdb_2 = _models_on_disk() observed_cns_input_list = rigidbody_module.prepare_cns_input_sequential( models_to_dock=[ [ @@ -162,7 +235,6 @@ def test_prepare_cns_input_sequential(mocker, rigidbody_module): input_pdb_2, ] ], - sampling_factor=1, ambig_fnames=["ambig1.tbl"], ) @@ -179,26 +251,20 @@ def test_prepare_cns_input_parallel(mocker, rigidbody_module): ) mock_prepare_engine = mock_engine_cls.return_value mock_prepare_engine.run.return_value = None + mock_prepare_engine.results = ["cns_input"] mocker.patch( "haddock.modules.sampling.rigidbody.prepare_cns_input", return_value="cns_input", ) - topology = Persistent(file_name="topology.psf", path=".", file_type=Format.TOPOLOGY) - input_pdb_1 = PDBFile( - Path("model1.pdb"), path=".", restr_fname="ambig1.tbl", topology=topology - ) - input_pdb_2 = PDBFile( - Path("model2.pdb"), path=".", restr_fname="ambig2.tbl", topology=topology - ) - observed_cns_input_list = rigidbody_module.prepare_cns_input_sequential( + input_pdb_1, input_pdb_2 = _models_on_disk() + observed_cns_input_list = rigidbody_module.prepare_cns_input_parallel( models_to_dock=[ [ input_pdb_1, input_pdb_2, ] ], - sampling_factor=1, ambig_fnames=["ambig1.tbl"], ) diff --git a/tests/test_module_topocg.py b/tests/test_module_topocg.py index c28901a0be..c601a98b7d 100644 --- a/tests/test_module_topocg.py +++ b/tests/test_module_topocg.py @@ -1,10 +1,11 @@ """Specific tests for topocg.""" +import random import shutil import tempfile +import warnings from math import isnan from pathlib import Path -import warnings from Bio.PDB.PDBExceptions import PDBConstructionWarning import pytest @@ -12,6 +13,7 @@ import haddock.modules.topology.topocg as topocg_mod from haddock.gear.yaml2cfg import read_from_yaml_config from haddock.libs import libpdb +from haddock.libs.libaa2cg import add_dummy from haddock.libs.libontology import Format, PDBFile from haddock.modules.topology.topocg import DEFAULT_CONFIG as topocg_params from haddock.modules.topology.topocg import HaddockModule as Topocg @@ -95,6 +97,17 @@ def test_generate_topology(topocg, protein): assert "BB" in contents, "CG markers not found in output PDB." +def test_dummy_bead_placement_uses_the_supplied_seed(): + beads = [("SC1", [1.0, 2.0, 3.0])] + + first = add_dummy(beads, rng=random.Random(494)) + second = add_dummy(beads, rng=random.Random(494)) + different = add_dummy(beads, rng=random.Random(495)) + + assert first == second + assert first != different + + def test_run_uses_presplit_models_without_splitting(monkeypatch, protein): """Models from the previous (topoaa) step are already split. diff --git a/tests/test_refinement_shape_family.py b/tests/test_refinement_shape_family.py new file mode 100644 index 0000000000..861f3c98a5 --- /dev/null +++ b/tests/test_refinement_shape_family.py @@ -0,0 +1,44 @@ +"""``flexref``, ``emref`` and ``mdref`` are one job shape. + +They differ in their CNS recipe and in nothing else that a schedule can see: +each takes one input model, refines it, and writes one model out. Test sets +that cover job shapes -- including the caching contract suite -- therefore +cover the shape through a single representative, and adding the same case +again for the other two would buy nothing. + +That is sound coverage only while the premise holds, so the premise is a +constraint on the code: + + a change to the job emission, seeding, indexing or cache participation of + one of the three must be applied to all three, or the divergence must be + deliberate and written down. + +Violating it does not turn any test red. The suite stays green while two of +the three modules carry the defect, because the representative has quietly +stopped representing -- which is worse than a coverage gap, since a gap is at +least visible in the case list. + +These tests pin the part of that constraint which can be checked cheaply: the +three modules schedule their replicas and derive their seeds through one +shared implementation rather than through three copies of it. +""" + +import pytest + +from haddock.libs import libcns +from haddock.modules.refinement import emref, flexref, mdref + + +REFINEMENT_FAMILY = (flexref, emref, mdref) + + +@pytest.mark.parametrize("module", REFINEMENT_FAMILY, ids=lambda m: m.__name__) +def test_refinement_family_shares_one_replica_schedule(module): + """One emission order for the shape, not one per module.""" + assert module.refinement_schedule is libcns.refinement_schedule + + +@pytest.mark.parametrize("module", REFINEMENT_FAMILY, ids=lambda m: m.__name__) +def test_refinement_family_shares_one_seed_derivation(module): + """One seeding rule for the shape, not one per module.""" + assert module.derive_seed is libcns.derive_seed