Repository navigation
Drift-aware, link-preserving mint/supersede publishing - #8
Merged
GertjanBisschop merged 2 commits intoJul 7, 2026
Merged
Conversation
Publishing was append-only: mint_publish skipped any term already in the id-map keyed solely on its old id, so a change to an already-published term -- whether to its content or to a batch-level wrapper (template, part-of, nanopub-type, license) -- was never detected or re-issued. Add a semantic drift fingerprint and an incremental publisher that acts on it: - fingerprint.py: a key-independent SHA-256 over the identity-defining inputs only (URDNA2015-canonicalized assertion + suggester/derived_from + license/introduces/nanopub-type/template), computed on the placeholder form so the thing URI never feeds its own hash. Excludes the build timestamp, signature and blank-node/order noise (no-op tier) and the signing key/toolchain (build-provenance tier), so a key rotation cannot read as content drift. - idmap.py: fourth `fingerprint` column; reads legacy 3-column files. - incremental.py: publish_incremental() -- per term, mint (new), skip (fingerprint unchanged), or supersede (drifted) against the existing fixed thing URI so the term keeps its identity across versions. Legacy rows without a fingerprint are backfilled as a baseline, not mass-superseded. - cli/mint_publish.py: build a matching SupersessionBuilder, drive publish_incremental, write minted and superseding nanopubs by their own artifact code, and persist fingerprints in the id-map. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two related changes so already-published terms can be reused and updated instead of re-minted, and so updates preserve inter-term link resolution. 1. Record drift fingerprints during migration. migrate_terms seeded the id-map with no fingerprint, so migrated terms landed empty and a later incremental run would hit the legacy-backfill branch and silently adopt current settings as baseline -- swallowing a wrapper change (e.g. a new template) made after migration. Now each term's fingerprint is recorded at migration time, computed over the as-given SOURCE assertion with the same wrapper, identical to what publish_incremental recomputes later -- so an unchanged re-run skips (no phantom drift) and only a real change supersedes. 2. Resolve inter-term references on supersede. The incremental supersede path rebuilt the assertion from source but kept references as old-id URIs, reverting the migration's resolution. Extract migrate's resolver as references.resolve_references() and use it in publish_incremental: rebuild from source with every reference resolved to its new thing URI via the id-map (all targets are already minted, so no ordering needed). The mint path (new terms) still mints as-given -- resolution applies on supersede, where targets are guaranteed minted. Also repoint the id-map at the migration's cyclic-link supersession so the supersedes chain stays linear. Full suite: 124 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
|
Hi Tobias, thanks a bunch for this. This is indeed the next thing that we'd be needing. But you beat me to it. |
Contributor
Author
|
Hi Gertjan! Sorry, if I interfered with your plans. I hope it's fine like this. This PR was intentional but good that you ask :) It looks all good to me and is tested, so ready to merge from my side. But let me know if you'd like to have more info, or like the PR in a different form. |
Contributor
|
Awesome! Merging. |
GertjanBisschop
pushed a commit
to eu-parc/biochementity-vocabulary
that referenced
this pull request
Jul 8, 2026
…late (#50) * Pin pubmate to v0.2.2 (drift-aware, link-preserving publishing) v0.2.2 lands eu-parc/pubmate#8: publish_incremental()/fingerprint_term() detect content or wrapper drift (template, part-of, type, license) on already-published terms and supersede them in place, keeping thing URIs and resolving inter-term links. Replaces the append-only v0.2.1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Point NANOPUB_TEMPLATE at the updated "Defining a biochementity" template Update the nt:wasCreatedFromTemplate tag stamped on every minted nanopub to the new published assertion template (RALm3XedpEbjtQy1nPJEQpOdsV0hPm5APkvvym-7P1Vpk), so Nanodash renders the terms with the current form. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes republishing update already-published defining nanopubs instead of only appending new ones, and preserves inter-term link resolution when it does.
Motivation
Publishing was append-only:
mint_publishskipped any term already in the id-map, keyed solely on its old id — blind to a change in the term's content or in a batch-level wrapper (template,dcterms:isPartOf, nanopub-type, license). There was no way to re-issue a published nanopub when, say, the assertion template changed.What this adds
Drift fingerprint (
fingerprint.py) — a key-independent SHA-256 over the identity-defining inputs only: the URDNA2015-canonicalized assertion plus suggester/derived_from/license/introduces/nanopub-type/template. Computed on the placeholder form so the thing URI never feeds its own hash. Excludes the build timestamp, signature and blank-node/order noise (no-op tier) and the signing key / toolchain (build-provenance tier), so a key rotation cannot masquerade as content drift.Id-map schema (
idmap.py) — a fourthfingerprintcolumn; still reads legacy 3-column files.Incremental publisher (
incremental.py) —publish_incremental(): per term, mint (new), skip (fingerprint unchanged), or supersede (drifted). A superseded term is re-stated against its existing fixed thing URI in a nanopub thatnpx:supersedesthe recorded one, keeping the term's identity; the id-map is repointed. Legacy rows without a fingerprint are backfilled as a baseline, not mass-superseded. Wired intocli/mint_publish.py.Drift-aware migration (
migrate.py) — the migration now records each term's fingerprint (over the as-given source assertion, same wrapper — identical to whatpublish_incrementalrecomputes), so a later incremental run over the migrated sources skips unchanged terms instead of hitting the backfill branch and silently swallowing a post-migration wrapper change.Link-preserving supersede —
migrate's reference resolver is extracted toreferences.resolve_references()and reused on the incremental supersede path: re-issuing a term resolves its inter-term references to new thing URIs via the id-map (all targets already minted) rather than reverting to old-id references. The mint path (new terms) still mints as-given. Migration also repoints the id-map at the cyclic-link supersession so the supersedes chain stays linear.Behavior
Tests
New
test_fingerprint.py/test_incremental.pyand additions totest_idmap.py/test_migrate.py/test_cli_mint_publish.pycover: determinism, cosmetic/no-op invariance, each identity field moving the fingerprint, key-independence, the mint/skip/supersede/backfill decisions, CLI end-to-end (superseding.trigwithnpx:supersedes+ repointed id-map), migration fingerprint consistency (no phantom drift), a template change superseding migrated terms, and reference resolution preserved on supersede. Full suite: 124 passed.🤖 Generated with Claude Code