Skip to content

Drift-aware, link-preserving mint/supersede publishing - #8

Merged
GertjanBisschop merged 2 commits into
eu-parc:mainfrom
knowledgepixels:drift-aware-publishing
Jul 7, 2026
Merged

GertjanBisschop merged 2 commits into
eu-parc:mainfrom
knowledgepixels:drift-aware-publishing

Conversation

@tkuhn

@tkuhn tkuhn commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Makes republishing update already-published defining nanopubs instead of only appending new ones, and preserves inter-term link resolution when it does.

Motivation

Publishing was append-only: mint_publish skipped any term already in the id-map, keyed solely on its old id — blind to a change in the term's content or in a batch-level wrapper (template, dcterms:isPartOf, nanopub-type, license). There was no way to re-issue a published nanopub when, say, the assertion template changed.

What this adds

Drift fingerprint (fingerprint.py) — a key-independent SHA-256 over the identity-defining inputs only: the URDNA2015-canonicalized assertion plus suggester/derived_from/license/introduces/nanopub-type/template. Computed on the placeholder form so the thing URI never feeds its own hash. Excludes the build timestamp, signature and blank-node/order noise (no-op tier) and the signing key / toolchain (build-provenance tier), so a key rotation cannot masquerade as content drift.

Id-map schema (idmap.py) — a fourth fingerprint column; still reads legacy 3-column files.

Incremental publisher (incremental.py) — publish_incremental(): per term, mint (new), skip (fingerprint unchanged), or supersede (drifted). A superseded term is re-stated against its existing fixed thing URI in a nanopub that npx:supersedes the recorded one, keeping the term's identity; the id-map is repointed. Legacy rows without a fingerprint are backfilled as a baseline, not mass-superseded. Wired into cli/mint_publish.py.

Drift-aware migration (migrate.py) — the migration now records each term's fingerprint (over the as-given source assertion, same wrapper — identical to what publish_incremental recomputes), so a later incremental run over the migrated sources skips unchanged terms instead of hitting the backfill branch and silently swallowing a post-migration wrapper change.

Link-preserving supersede — migrate's reference resolver is extracted to references.resolve_references() and reused on the incremental supersede path: re-issuing a term resolves its inter-term references to new thing URIs via the id-map (all targets already minted) rather than reverting to old-id references. The mint path (new terms) still mints as-given. Migration also repoints the id-map at the cyclic-link supersession so the supersedes chain stays linear.

Behavior

State Condition Action
new absent from id-map mint defining nanopub, record fingerprint
unchanged fingerprint matches skip
drifted fingerprint differs supersede against fixed thing URI (references resolved), repoint id-map
legacy fingerprint empty (3-col row) backfill baseline, skip

Tests

New test_fingerprint.py / test_incremental.py and additions to test_idmap.py / test_migrate.py / test_cli_mint_publish.py cover: determinism, cosmetic/no-op invariance, each identity field moving the fingerprint, key-independence, the mint/skip/supersede/backfill decisions, CLI end-to-end (superseding .trig with npx:supersedes + repointed id-map), migration fingerprint consistency (no phantom drift), a template change superseding migrated terms, and reference resolution preserved on supersede. Full suite: 124 passed.

🤖 Generated with Claude Code

tkuhn and others added 2 commits July 6, 2026 16:55
Publishing was append-only: mint_publish skipped any term already in the
id-map keyed solely on its old id, so a change to an already-published
term -- whether to its content or to a batch-level wrapper (template,
part-of, nanopub-type, license) -- was never detected or re-issued.

Add a semantic drift fingerprint and an incremental publisher that acts
on it:

- fingerprint.py: a key-independent SHA-256 over the identity-defining
  inputs only (URDNA2015-canonicalized assertion + suggester/derived_from
  + license/introduces/nanopub-type/template), computed on the placeholder
  form so the thing URI never feeds its own hash. Excludes the build
  timestamp, signature and blank-node/order noise (no-op tier) and the
  signing key/toolchain (build-provenance tier), so a key rotation cannot
  read as content drift.
- idmap.py: fourth `fingerprint` column; reads legacy 3-column files.
- incremental.py: publish_incremental() -- per term, mint (new), skip
  (fingerprint unchanged), or supersede (drifted) against the existing
  fixed thing URI so the term keeps its identity across versions. Legacy
  rows without a fingerprint are backfilled as a baseline, not
  mass-superseded.
- cli/mint_publish.py: build a matching SupersessionBuilder, drive
  publish_incremental, write minted and superseding nanopubs by their own
  artifact code, and persist fingerprints in the id-map.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two related changes so already-published terms can be reused and updated
instead of re-minted, and so updates preserve inter-term link resolution.

1. Record drift fingerprints during migration. migrate_terms seeded the
   id-map with no fingerprint, so migrated terms landed empty and a later
   incremental run would hit the legacy-backfill branch and silently adopt
   current settings as baseline -- swallowing a wrapper change (e.g. a new
   template) made after migration. Now each term's fingerprint is recorded
   at migration time, computed over the as-given SOURCE assertion with the
   same wrapper, identical to what publish_incremental recomputes later --
   so an unchanged re-run skips (no phantom drift) and only a real change
   supersedes.

2. Resolve inter-term references on supersede. The incremental supersede
   path rebuilt the assertion from source but kept references as old-id
   URIs, reverting the migration's resolution. Extract migrate's resolver
   as references.resolve_references() and use it in publish_incremental:
   rebuild from source with every reference resolved to its new thing URI
   via the id-map (all targets are already minted, so no ordering needed).
   The mint path (new terms) still mints as-given -- resolution applies on
   supersede, where targets are guaranteed minted. Also repoint the id-map
   at the migration's cyclic-link supersession so the supersedes chain
   stays linear.

Full suite: 124 passed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@GertjanBisschop

Copy link
Copy Markdown
Contributor

Hi Tobias, thanks a bunch for this. This is indeed the next thing that we'd be needing. But you beat me to it.
Just checking, is this PR ready for evaluating and merging, or was Claude getting quite trigger happy again?

@tkuhn

tkuhn commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi Gertjan! Sorry, if I interfered with your plans. I hope it's fine like this.

This PR was intentional but good that you ask :)

It looks all good to me and is tested, so ready to merge from my side. But let me know if you'd like to have more info, or like the PR in a different form.

@GertjanBisschop

Copy link
Copy Markdown
Contributor

Awesome! Merging.

@GertjanBisschop
GertjanBisschop merged commit 6ef8128 into eu-parc:main Jul 7, 2026
3 checks passed
GertjanBisschop pushed a commit to eu-parc/biochementity-vocabulary that referenced this pull request Jul 8, 2026
…late (#50)

* Pin pubmate to v0.2.2 (drift-aware, link-preserving publishing)

v0.2.2 lands eu-parc/pubmate#8: publish_incremental()/fingerprint_term()
detect content or wrapper drift (template, part-of, type, license) on
already-published terms and supersede them in place, keeping thing URIs
and resolving inter-term links. Replaces the append-only v0.2.1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Point NANOPUB_TEMPLATE at the updated "Defining a biochementity" template

Update the nt:wasCreatedFromTemplate tag stamped on every minted nanopub
to the new published assertion template
(RALm3XedpEbjtQy1nPJEQpOdsV0hPm5APkvvym-7P1Vpk), so Nanodash renders the
terms with the current form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants