Implement the Subject Sources foundation - #1265
Draft
JeroenDeDauw wants to merge 8 commits into
Draft
Conversation
This was referenced Aug 7, 2026
JeroenDeDauw
force-pushed
the
subject-sources-foundation
branch
from
August 7, 2026 19:05
3dbeeab to
68c40df
Compare
JeroenDeDauw
marked this pull request as ready for review
August 9, 2026 21:23
Member
Author
|
I did not fully review this yet |
JeroenDeDauw
force-pushed
the
subject-sources-foundation
branch
from
August 25, 2026 20:55
2392e27 to
1e809f2
Compare
alistair3149
force-pushed
the
subject-sources-foundation
branch
from
August 31, 2026 17:19
1e809f2 to
cf92872
Compare
Member
Author
|
Note: before merging I'd like to ALSO finish my own review. Prior/additional review by others is useful and welcome |
JeroenDeDauw
force-pushed
the
subject-sources-foundation
branch
2 times, most recently
from
September 1, 2026 23:01
4402842 to
087bcdd
Compare
Member
Author
|
Thanks @alistair3149. I've resumed my review and identified some structural issues (not related to your changes). Will continue on this tomorrow. |
JeroenDeDauw
marked this pull request as draft
September 1, 2026 23:42
This was referenced Sep 7, 2026
JeroenDeDauw
added a commit
that referenced
this pull request
Sep 7, 2026
For #630 The Relations work map had drifted from master. Both authoring paths for intermediate structures have shipped (mapping-side node synthesis, the tree editor with in-place target creation), the relation representation in RDF is recorded as decided in NativeRdfProjection.md, the cross-Source guard moved from the closed #1043 to #1265, Subjects move between pages on their own, the relations endpoint is specified and measured in #1324, and two links and an anchor were broken. The doc also listed shipped in-flow creation as forward work. Rewrite it as a work map for someone starting the implementation: what exists, the six proposed decisions and which of them ratification gates, the questions that are still open, and the work ordered by what it waits on. A fifth shorter, with the PR-by-PR recital, the decided RDF question, and the design-call reassurances gone. Considered, omitted: a GitHub Discussion link, since no thread exists and the epic is the venue; roadmap dates, which live in the ECHOLOT roadmap. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 7, 2026
A Subject's id becomes the Source that produced it paired with that Source's own id for it (ADR 23). A bare `s` + 14 nanoid characters keeps meaning a local Subject, so nothing stored changes; everything else serializes as `sourceKey:localId`, split at the first colon. Canonicalizing an id that names the local wiki explicitly needs to know which Source key is local, which the value object deliberately does not, so that is SubjectIdParser's job. Every place an id arrives as text - a REST path, a revision slot, a relation target, a Lua call - goes through the parser rather than constructing a SubjectId directly, so an id naming this wiki explicitly resolves to the same identity as its bare form. A caller-supplied id on creation must be a local one, since no path creates a Subject in another Source. Both suites run the same parse vectors, so PHP and TypeScript are asserted equal rather than assumed so, and the slot serialization of a local-only page is pinned byte for byte. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Co-Authored-By: alistair3149 <alistair31494322@gmail.com>
A Subject now comes from a Source, per the contract ADR 23 freezes: the object that knows how to fetch it, what its ids look like, which Schemas come with it, and whether it may be edited. The local revision slot becomes one such Source, registered under the MediaWiki Wiki ID, which is what a bare Subject id resolves to. Behaviour-preserving: with only the local Source registered, resolving through the registry reaches exactly what the repository did. An id naming a Source this wiki does not have resolves to no Subject and logs a warning, rather than breaking the page that names it. Sources are registered up front but built on first use, because building the local one reaches the graph projection, which a wiki may not have configured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…targets
A Schema is now referenced as (Source, name) rather than by name alone (ADR 23),
in the two places a reference is stored: a Subject's `schema` field and a
relation property's `targetSchema`. Schema identity stays the name (ADR 17), so
SchemaName is untouched; the reference wraps it.
A Schema of this wiki keeps being written bare, which is also its page title, so
stored Schemas and Subjects are unchanged. Resolution routes through the Source
the reference names, which need not be the Source of the Subject using it; a
reference this wiki cannot resolve reports schema-not-found, the degraded state
the editor already renders, rather than failing.
A local Schema name may itself contain a colon (`ISO:9001`), so a stored string
is never split: it is always one local name. A Schema from elsewhere is stored
as a `{source, name}` object instead, which no local name can be mistaken for.
The qualified `source:name` spelling survives only as a one-way rendering for
people.
Resolution reaches the projection boundary too: the Neo4j and RDF projectors now
resolve a Subject's Schema through its own Source. A relation to a Subject of
another Source gets no Neo4j edge, since the stub node would carry this wiki's
wiki_id, and its RDF triple is named under that Source's own base URI.
Relation targets get ADR 23's v1 guard: a Relation whose target names a Source
this wiki has not registered is refused on the write and validate paths, since
nothing can resolve it. Cross-Source relations open up once resolution for them
exists.
Refused whatever $wgNeoWikiEnforceValidation is set to, because enforcement is
about how strictly a wiki holds data to its Schemas and this is not a Schema
question. Only a violation the edit introduces blocks it, so a Subject that
already carries such a target stays editable, and no read of persisted data is
ever rejected.
For the same reason the check is not Schema-scoped, unlike every other one here:
it runs over every relation value on the proposed Subject, including a Statement
the Schema does not declare and a Subject whose Schema cannot be loaded at all.
Such a target would be unreadable from the moment it is written, which no later
Schema change fixes.
The server is the only validator (ADR 25); the editor renders the violation
through the field-error path it already has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: alistair3149 <alistair31494322@gmail.com>
A Source vouches for what it returns: everything it hands back is readable by every reader of this wiki. NeoWiki performs no per-user authorization on a sourced Subject, because there is no page here to authorize against. A Source over data with restrictions of its own must serve only the unrestricted part, which it decides without knowing who is asking. The read paths now say so rather than arriving at it. GetSubjectQuery discriminates on whether the id is local: a local Subject keeps the #1046 behaviour exactly, page gate and revision-read allowance included, while a sourced one is served without consulting the subject-to-page index at all — that index holds local ids only (ADR 32), so asking it was always going to answer nothing. The page-subjects relation expansion stops dropping sourced targets. It gated every target on a resolvable, readable page, which no sourced target can have, so a relation to another Source rendered as an absent target next to the same relation being served by the single-Subject endpoint. The accepted limitation is that sourced data has no per-user granularity. ADR 27 would have to lift it before a Source may serve data readable to some users and not others. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Schema of another Source is stored as `{source, name}` (ADR 23), and the
server emits references faithfully. Three consumers still assumed the bare
string every reference used to be.
`nw.getSchema` fataled on it. The documented idiom is
`nw.getSchema( subject.schema )`, and a sourced Subject's `schema` field is a
table, which the `?string` parameter turned into a TypeError rather than a Lua
error or a nil. It now takes either form and resolves through the Source the
reference names, so the idiom holds for every Subject; a table that is not a
reference returns nil, like a name nothing offers.
The create dry-run validated against a Schema of this wiki whichever Source the
reference named, so previewing a Subject that follows a sourced Schema answered
schema-not-found. It now resolves the reference the way its sibling
ValidateSubjectUpdateQuery does. The REST endpoint's `schema` body parameter is
still typed as a string, so only the local form reaches it over HTTP.
UpdateStatementAction was the one with teeth: it derived a Statement's property
type from a same-named Schema of this wiki while validating the result against
the foreign one, so setting a value on a sourced Subject was coerced to whatever
type the local Schema gave that property.
The frontend still carries a bare Schema name, so the deserializer collapses a
reference to its name rather than dropping it. Interim, until sourced Subjects
render; the wire type now says what the server actually sends.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A Subject of another Source is fetched from that Source; it is never stored in a page's content (ADR 23). The slot's consumers disagreed about what to do with an entry keyed otherwise, which imported or hand-edited content can carry: the subject-to-page index drops it (ADR 32 keys the index by bare local ids), while both projectors wrote it — Neo4j giving a foreign id this wiki's `wiki_id`, against what graph-model.md says that property means. Deserialization now applies the index's rule: a `subjects` key that is not a local id is skipped, and a `mainSubject` that is not one is read as absent. Both are warnings rather than throws, because the content is persisted and a page has to render regardless — which is also why the qualified-but-local spelling `thisWikiId:s...` is skipped rather than canonicalized: the index cannot record it, so admitting it here would put a Subject in the graph that nothing can find a page for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three claims about the registry had drifted from the code. Asking whether an id resolves built the Source it names. canResolve() went through getSource(), which calls a Closure registration and caches what it returns, so a validation pass over relation targets connected to every Source those targets name — to answer a question the registration alone settles. It now answers from key presence. The lazy-registration rationale named a dependency that no longer exists: since ADR 32 the subject-to-page index is the neowiki_subject_page table, not part of the graph projection, so the local Source builds fine without a graph backend. The Closure overload still earns its place — registration runs under an early hook on every request, whether or not anything reads a Subject — so all four texts say that instead. localId validation was described as source-delegated in ADR 23 and on SubjectId::isValid(). Source::isValidLocalId() is the seam for it and no caller consults it: a foreign localId is checked against the shared serialization grammar alone. Both texts now say so, and the method stays. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two guards. The relation-property editor showed an empty Schema picker for a target Schema from another Source, inviting a selection that would replace the reference with a bare local name; it now shows the reference read-only, so only what the editor can express is editable there. The subjects manager derived a copy-IRI under this wiki's base for any id, which for a Subject of another Source asserts ownership of an entity elsewhere — the refusal the export side already makes in SubjectIriResolver. Neither Subject can reach those surfaces today; both are guards for when one can. The docs catch up with the id and reference forms: the Lua and parser-function pages describe the qualified id, and the Lua page says a Subject's `schema` may be a reference table that getSchema takes. The Neo4j uniqueness constraint says why a bare Subject id is unique on its own rather than assuming it. The id vectors point at docs/api/, where subject-format.md lives. The set-main-subject and ordering endpoints stop advertising a qualified id: they address Subjects on one page, which are local by construction. Plus nits: the create endpoint's 400 gives the reason rather than blaming the format; a detached docblock, a duplicate assignment, and a dead initialisation; the ADR 23 amendment note moves to its Status line; and the batch read path notes an unregistered Source at debug rather than warning on every page view, since the write path already refuses such a target as a violation. Tests cover two deliberate edges that had none: a wiki whose id is not a well-formed Source key still registers and resolves its own bare ids, and cannot be named explicitly. The Source contract gains the keying invariant its callers depend on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
JeroenDeDauw
force-pushed
the
subject-sources-foundation
branch
from
September 7, 2026 23:18
65f3418 to
9412a6d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #993
The ADR 23 foundation. With
only the local Source registered, every path resolves what it did before: nothing user-visible changes and nothing
stored changes.
What lands
T1 —
SubjectIdis a(source, localId)pair. A bares+ 14 nanoid characters still means a local Subject andis still stored as that bare string; anything else is
sourceKey:localId. Every place an id arrives as text — a RESTpath, a revision slot, a relation target, a Lua call — goes through
SubjectIdParser, which is handed the local Sourcekey and resolves nothing. Local slot JSON, Neo4j
idand API output are unchanged, pinned by a byte-identical slottest.
T2 — the Source contract, its registry, and
LocalSource. One interface, one registry keyed by Source key, thelocal revision slot refactored into the default registered Source under the MediaWiki Wiki ID. Extensions contribute
Sources through
NeoWikiRegistrar::addSource(), alongside the Property Type and Page Property Provider registries.ADR 23's "Source interface contract" open question is resolved in the ADR.
T3 — a Schema reference carries its Source.
SchemaReferencepairs a Source with the unchangedSchemaName, inthe two places a reference is persisted: a Subject's
schemaand a relation property'stargetSchema. Resolutionroutes through the Source the reference names, which need not be the Subject's.
T4 — a relation target naming an unregistered Source is an
error-severity violation, raised on the write andvalidate paths and never on read of persisted data. It blocks like every other error (ADR 26): under
$wgNeoWikiEnforceValidation, and only when the edit introduces it, so a Subject already carrying such a target stayseditable. Where one is stored anyway, it degrades on read — the projections drop the edge and the triple, and
resolution yields no Subject.
Design calls
sourceKey:localId, split at the first colon. Source key[A-Za-z][A-Za-z0-9_-]{0,63}; aforeign localId is RFC 3986
pchar, minus percent-encoding and minus&and', capped at 256 characters. EveryPHP grammar is
\z-anchored, so a trailing newline cannot pass.still resolves its own bare ids; only the explicit
<wikiId>:<localId>spelling is unusable there.has one identity — which the id-keyed maps rely on.
{source, name}. A bare string is always a local name, so a Schema titlecontaining a colon (
ISO:9001) stays valid.already has.
constructing clients would be waste, and a Source nobody references is never built.
NeoWiki performs no per-user authorization on sourced Subjects; a Source over restricted data serves only its
unrestricted part. Read paths decide on
isLocal(), never on whether a hosting page was found — local Subjectskeep the page-based checks. Per-user granularity for sourced data is out of scope until demanded (ADR 27).
deserialization skips a non-local subject key with a warning and the index, Neo4j and RDF projections agree. Over
REST and Lua,
schemaandtargetSchemaare references — a bare string when local, the reserved{source, name}object otherwise; consumers tolerate the object form, source-aware display is
Frontend source-awareness: display sourced Subjects and Schema references, enter cross-source relation targets #1373.
isEditable()and foreign-sourceisValidLocalId()are contract surface with no consumer until sourced-subjectrendering lands. They are frozen now because a Source author must answer them from the start.
project. The projection skip covers every foreign target, resolvable or not; T4 refuses only the unresolvable ones.
RelationIdsharedSubjectId's$-anchor newline flaw, fixed here. Pre-existing, one line.Considered, omitted
Source. Write-back brings its own method when it is built.SubjectIdParser. Only canonical ids reach the frontend, so there is nothing to canonicalize there.reinterpret a Schema name that already contains a colon.
wiki_id, and one without it scopesno query correctly.
Follow-ups filed
(pre-existing).
ViolationDiffidentity ignoresargs.Reviewing
The three commits stage the foundation — identity, the Source contract, Schema references and the target guard —
with the review findings from #1330 folded in and co-authored. Five
further commits act on three independent blind-spot reviews (authorization model, tolerant read-side boundary,
local-only slots, registry honesty, UI guards and docs). PHP and TypeScript assert the id grammars against one shared fixture,
tests/vectors/subject-ids.json.Production notes
Design and orchestration by
Fable 5 (max); implementation and the post-review fix batch byOpus 5 (max)subagents.Four independent AI review passes — code, security, tests, and a diff scan against a superseded branch — with every
blocking finding fixed before this description was written.