feat(diagnostics): canonical-graph validation gate - identity, direction, hyperedge schema, determinism

[dolakzdenek]

Companion to #2548 (fixes the root cause of the split-node bug this PR's tests reproduce; this PR adds the detection/defense-in-depth layer so a similar defect is machine-checkable instead of requiring a by-hand trace).

Summary

Adds the validation stage a graph needs before its structural metrics (god nodes/centrality, community detection, dependency analysis) can be trusted, versus treated as informational only. Motivated by tracing a real docs corpus's god-node ranking by hand and finding two of the top three "most connected" nodes were partly an artifact of a duplicate-node bug (see #2548) rather than genuine architectural coupling — there was no machine-checkable way to know that without the manual trace.

What changed

graphify/diagnostics.py (diagnose_extraction):

• canonical: bool + canonical_issues: list[str] — the overall verdict. False on: schema errors, dangling/missing edge endpoints, likely-reversed references/cites edges, or a hard duplicate-node match. Collapse counts, self-loops, and soft duplicate candidates stay informational (visibility, not proof) and never flip it.
• edge_direction_suspects — flags references/cites edges between two whole-file document/paper nodes whose own source_file matches the target node's file rather than the source's. Precise by construction: a node's source_file is where it was authored, an edge's source_file is where the citing assertion was found; when those disagree in exactly this way, the edge is very likely drawn backwards.
• duplicate_node_candidates (hard) / duplicate_node_candidates_soft — canonical node identity. Hard = id B equals id A + _document/_file/_page, the exact pattern a real corpus produced for one file split into two nodes. Soft = label-prefix match on nodes with different source_file, informational only (two genuinely distinct files can coincidentally share a label prefix).
• format_diagnostic_report: prints a CANONICAL/NON-CANONICAL verdict banner plus each new finding; format_diagnostic_json already picks up the new summary fields with no change needed.

graphify/validate.py: hyperedges were never schema-checked (only nodes/edges were) — validate_extraction now checks required fields, the confidence enum, the ≥3-member rule extraction-spec.md's own hyperedge guidance states, and dangling members. Required fields are scoped to what build.py's hyperedge handling (id/label/nodes) actually needs at build time, not the full LLM extraction-time contract, so a minimal hand-constructed hyperedge (e.g. the existing alias-normalization tests) still validates clean.

graphify/cli.py: graphify diagnose multigraph --fail-on-noncanonical, opt-in (default off, so existing scripts/CI keep exit 0) — exits 1 on a non-canonical graph for CI-style gating.

Skill Step 4.5: prints the CANONICAL verdict and, when non-canonical, instructs the agent to caveat god-node/centrality/community claims as informational only and name the specific defect rather than presenting inflated metrics as settled fact.

tests/test_graph_validation.py: hyperedge schema (valid/missing fields/under-3-members/dangling member/no hyperedges key at all), canonical-identity hard vs. soft duplicates (including a direct regression for the docs_architecture split, and negative cases for heading nodes and unrelated documents), citation-edge direction (reversed/correct/scoped away from calls/scoped to whole-file nodes), the CLI flag (both directions, including the backward-compat default), and an end-to-end determinism test — same extraction JSON, full build+cluster+export pipeline run twice, byte-identical graph.json (Louvain is already seeded, but nothing previously locked that guarantee in as a regression test).

Test plan

• All 5 skillgen structural validators pass
• Full pytest suite green (4129 passed; same pre-existing unrelated flaky test as fix(skill): structurally extract docs, fix whole-file node ID drift #2548)
• Ran the new diagnostics against a real 460-node docs corpus and confirmed it correctly flagged all 8 pre-existing duplicate-node pairs, with zero false positives on the clean majority of the graph

🤖 Generated with Claude Code

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds a new --fail-on-noncanonical flag to the graphify diagnose multigraph CLI command, which is opt-in and defaults to off to preserve existing exit-code behavior. It extends diagnose_extraction in diagnostics.py to run schema validation and two new heuristic checks—detecting likely-reversed citation (references/cites) edges and probable duplicate whole-file nodes—and to surface a new canonical boolean plus related fields in the report. Corresponding skillgen fragments, expected skill outputs, and validation tests appear to be updated to reflect these additions. The surface area spans the CLI dispatch logic, the diagnostics module (new helper functions and report fields), skill/fragment content, and test cases.

No blocking issues surfaced. 8 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1400 functions depend on the 839 functions this change touches.

Health — this change adds coupling hotspots:

• worse: build_from_json() — 153 callers, 17 callees
• worse: diagnose_extraction() — 26 callers, 13 callees

Verification — 1400 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 1310 function(s) in the blast radius were not formally verified this run

· 1 grounded finding(s) anchored inline below; 1 more finding(s) on lines outside this diff (see the check run).

[graphify-labs]

⚠️ Health regression — diagnose_extraction()

fans out to 13 callees (efferent coupling); 26 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.