fix(skill): structurally extract docs, fix whole-file node ID drift

[dolakzdenek]

Summary

Part A of the skill only ever ran deterministic AST extraction over code files, even though extract() already ships a structural Markdown extractor (extract_markdown, .md/.mdx/.qmd/.skill) that mints a stable whole-file node + one node per heading. Every markdown file went through the LLM semantic path only, so nothing upstream had already decided that file's canonical node id — each Part B subagent had to invent one independently.

Running /graphify on a 48-file real-world docs corpus reproduced the failure mode this causes: two subagents extracting the same doc suite in parallel independently produced docs_architecture and docs_architecture_document for one file (ARCHITECTURE.md), splitting it into two disconnected nodes and inflating its apparent "god node" degree with edges that actually belonged to the other shard. The same pattern showed up for 8 files in that corpus.

What changed

• graphify/extract.py: adds structural_extensions(), a public accessor for which extensions have a registered structural extractor (derived from the existing _DISPATCH table, not a hand-maintained duplicate list — the exact drift hazard graphify.ids documents for node-ID recipes). Exposed via graphify.__init__'s lazy-import map.
• Skill Part A: also structurally extracts document/paper files whose extension is in structural_extensions(), not just code. Part C's existing "AST nodes first, semantic deduped by id" merge then naturally unifies any LLM-created whole-file node into the AST-authoritative one, once both agree on the id — no new dedup logic needed.
• extraction-spec.md (verbose + compact):
  1. Defines the whole-file node ID convention explicitly — bare {stem}, matching extract_markdown's own file_nid, never a _document/_file suffix.
  2. Adds the citer→citee direction rule for references/cites edges between whole-file document nodes, mirroring the existing calls rule for code, plus a self-check subagents can apply.
  3. A stub node representing a file outside the subagent's own FILE_LIST still uses the citing file's path as its source_file — the normal rule, unchanged. An earlier draft of this fix pointed a stub node's source_file at the referenced file instead (on the theory that a node should describe the file it represents); running that draft as a live incremental --update against a real 460-node docs graph destroyed 293 of those nodes (64%) in one run, because build_merge's replace-on-re-extract logic treats any node's source_file appearing in a new extraction as proof that file was just re-scanned. tests/test_build_merge_hyperedges_and_prune.py adds a regression test reproducing this exact mechanism in miniature.
• tools/skillgen/gen.py: renames SHARED_INTRO_ALLOWLIST → SHARED_CORE_ALLOWLIST (now holds intro consolidation and shared heading renames) and allowlists the Part A heading rename for the per-host coverage audit; adds _is_structural_docs_fix_line to the monolith-roundtrip guard for aider/devin.
• tests/test_extraction_spec_ids.py: locks the whole-file-node ID convention to the real extract._make_id/_file_stem functions, plus a direct regression test for the exact docs_architecture/docs_architecture_document split.

Test plan

• All 5 skillgen structural validators pass: --check, --audit-coverage, --schema-singleton, --monolith-roundtrip, --always-on-roundtrip
• Full pytest suite green (4126 passed; one pre-existing flaky test unrelated to this change, confirmed flaky on a clean v8 checkout too)
• Manual end-to-end smoke test: two interlinked markdown files, confirmed whole-file nodes converge on the same id and citation edges resolve in the correct direction
• Regression test for the build_merge data-loss hazard found while developing this fix

🤖 Generated with Claude Code

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR extends the skill pipeline's "Part A" structural extraction so it runs over documentation/paper files whose extensions have a registered structural extractor (e.g. .md, .mdx, .qmd, .skill), not just code files. To support this, it adds a new structural_extensions() helper in graphify/extract.py (exposed via graphify/__init__.py) that returns the set of extensions backed by a structural extractor, and rewrites the Part A snippets across the many per-agent skill markdown files and their generated/expected test fixtures to use it. The changes span the skill generator tooling, expected-output fixtures, and associated tests.

No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2562 functions depend on the 1464 functions this change touches.

Health — grade A; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

• extract() — 375 callers, 39 callees (high)
• _rebuild_code() — 93 callers, 52 callees (high)
• extract_xaml() — 19 callers, 17 callees (high)
• dispatch_command() — 2 callers, 118 callees (high)
• _get_extractor() — 24 callers, 6 callees (high)
• run_pipeline() — 8 callers, 13 callees (high)
• collect_files() — 17 callers, 6 callees (high)
• render() — 13 callers, 5 callees (high)
• …and 2 more

Verification — 2562 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 2432 function(s) in the blast radius were not formally verified this run