fix(llm): add rationale guidance to API extraction prompt

[hopstreax]

PR Summary

Fixes #2482

Align the standard API extraction prompt (_EXTRACTION_SYSTEM) with the documented rationale extraction behavior by explicitly instructing API backends to capture design rationale as a rationale attribute on the relevant node. The prompt schema has also been updated to advertise the optional rationale field, and prompt-focused tests have been added to prevent regressions.

What changed

• Added rationale extraction guidance to _EXTRACTION_SYSTEM in graphify/llm.py.

• Added the optional rationale field to the node schema example in the prompt.

• Added unit tests verifying that:

  • the API extraction prompt requests rationale extraction, and
  • the prompt schema advertises the rationale field.

Why

The Claude subagent extraction specification already documents how design rationale should be captured, but the standard API extraction prompt did not contain equivalent guidance. As a result, API backends could omit rationale entirely even when the source material contained design intent or trade-offs.

This change makes the API extraction prompt explicitly request rationale extraction while keeping the implementation narrowly focused on prompt behavior without modifying the extraction pipeline or cleanup logic.

Testing

• uv run pytest tests/test_llm_backends.py
• uv run pytest tests/test_semantic_cleanup.py
• uv run pytest tests/test_rationale.py

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds a rationale-capture instruction to the native LLM extraction system prompt in graphify/llm.py. It tells the model to store the reasoning behind decisions as a rationale attribute on nodes rather than as separate rationale nodes, and adds "rationale":null to the node schema example. A new test, test_native_extraction_prompt_requests_rationale, is added to verify the prompt (for both deep and non-deep modes) contains the new instruction phrases and the updated schema field.

No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 660 functions depend on the 251 functions this change touches.

Health — grade B; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

• deduplicate_entities() — 49 callers, 21 callees (high)
• build_merge() — 41 callers, 14 callees (high)
• extract_files_direct() — 15 callers, 20 callees (high)
• dispatch_command() — 2 callers, 118 callees (high)
• extract_corpus_parallel() — 23 callers, 10 callees (high)
• _call_claude_cli() — 26 callers, 8 callees (high)
• _call_openai_compat() — 23 callers, 8 callees (high)
• _extract_with_adaptive_retry() — 14 callers, 9 callees (high)
• …and 2 more

Verification — 660 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 435 function(s) in the blast radius were not formally verified this run

[graphify-labs]

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds a new instruction to the LLM extraction system prompt (_EXTRACTION_SYSTEM) directing the model to capture design rationale as a rationale attribute on nodes rather than as separate rationale nodes, and updates the output schema example to include a "rationale":null field on nodes. It also adds a corresponding test (test_native_extraction_prompt_requests_rationale) that verifies the prompt contains the new rationale wording and schema entry for both the deep and non-deep extraction variants. The changes touch graphify/llm.py and tests/test_llm_backends.py.

No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 660 functions depend on the 251 functions this change touches.

Health — grade B; 10 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

• deduplicate_entities() — 49 callers, 21 callees (high)
• build_merge() — 41 callers, 14 callees (high)
• extract_files_direct() — 15 callers, 20 callees (high)
• dispatch_command() — 2 callers, 118 callees (high)
• extract_corpus_parallel() — 23 callers, 10 callees (high)
• _call_claude_cli() — 26 callers, 8 callees (high)
• _call_openai_compat() — 23 callers, 8 callees (high)
• _extract_with_adaptive_retry() — 14 callers, 9 callees (high)
• …and 2 more

Verification — 660 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 435 function(s) in the blast radius were not formally verified this run