feat(yaml): extract Docker Compose services and GitHub Actions jobs - #2541
feat(yaml): extract Docker Compose services and GitHub Actions jobs#2541bruno-growthsales wants to merge 1 commit into
Conversation
.yaml/.yml had no AST extractor, so infra-as-config contributed nothing structural to the graph. On a repo whose deploy path is docker-build.yml -> Portainer/Swarm, the deploy topology was the one part of the architecture the graph could not answer questions about. Adds extract_yaml behind an optional [yaml] extra (tree-sitter-yaml), mirroring the sql/terraform pattern: - Compose: service nodes; depends_on edges for both the list and the long-form mapping shape; extends -> base service. - Actions: job nodes; needs -> depends_on (scalar and list forms); step-level and job-level `uses` -> the action / reusable workflow. - Data YAML (k8s manifests, OpenAPI specs, fixtures) returns an empty result and is left to the semantic pass, the same way _is_config_json skips data JSON (Graphify-Labs#1224). A 1 MiB ceiling with the same bounded read keeps multi-MB lockfiles from being parsed for nothing. Extensions stay in DOC_EXTENSIONS. collect_files keys off _DISPATCH, not CODE_EXTENSIONS, so registering the extractor is enough to get the structural pass without removing YAML from the semantic one -- the same arrangement .md already has, and it avoids the Graphify-Labs#1689 "classified as code but no extractor" warning. Definitions are file-scoped and carry a contains edge; cross-file references are sourceless stubs so _rewire_unique_stub_nodes can collapse a Compose overlay's depends_on onto the base file's service (Graphify-Labs#2324, Graphify-Labs#1402). An external `uses` target is marked type=module: the same action pinned by ten workflows is ONE action, so it takes the module-anchor exemption in _disambiguate_colliding_node_ids (Graphify-Labs#1327) instead of scattering into ten path-salted nodes. uv.lock also picks up the 0.9.31 -> 0.9.35 project-version correction it had drifted from; CI runs --frozen, so the lock had to be regenerated for the new extra regardless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013EeP6fFjQkaE2CxNz3RTnY
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR expands CI/CD and repository tooling for the graphify project. It adds several GitHub Actions workflows (a skillgen artifact-consistency check, PyPI trusted publishing, a self-graph release asset builder, and security scanning via bandit/pip-audit), switches the existing test workflow from pip to uv with full-history checkouts, and broadens the trigger branches. It also introduces supporting config files: .dockerignore, .gitattributes (Linguist HTML handling), .github/FUNDING.yml, a pre-commit hook running the skillgen check, and expanded .gitignore entries. The changed-symbols list additionally references many test, extractor, and skillgen-related files across the codebase, suggesting accompanying source/test/fixture changes not shown in the truncated diff. Reviewers should look at the full diff for those areas since only the CI/config portion is visible here.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 12643 functions depend on the 12624 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
extract()— 373 callers, 39 callees - worse:
_rebuild_code()— 93 callers, 52 callees - worse:
build_from_json()— 153 callers, 17 callees - worse:
detect()— 82 callers, 13 callees - new:
deduplicate_entities()— 49 callers, 21 callees - new:
build_merge()— 41 callers, 14 callees - worse:
save_semantic_cache()— 50 callers, 9 callees - new:
_extract_generic()— 18 callers, 20 callees - …and 198 more
Verification — 12643 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 12643 function(s) in the blast radius were not formally verified this run
|
Disclosing a limitation I hit while running this on a real monorepo, since a reviewer would reasonably ask about it. The extractor is not reachable from code_files = [Path(p) for p in files_by_type.get("code", [])]
In practice the build is two steps: This is pre-existing and not introduced here — Two ways forward, whichever you prefer:
What I'd argue against is moving Separately, while testing this I ran into an unrelated bug that made the results confusing at first — filed as #2543. |
Problem
.yaml/.ymlhave no AST extractor, so infrastructure-as-config contributes nothing structural to the graph. On a repo whose deploy path isdocker-build.yml→ Portainer/Swarm, the deploy topology was the one part of the architecture the graph could not answer questions about — the files are read by the semantic pass, but the service and job dependencies never become edges.What this adds
extract_yaml, behind an optional[yaml]extra (tree-sitter-yaml), mirroring thesql/terraformpattern:depends_onedges for both the list form and the long-form mapping ({db: {condition: ...}});extends→ base service.needs→depends_on(scalar and list forms); step-level and job-leveluses→ the action or reusable workflow._is_config_jsonskips data JSON (AST pass explodes data .json into orphan key-nodes (CODE_EXTENSIONS includes .json) — 561 isolated nodes on a real repo #1224). A 1 MiB ceiling with the same bounded read keeps multi-MB lockfiles from being parsed for nothing.On a real monorepo:
docker-compose.yml→ 5 services and 4depends_onedges;ci.yml→ 12 jobs, thebuild → lint-and-typecheckdependency, and 22usesedges.Two design decisions worth reviewing
1. The extensions stay in
DOC_EXTENSIONS.collect_fileskeys off_DISPATCH, notCODE_EXTENSIONS, so registering the extractor is enough to get the structural pass without removing YAML from the semantic one — the same arrangement.mdalready has. It also avoids the #1689 "classified as code but no extractor" warning that moving the extensions would otherwise require handling.2. External
usestargets are markedtype=module. Definitions are file-scoped with acontainsedge; cross-file references are sourceless stubs so_rewire_unique_stub_nodescan collapse a Compose overlay'sdepends_ononto the base file's service (#2324, #1402). But anactions/checkout@v4pinned by ten workflows is never defined anywhere in the corpus, so there is nothing to rewire onto, and_disambiguate_colliding_node_idssalts each stub with its own path — scattering one shared action into ten nodes. Marking themtype=moduletakes the module-anchor exemption (#1327), whose rationale reads exactly like this case: "those are the same module, not distinct same-named symbols". Happy to take a different approach here if you would rather not overloadtype.Tests
tests/test_yaml_config.py— 19 tests covering both shapes, the sequence-marker trap (- apivsapi), forward references binding locally, data YAML staying empty, and the two cross-file cases (overlay rewire, shared-action merge) exercised throughextract()andbuild_from_json.Full suite: 4053 passed, 0 failed.
ruff checkclean.Note on
uv.lockCI runs
uv sync --frozen, so the lock had to be regenerated for the new extra. Regenerating also picked up a0.9.31 → 0.9.35project-version correction the lock had drifted from — unrelated to this change, but not separable from it.🤖 Generated with Claude Code
https://claude.ai/code/session_013EeP6fFjQkaE2CxNz3RTnY