build(docs): add documentation check and reference generation scripts

[Suneha Bose (bosesuneha)]

PR 1 of 3. Tooling only, no documentation changes.

Why

An audit of docs/ found ~60 inaccuracies against 2.3.0. About 70% were hand-copied restatements of package.json: command IDs, menu paths, setting defaults, pinned versions. Nothing ran before merge to catch them, because website.yaml only triggers on push to main.

aks.simplifiedMenuStructure defaulting to true invalidated the "how to open this" instructions across 11 pages in one change. That is mechanically detectable, so it should be mechanically detected.

What this adds

Script	Purpose
npm run docs:check	Checks that need to understand package.json
npm run docs:reference	Regenerate the generated reference pages
npm run docs:reference:check	Fail if those pages are stale
npm run test:scripts	Unit tests for the above, plain node + mocha, no compile step

scripts/lib/menu-graph.js resolves the cluster context menu from package.json, evaluating each when clause once per value of aks.simplifiedMenuStructure. Both scripts share it, so they cannot disagree about the menu layout.

docs-check.js runs four checks, each selectable by name (npm run docs:check menu-paths):

Check	Catches
identifiers	Command IDs in prose that package.json does not contribute
menu-paths	Menu breadcrumbs that do not match the real menu
menu-syntax	Menu navigation written as prose instead of **A** > **B**
coverage	Commands documented nowhere in prose (warning)
orphans	Images no page references (warning)

It deliberately does not check links, images, anchors, or SUMMARY completeness. lychee --offline --include-fragments covers the first three and handles URL fragments and raw HTML properly. mdbook build with create-missing = false fails on a SUMMARY entry with no page. Both are intended as CI gates, so duplicating them here was redundant, and measurably worse: an earlier version of this file missed images referenced with <img> tags, which lychee caught.

generate-docs-reference.js renders the menu graph, settings, and pinned third-party versions as markdown. --check regenerates in memory and fails on drift, so committed output cannot silently fall behind package.json.

Review fixes

Three defects in the menu graph, each now covered by a test:

• Submenu cycle guard never fired. It keyed on bucket plus breadcrumb, and the breadcrumb grows on every hop, so the key was fresh on each recursion. An A → B → A pair recursed to stack overflow rather than stopping. Now keys on the bucket alone, and runs before the label is recorded rather than only before the recursion, so a cycle contributes no path instead of one dangling breadcrumb.
• false and never read as reachable. evaluateWhen skipped any conjunct matching no rule, leaving the disjunct satisfiable, so a command hidden that way would be documented as present in the menu and a breadcrumb pointing at it would validate. buildPaletteVisibility already handled both for the palette; the menu walk now does too.
• Only the subscription node tolerated the equality form. viewItem == aks.cluster fell through to any and rendered as Any > ... with nothing flagged. Every NODE_PATTERNS row now accepts both =~ and ==. != is deliberately excluded, since it would resolve a negation to the node it excludes.

Also addressed:

• The unresolved-node diagnostic filtered on "Other", which nodeOf never returns, so it was always empty and NODE_LABEL.other unreachable. It now targets the label for any, which is the value that actually signals an unresolved node, and would have caught the equality-form gap on its own.
• process.exit() immediately after console.log can discard buffered stdout when it is a pipe rather than a TTY, which is how CI runs these. Both scripts now set process.exitCode and return.
• identifiers no longer scans fenced code blocks, so a settings.json sample naming another extension's azure.* key is not a hard error. Fences are kept so line numbers stay accurate; inline code is untouched, because coverage relies on it naming real commands.
• Dropped contributions().version, the pkg and readJson bindings in the generator, and the friendlyFlag alias, none of which had a reader.
• The classic-menu opt-out was documented only in the source, so a docs author hitting a false positive had no way to discover it. Documented on the Package Scripts page along with the three docs scripts, which were never added there when they were introduced.

Rebased onto main. Worth flagging for anyone reviewing the diff: docs/package-scripts.md was deleted in #2370 as a stale fork, so that documentation went to docs/book/src/development/development.md instead.

Testing

npm run test:scripts — 53 tests on splitTop, stripOuterParens, normaliseBoolean, evaluateWhen, walk, crumbsIn, proseNavIn, withoutFencedCode and withoutUrls, including a regression for each of the three defects above. The cycle test builds a synthetic A → B → A submenu pair and asserts the walk terminates; against the previous guard it raises RangeError: Maximum call stack size exceeded.

Nothing in CI runs them yet. eslint.config.mjs ignores **/*.js and format-check.yml only triggers on ts/tsx, so test:scripts, docs:check and Prettier over scripts/ are all green locally and unenforced. Wiring that up is follow-up work, together with the lychee gate.

menu-paths was verified by deliberately breaking things: renaming a submenu in package.nls.json flags every affected breadcrumb with the correct new path, and moving a command to another tree node is reported as such. --check was verified to exit 1 on drift and 0 after regeneration.

Current state against main

npm run docs:check reports 2 errors: two retina-capture.md menu paths describe the classic menu rather than the grouped default. Fixed in #2372, left visible here so this can be reviewed as tooling on its own.

npm run docs:reference:check fails on a clean checkout, because the generated pages under docs/book/src/reference/ are not committed and are not in SUMMARY.md. Both land in #2372, which keeps this PR to tooling.

The > convention, and menu-syntax

menu-paths recognises a breadcrumb only when > separates the steps. The book writes navigation as prose ("and select", "and then click on"), so 23 of the right-click instructions naming a bold UI element were skipped rather than validated.

That failure was silent, which is the same shape of problem these checks exist to prevent. It is why retina-capture.md:5 went unreported while lines 11 and 33 of the same file were caught, despite all three describing the same stale path.

Why not teach the parser the connector phrases

Prototyped and rejected. It works, and takes menu-paths from 2 errors to 18, including all five cases raised in review. But the 16 new ones are not uniform:

• ~13 genuinely stale
• 3 misdiagnosed. image-cleaner-eraser-tool.md:5 reports "on the k8s-cluster node, not cluster" when it is on the cluster node, at Troubleshoot & Diagnose > Run Eraser Image Cleanup. The real fault is an omitted parent level, so the message points at the wrong fix.
• at least 1 false positive. show-properties-azureportal-start-stop.md:25 writes **Create Cluster** and select **Create Standard Cluster**, where the second is a button in the wizard the first opens.

The false positive is not fixable with a better phrase list. Prose uses the same connector for a menu hop and an in-app action, so the parser has no signal to separate them, and the list would never be complete either.

What this does instead

menu-syntax keeps > required and makes omitting it a visible error rather than silence. It checks the convention, not the menu, so it cannot produce the wizard-step mistake.

Right-click your AKS cluster > **Troubleshoot & Diagnose** > **Collect TCP Dumps**

> also states where the menu ends, which prose cannot, so it helps readers for the same reason it helps the checker.

A line menu-paths can already read is exempt, so nothing is reported twice. Bold preceding the right-click is ignored, since naming a UI element is not an instruction. A page can opt out with docs-check: not-a-menu.

23 errors today; #2372 does the rewrites. Worth flagging for that PR: it is more than punctuation, since several pages describe commands as sitting directly on the cluster menu when they are nested. The real paths should come from the generated reference/commands.md.

Requiring docs-check.js as a module no longer runs the checks, so the line-level parsers are testable directly. Doing that immediately caught a wrong menu path in the example added to the development page, which had omitted Troubleshoot Network Health.

Follow-up work

• CI gates: lychee, mdbook build, docs:check, docs:reference:check, test:scripts, and Prettier/ESLint coverage for scripts/
• Findings the generator surfaces, worth filing separately: dead submenu aks.tcpDataCollectionSubMenu, duplicate palette title "Create KAITO Workspace", AKS: AKS: double-prefix on two kickstart commands, and five GitHub Actions version skews

[Tatsat (Tats) Mishra 🐉 (Tatsinnit)]

Checked this out and ran both scripts against the tree, plus probed the parsers directly. The direction is right — deriving menu paths from package.json instead of hand-copying them is the correct fix, and sharing menu-graph.js between the checker and the generator so they can't disagree is the right structure. A few observations before this goes in.

1. menu-paths doesn't fire on the prose the PR exists to fix

RIGHT_CLICK requires a literal > between "right-click" and the bold segment. Against the current book that matches 0 breadcrumbs across 47 right-click mentions. The docs write navigation as prose, not breadcrumbs.

The clearest illustration is in the very file this PR uses as its example, docs/book/src/features/retina-capture.md:

 5: Right click on your AKS cluster and select **Troubleshoot Network Health**
    and then click on **Run Retina Capture** ...            <- not reported
11: #### Step 1: ... **Troubleshoot Network Health**
    > **Run Retina Capture** > **Download Artifacts Locally**   <- reported
33: #### Step 1: ... same shape                                 <- reported

All three describe the same stale path (the default menu nests it under Troubleshoot & Diagnose). Lines 11 and 33 are caught, line 5 is not — same file, six lines apart, purely because it says "and then click on" instead of >.

Resolving the other prose instructions against the graph this PR builds turns up more that go undetected:

Prose	Actual default path
features/tcp-dumps.md:5 — Troubleshoot Network Health ... Collect TCP Dumps	classic only; default is Troubleshoot & Diagnose > Troubleshoot Network Health > Collect TCP Dumps
features/kaito-install-deploy.md:7,17 — Deploy an LLM with KAITO ... Install KAITO	Develop & Deploy > Deploy a LLM with KAITO > Install KAITO (title differs too — "a" not "an")
features/inspektor-gadget.md:11 — Show Inspektor Gadget ... Gadget Commands	not present in either menu mode
features/aks-diagnostics.md:5 — Run AKS Diagnostics	classic only
features/aks-compare-cluster.md:5 — "right click on your AKS cluster" ... Compare AKS Cluster	lives on the subscription node

These are the simplifiedMenuStructure regressions the PR description is about. Two ways to close it, either works: teach crumbsIn to treat "and then select / and choose / and click on" as separators, or normalise the prose to > in #2372 and note that here.

2. The checks dropped in 285c765 aren't covered by anything yet

The commit removes links / images / anchors / summary on the grounds that lychee and mdbook already do it. The reasoning is sound, but neither gate exists on PRs today:

• lychee appears nowhere in the repo — not in .github/workflows/, not in docs/book/Makefile.
• create-missing = false is in book.toml on main (thanks to docs: fix broken links, restore missing Development page, drop duplicate copies #2370), but website.yaml triggers only on push to main, so it never runs on a PR.

Net effect is a window where link/image/anchor checking on PRs is covered by nothing. Landing the lychee workflow alongside this would close it.

3. docs:reference:check fails on a clean checkout

The generated pages aren't committed, so on a fresh clone:

$ node scripts/generate-docs-reference.js --check
[stale] docs/book/src/reference/commands.md
[stale] docs/book/src/reference/settings.md
[stale] docs/book/src/reference/pinned-versions.md
exit 1

They're also not in SUMMARY.md, so mdbook won't publish them. Assuming that's #2372 — worth stating in the description, since as-is the script ships unable to pass.

4. The submenu cycle guard doesn't guard

scripts/lib/menu-graph.js:

const key = `${bucket}|${breadcrumb.join(">")}`;
if (active.has(key)) return; // submenu cycle guard

breadcrumb grows on every hop, so the key is fresh each recursion and the guard never hits. Reproduced with an A→B→A submenu pair: unbounded recursion to stack overflow rather than a clean stop. Keying on bucket alone fixes it.

5. false / never in a view menu reads as reachable

evaluateWhen skips any conjunct matching no CONJUNCT_RULES row:

evaluateWhen("viewItem =~ /aks\.cluster/i && never", true)
  => [{ node: 'cluster', flags: [] }]

buildPaletteVisibility handles false / never for the palette, but the menu walk doesn't — so a command hidden that way would be documented as present in the menu, and a breadcrumb pointing at it would validate.

6. NODE_PATTERNS asymmetry, and a dead diagnostic

The cluster row requires =~ /aks\.cluster; the subscription row also accepts == via [=~]+. So:

nodeOf("view == kubernetes.cloudExplorer && viewItem == aks.cluster") => 'any'

which renders as Any > ... with nothing flagged. Relatedly, the orphanNode check in generate-docs-reference.js looks for paths starting with "Other", but nodeOf only ever returns cluster | subscription | fleet | k8s-cluster | azure | any — so NODE_LABEL.other and that diagnostic are both unreachable. Pointing it at "Any" would make it live.

7. Nothing in CI looks at this code

eslint.config.mjs ignores **/*.js, and format-check.yml globs ts,tsx,json,css,md and only triggers on PRs touching ts/tsx. All checks are green here, but none of them read the new 963 lines. Prettier passes today (I ran it), just not by any gate.

Given splitTop, stripOuterParens, normaliseBoolean and evaluateWhen are the load-bearing parts, a few unit tests would be worth it — items 4, 5 and 6 above are each about five lines of test away from being caught automatically.

Smaller things

• process.exit() right after console.log (docs-check.js, generate-docs-reference.js) can truncate piped stdout in CI. process.exitCode is safer.
• contributions().version is computed and never read. const pkg = readJson("package.json") in generate-docs-reference.js is unused — menu-graph already exports pkg.
• CLASSIC_MARKER is the only escape hatch and is documented only in the source. There's no general "this bold chain isn't a menu path" opt-out, so a future page describing portal navigation as **Settings** > **Storage** will false-positive, since Storage is a real command title.
• identifiers doesn't skip fenced code blocks. Clean today, but a settings.json example naming another extension's azure.* setting becomes a hard error.
• Branch is 9 commits behind main (2.5.0, docs: fix broken links, restore missing Development page, drop duplicate copies #2370, Publish release 2.5.0 #2388, Re-enable release pipeline publishing #2389) — worth a rebase.

What's good

The incidental findings the generator prints are real and worth filing on their own:

dead submenus (declared, never referenced): aks.tcpDataCollectionSubMenu
duplicate palette title "Create KAITO Workspace": aks.aksKaitoGenerateYaml, aks.aksKaitoCreateCRD
title repeats its category, palette shows "AKS: AKS: Launch Kickstart Agent": aks.kickstart.launchExperience
title repeats its category, palette shows "AKS: AKS: Configure Kickstart Cluster": aks.kickstartCluster
version skew actions/checkout: v3 vs v7.0.0
version skew azure/login: v1.4.6 vs v3.0.0
version skew Azure/k8s-deploy: v4 vs v6.0.0
version skew azure/aks-set-context: v3 vs v5.0.0
version skew azure/use-kubelogin: v1 vs v1.3

normaliseBoolean collapsing == true / != false is a genuine fix, and the commit messages explaining the reversal in 285c765 are unusually clear.

Items 1–3 are the ones I'd want resolved before merge; the rest are small.

[Tatsat (Tats) Mishra 🐉 (Tatsinnit)]

Line-level notes to go with the summary above. Nothing new versus that comment — just pinned to the relevant lines so they're easier to action. Items on RIGHT_CLICK, the cycle guard, !rule, and NODE_PATTERNS are the substantive ones; the rest are cleanup.

[Copilot]

Pull request overview

Adds documentation tooling to mechanically validate docs against package.json and to generate “reference” pages (commands, settings, pinned versions) so the docs can’t silently drift from the extension’s contributed surface area.

Changes:

• Introduces scripts/docs-check.js with checks for identifiers, menu breadcrumb paths, documentation coverage, and orphaned images.
• Adds scripts/generate-docs-reference.js to generate (and --check) reference markdown pages under docs/book/src/reference/.
• Adds a shared scripts/lib/menu-graph.js that derives the cluster context menu structure from package.json, used by both scripts to ensure consistent interpretation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
scripts/lib/menu-graph.js	Builds a menu graph from package.json contributions and evaluates when clauses across menu modes.
scripts/generate-docs-reference.js	Generates (and checks for drift in) reference docs pages: commands, settings, pinned versions.
scripts/docs-check.js	Implements docs validations that require package.json knowledge (IDs, menu paths, coverage, orphans).
package.json	Adds docs:check, docs:reference, and docs:reference:check npm scripts.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Suneha Bose (bosesuneha)]

Will be adding a PR gate for lychee link checks in subsequent PR.