PRDCT-377: backfill frontmatter description across docs

[Iamfle4ka]

What

Backfills a description: into ~253 help/dev pages that had none. The field feeds search snippets, the page lede, <meta> tags, and Ask Kai / RAG retrieval — previously empty site-wide.

Adds the authoring tooling behind it:

• scripts/content-lint.mjs — flags pages missing a description (+ a few terminology/liquid/link checks); npm run lint:content.
• scripts/backfill-descriptions.mjs — generator used for this pass; npm run backfill:desc.

Notes / scope

• transformations/snowflake-plain/index.md is intentionally left out — its description lands with the Snowflake split (PRDCT-372: split snowflake transformation page #988).
• The linter is not yet wired into CI: it currently also reports pre-existing terminology issues, so making the build fail on it needs a separate cleanup pass (the PRDCT-377 acceptance criterion of a CI gate is deferred to that follow-up).
• Content changes are purely additive frontmatter; npm run build is clean (254 pages).

Linear: PRDCT-377

🤖 Generated with Claude Code

[linear]

PRDCT-377

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
connection-docs	Ready	Preview	Aug 8, 2026 12:08am

[keboola-pr-reviewer-bot]

Verdict: needs_human (risk 2/5) · profile docs

Several auto-generated descriptions are not fit for public <meta> tags or search snippets and must be fixed before merging.

Concerns:

• src/content/docs/components/extractors/communication/email-imap/query.md: Typo 'matchess' (double-s) in description — will ship to public tag.
• src/content/docs/components/extractors/database/mongodb/mapping.md: Description is raw HTML table fragment, not a readable sentence for search snippets.
• src/content/docs/components/extractors/other/telemetry-data/telemetry-data.md: Description contains internal template file references scraped from an HTML comment.

Suggested reviewers: keboola/docs

[jordanrburger]

Review summary

Verdict: changes requested.

Structurally this is exactly what it claims — a clean, purely-additive backfill. All 253 content files add exactly one description: line directly after title: inside the frontmatter fence; no titles/slugs/other keys are touched, and the sole -1 deletion is a benign trailing-comma edit in package.json to append the two new npm scripts (lint:content, backfill:desc). YAML is valid throughout.

The problem is that the generator scraped page-body content for a handful of pages, producing descriptions that are unfit for the public <meta> tags / search snippets / RAG they feed.

Blocking

• src/content/docs/components/extractors/database/mongodb/mapping.md — description is a raw HTML table fragment (extractors <table> <tr><th>id</th>…</table>). Garbage as a meta description.
• src/content/docs/components/extractors/other/telemetry-data/telemetry-data.md — scraped from inside an HTML comment: ends with --> and exposes internal template paths (data/telemetrytables.yml, includes/telemetry-table.html).
• src/content/docs/transformations/dbt/cli/cli.md — description is a raw YouTube <iframe …> embed. Valid YAML (escaped quotes) but garbage content.
• src/content/docs/components/extractors/communication/email-imap/query.md — typo "matchess" (double-s) in a mid-sentence fragment; will ship to the public <meta> tag.
• PR state: mergeable: CONFLICTING — needs a rebase on main before it can merge regardless of content fixes.

Root cause / suggested fix

These all trace to backfill-descriptions.mjs grabbing the first body text without stripping HTML / comments / embeds. Recommend fixing the generator (strip HTML tags, <!-- --> comments, and iframe/embed blocks; prefer a real prose sentence) and re-running it on the affected pages, rather than hand-patching — since the same generator is likely to be reused.

Nits (non-blocking)

• ~10 descriptions are body-scrape truncations ending in "…", a few cut mid-word (e.g. tutorial/manipulate/index.md, components/writers/index.md, kai/index.md).
• management/project/export/index.md ends mid-sentence with no ellipsis ("…This can come in handy when").
• A few end with a dangling colon where the source introduced a list (transformations/dbt/cloud/cloud.md, transformations/dbt/index.md, transformations/duckdb/index.md).

Scope note

This was a sampling review: I mechanically verified all 253 additions for structure (1 line each, correct frontmatter placement) and ran full-corpus scans for YAML hazards, HTML/comment/iframe artifacts, typos, and >170-char lengths — so the blocking list is exhaustive for those patterns. I did not hand-read all 253 strings for factual accuracy, so subtle inaccuracies in otherwise-clean sentences could remain. The bulk of descriptions look accurate and well-formed.

Automated review pass (Claude Code), reviewed by a human maintainer.

[Iamfle4ka]

Re-done per the review — generator hardened, everything regenerated, no hand-patching of scraped values.

What changed in backfill-descriptions.mjs:

• HTML comments are stripped before parsing (they can span lines) — kills the telemetry-data leak of template paths at the source.
• A non-prose line (table, <iframe>, list, heading) now terminates the paragraph instead of being absorbed as continuation — the root cause of the MongoDB table and dbt iframe descriptions.
• Multi-line HTML blocks are skipped via tag-depth tracking (blank lines inside <table> no longer fool the scanner).
• Unusable paragraphs (labels like "Video:", Note: asides, "As you can see" deixis) fall through to the next prose paragraph.
• New finish() pass: list-introducing colon → period, mid-sentence cut → retreat to the previous sentence or explicit … (fixes the dangling-colon and "handy when" nits); sentence-boundary truncation preferred more aggressively.
• CRLF-safe insertion (15 files on main are CRLF; the anchor line keeps its \r) and backslash-safe YAML escaping.

Instead of rebasing, the branch was rebuilt on current main and the generator re-run — that sidesteps all 17 conflicts and every description now reflects today's content ("matchess" was already fixed on main, so that one healed itself). 250 pages generated + 4 reference pages hand-filled via the script's documented fallback (no usable intro prose: MongoDB mapping examples, Bing Ads report presets, Adform metrics/dimensions). 258/258 pages now carry a description.

Verified: purely additive — every content change is exactly +1 description line (the sole -1 is the same package.json trailing comma); full-corpus scan clean for your review's patterns (HTML tags, -->, template paths, trailing ,;:, mid-sentence endings, >170 chars — 0 hits); npm run build clean (258 pages); lint:content --rule=description 0 issues; second generator run = 0 changes (idempotent).

[Iamfle4ka]

@jordanrburger — regenerated rather than hand-patched, as you asked. Also re-cut off current main, which clears the CONFLICTING state (the old branch was 99 commits behind and 11 of its files had been deleted from main in the data-apps restructure).

Generator fixes

Both blocking-class defects traced to the same thing: a sentence whose object is the following list got cut at the list marker.

1. A paragraph that has not closed its sentence now absorbs the list items instead of ending on a dangling preposition — across a blank line too, when a list is what follows.
2. A trailing colon counts as sentence-final, so a colon-introduced enumeration is not swallowed into a run-on.

Your four blockers

page	now
mongodb/mapping.md	skipped — no usable prose, so no description at all rather than a <table> fragment
telemetry-data.md	The Telemetry Data connector allows you to retrieve data about your project or your entire organization.
dbt/cli/cli.md	Let's set up the local development with Keboola CLI. — no iframe
email-imap/query.md	matchess typo gone

Corpus scan

279 pages, each exactly +1 line, zero deletions — same purely-additive shape you verified before. Across all written descriptions: 0 containing HTML / comments / embeds / table fragments, 0 ending in a colon, 0 dangling fragments. 38 end in … — every one of them the 160-char meta-description clip on a word boundary, none mid-word.

Two pages are deliberately left without a description (pure reference tables, no prose to draw from): components/extractors/database/mongodb/mapping.md and components/extractors/marketing-sales/bing-ads/report-presets-columns-and-pk/index.md.

npm run build clean, 306 pages.