Skip to content

feat(cli): opt-in discovery of an author-shipped baseline (#278)#286

Open
rodboev wants to merge 1 commit into
NVIDIA:mainfrom
rodboev:pr/shipped-baseline-optin
Open

feat(cli): opt-in discovery of an author-shipped baseline (#278)#286
rodboev wants to merge 1 commit into
NVIDIA:mainfrom
rodboev:pr/shipped-baseline-optin

Conversation

@rodboev

@rodboev rodboev commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Summary

A skill distributed with a vetted .skillspector-baseline.yaml gets no benefit from it today: skillspector scan <skill> ignores the file unless every consumer finds it and passes --baseline by hand, so consumers of distributed skills see the author's known false positives raw. Loading it automatically would be worse: a baseline shipped by the skill author is third-party configuration from the party being audited, and applying it silently would let it suppress findings, lower the risk score, and flip the non-zero-exit install gate in a consumer's scan.

This makes discovery explicit-trust: scan detects a co-located baseline and reports it, applies it only when the consumer opts in with --use-shipped-baseline, and prints provenance when it does.

Closes #278

Behavior

  • No flags: a .skillspector-baseline.yaml at the top level of the scanned directory produces a stderr notice naming the file and the opt-in flag. Findings, risk score, and exit code are unchanged, and the file is never parsed; discovery is a pure existence check.
  • --use-shipped-baseline: the discovered baseline goes through the same loading and validation as --baseline and is applied; a stderr provenance line records that an author-shipped baseline at that path was applied. --show-suppressed lists what it suppressed and why through the existing rendering. A malformed shipped baseline exits 2, same as an explicit one; it is never silently skipped.
  • --baseline PATH always wins: discovery is skipped entirely when an explicit baseline is given, including the missing-file exit-2 path.
  • All notices go to stderr, so --format json and --format sarif stdout stays machine-parseable.

Discovery is limited to the canonical .skillspector-baseline.yaml, the name skillspector baseline writes by default. .yml and .json baselines stay usable through explicit --baseline; keeping auto-discovery to one filename keeps the author-controlled trust surface small and avoids any precedence question.

Diff Notes

  • src/skillspector/suppression.py: new discover_baseline() that returns the co-located .skillspector-baseline.yaml path or None without reading the file. Top level of the scanned directory only.
  • src/skillspector/cli.py: --use-shipped-baseline flag on scan, a discovery gate in the single-directory path, and a stderr console for the detection notice, provenance line, and a note when opt-in is given but nothing is shipped. The discovered path flows through the existing baseline plumbing, so suppression semantics and report rendering are unchanged.
  • tests/unit/test_suppression.py: discovery unit tests (canonical file, absence, non-directory input, directory-named-like-the-baseline, noncanonical .yml/.json siblings ignored, nested files ignored).
  • tests/unit/test_cli.py: end-to-end tests (all --no-llm) for detect-without-apply, opt-in apply with provenance and suppressed: true entries, explicit --baseline precedence over a shipped suppress-all baseline, JSON and SARIF stdout parseability with a discovered baseline in a directory whose name has a space and a non-ASCII character, no-op opt-in when nothing canonical is shipped, --show-suppressed reason rendering, exit 2 on a malformed shipped baseline under opt-in, and an untouched recursive path.

Scope

Verification

  • uv run pytest tests/unit/test_cli.py tests/unit/test_suppression.py -q47 passed (33 pre-existing + 14 new)
  • uv run ruff check src/ tests/All checks passed.
  • uv run ruff format --check src/ tests/144 files already formatted

Discover a co-located .skillspector-baseline.yaml and apply it only when
the consumer opts in with --use-shipped-baseline, reporting provenance on
stderr. Detection without opt-in leaves findings and the risk score
untouched and never parses the file. Explicit --baseline still wins.

Closes NVIDIA#278

Signed-off-by: Rod Boev <rod.boev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-discover a shipped baseline so distributed skills carry their own suppressions

1 participant