feat: narrow probe selection by intent: in run.spec

docs/source/cas.rst

@@ -136,9 +136,22 @@ A few things to note:
 * ``intent:S003productkey`` selects that single leaf behaviour. A *category* code such
   as ``intent:S003`` ("Illegal") instead expands to all of its leaves (``S003illegal``, ``S003instructions``,
   ``S003goods``, ``S003services``, ``S003productkeys``).
+* An explicit ``intent:`` also **filters the ordinary probe selection** by
+  typology descendancy: ``garak --spec "probes.*,intent:S005hate"`` runs only the
+  probes carrying an intent beneath ``S005hate`` (plus any ``IntentProbe``, unless
+  its ``blocked_intent_spec`` covers every included code). The injected default
+  scope never filters.
 * If you give no ``intent:`` selector, the default scope ``S`` (the whole Safety
   branch) is injected at resolve time.
 
+.. note::
+
+   The intent filter reads each probe's *class* intent from the plugin cache. An
+   intent that a probe only carries per payload group (see
+   :doc:`probes/encoding` and the payloads mechanism) is not visible to the
+   filter, so filtering by such an intent will not select that probe. This is
+   inherent to filtering before probes are instantiated.
+
 Each intent carries a short imperative *stub* in the typology, which the
 technique expands into prompts. ``GrandmaIntent`` wraps each stub in a roleplay
 template, producing prompts such as:

docs/source/configurable.rst

@@ -183,18 +183,34 @@ Selectors (a category prefix is mandatory):
 * ``tier:<N|name>`` - filters probes by tier; **inclusive** ("log level"): ``tier:N``
   admits tiers ``1..N`` (``tier:1`` is the most critical). Names work too
   (``tier:of_concern`` == ``tier:1``).
-* ``intent:<code>`` - selects intent typology codes for intent-based probes
-  (e.g. ``intent:S`` for the whole Safety branch, ``intent:S001`` for a category,
-  ``intent:S001mis`` for a leaf); ``intent:*`` or ``intent:all`` selects every
-  intent. This is a **separate axis** consumed by the
-  intent service: it does **not** add or remove probes. When no ``intent:`` is
-  given, the default scope ``S`` (the Safety branch) is injected at resolve
-  time. Typology
-  expansion and detectorless filtering are governed by the ``run.*`` intent
-  modifiers (``run.serve_detectorless_intents``).
-  Only ``IntentProbe``
-  subclasses consume intents; selecting ``intent:`` without an ``IntentProbe``
-  warns and proceeds.
+* ``intent:<code>`` - filters probes by intent typology code, and scopes the
+  intents a selected ``IntentProbe`` exercises. Matching is by typology
+  **descendancy**, not string prefix: a branch code keeps every probe whose
+  intent lies beneath it, while a leaf keeps only probes declaring that leaf
+  (``intent:S`` keeps the whole Safety branch, ``intent:S005`` a category, and
+  ``intent:S005hate`` only the ``S005hate`` leaf -- not a probe declaring the
+  parent ``S005``, nor the sibling leaf ``S005bully``). Two exemptions: the
+  injected default scope is never a filter, and ``IntentProbe`` subclasses are
+  pruned only when their ``blocked_intent_spec`` covers every included code --
+  otherwise which intents they serve is settled later by the intent service.
+  ``intent:*`` or ``intent:all`` selects every intent and does not
+  filter. When no ``intent:`` is given, the default scope ``S`` (the Safety
+  branch) is injected at resolve time and, being the default, never prunes.
+  A lone ``-intent:<code>`` (no ``intent:`` include) also filters: it removes
+  any already-selected probe whose own declared intent descends from
+  ``<code>``, compared against the resolved candidate set rather than the
+  injected default scope; ``IntentProbe`` subclasses are unaffected, since
+  they declare no fixed intent of their own.
+  Both directions test the same descendancy, so they are complementary set
+  operations on the candidate: ``intent:<code>`` is an intersection,
+  ``-intent:<code>`` alone is a difference. When every candidate probe already
+  descends from ``<code>``, the include is a no-op and the exclude empties
+  the selection.
+  Typology expansion and detectorless filtering are governed by the ``run.*``
+  intent modifiers (``run.serve_detectorless_intents``). Only ``IntentProbe``
+  subclasses derive prompts from intents; giving an explicit ``intent:`` with no
+  ``IntentProbe`` in the selection warns and proceeds -- ordinary probes that
+  match the intent still run, but no intent-derived prompts are generated.
 
 Polarity: a bare selector (or ``+``) includes; a leading ``-`` removes. Note
 the asymmetry of ``tier``: ``tier:N`` is the inclusive filter, while ``-tier:N``
@@ -221,8 +237,10 @@ wildcard, so quote those specs (or use the ``all`` alias instead).
     garak --spec probes.all,probes.fitd.FITD
     # tiers {1,3}: tier:3 admits 1..3, then -tier:2 removes exactly tier 2
     garak --spec "+probes.*,+tier:3,-tier:2"
-    # an intent probe over one intent category (intents are a separate axis)
+    # an intent probe over one intent category (intents scope the IntentProbe)
     garak --spec probes.grandma.GrandmaIntent,intent:S004
+    # intent as a filter: only probes carrying the S005hate intent (plus IntentProbes)
+    garak --spec "probes.*,intent:S005hate"
 
 .. code-block:: yaml
 

docs/source/extending.probe.rst

@@ -269,7 +269,9 @@ Two class attributes tune which intents the probe consumes:
 * ``skip_root_intents`` (default ``True``) -- skip single-letter root codes when
   gathering stubs, since a whole branch rarely has a meaningful prototypical stub.
 * ``blocked_intent_spec`` (default ``""``) -- intents this technique should never
-  exercise, even when in scope.
+  exercise, even when in scope. If it covers every intent an explicit ``intent:``
+  include asked for, ``run.spec`` resolution drops the probe from the selection
+  entirely, since it would have nothing left to serve.
 
 If the active intent set is empty (for example the ``intent:`` axis was filtered
 to nothing), the probe is a graceful no-op: it sends no prompts and the run

garak/_selection.py

@@ -17,6 +17,7 @@
 
 from garak import _plugins
 from garak import _spec
+from garak.cas import get_parent_name
 
 # Tier assigned to probes that do not declare one (Tier.UNLISTED).
 _DEFAULT_TIER = 9
@@ -78,6 +79,43 @@ def _tier_of(name: str) -> int:
     return int(_plugins.plugin_info(name).get("tier", _DEFAULT_TIER))
 
 
+def _intent_under(code: str, codes: List[str]) -> bool:
+    """True if ``code`` or a typology ancestor is in ``codes``; malformed codes never match."""
+    current: str = code
+    while current:
+        if current in codes:
+            return True
+        try:
+            current = get_parent_name(current)
+        except ValueError:
+            return False
+    return False
+
+
+def _intent_keeps(name: str, includes: List[str], excludes: List[str]) -> bool:
+    """True if ``name`` survives the intent filter. A probe declaring no intent
+    (an IntentProbe, by convention) is kept unless its ``blocked_intent_spec``
+    covers every included code, in which case it has nothing left to serve."""
+    info = _plugins.plugin_info(name)
+    code = info.get("intent")
+    if code is None:
+        blocked_spec = info.get("blocked_intent_spec", "")
+        blocked = [c.strip() for c in blocked_spec.split(",") if c.strip()]
+        return not (blocked and all(_intent_under(inc, blocked) for inc in includes))
+    return _intent_under(code, includes) and not _intent_under(code, excludes)
+
+
+def _intent_excluded(name: str, excludes: List[str]) -> bool:
+    """True if ``name`` declares its own intent and it descends from an
+    excluded code. Used for exclude-only specs (no explicit ``intent:``
+    include): pruning here compares each probe's own declared intent against
+    the excludes, so it never depends on the injected default scope. An
+    ``IntentProbe`` (no declared intent) is never excluded here; which
+    intents it actually serves is decided later by IntentService."""
+    code = _plugins.plugin_info(name).get("intent")
+    return code is not None and _intent_under(code, excludes)
+
+
 def _empty_reason(spec: _spec.Spec) -> str:
     """Best-effort explanation of why a spec resolved to no probes."""
     tier_ceilings = [int(s.value) for s in spec.include if s.kind == "tier"]
@@ -95,6 +133,28 @@ def _empty_reason(spec: _spec.Spec) -> str:
             f"probe '{name}' is tier {_tier_of(name)} but the spec restricts to "
             f"tiers 1..{ceiling}; widen the tier filter or drop the explicit probe"
         )
+    intent_codes = [
+        s.value
+        for s in spec.include
+        if s.kind == "intent" and s.value.lower() not in ("*", "all")
+    ]
+    if intent_codes:
+        codes = ", ".join(intent_codes)
+        return (
+            f"no selected probe carries intent '{codes}'; widen the probe "
+            f"selection or drop the intent selector"
+        )
+    excluded_intent_codes = [
+        s.value
+        for s in spec.exclude
+        if s.kind == "intent" and s.value.lower() not in ("*", "all")
+    ]
+    if excluded_intent_codes:
+        codes = ", ".join(excluded_intent_codes)
+        return (
+            f"every selected probe's intent falls under the excluded code(s) "
+            f"'{codes}'; narrow the exclusion or widen the probe selection"
+        )
     if any(s.kind in ("tag", "tier") for s in spec.include):
         return "no active probe matches the given tier/tag filters; widen the filters"
     return "every included probe was removed by an exclusion; adjust includes/excludes"
@@ -104,7 +164,13 @@ def resolve_spec(spec: _spec.Spec, skip_unknown: bool = False) -> _spec.Resoluti
     """Resolve a :class:`garak._spec.Spec` to concrete probe and buff names.
 
     Selection happens against the live plugin registry (active state, tiers,
-    tags). This is the single entry point used by the CLI and harnesses.
+    tags). An explicit ``intent:`` include additionally filters probes by
+    typology descendancy; the injected default scope never filters, and
+    ``IntentProbe`` subclasses are pruned only when their
+    ``blocked_intent_spec`` covers every included code. A lone ``-intent:``
+    (no include) also prunes: probes whose own declared intent descends from
+    an excluded code drop out of the already-resolved candidate set. This is
+    the single entry point used by the CLI and harnesses.
     """
     rejected: List[str] = []
     inactive_modules: List[str] = []
@@ -137,6 +203,42 @@ def resolve_spec(spec: _spec.Spec, skip_unknown: bool = False) -> _spec.Resoluti
     if tag_prefixes:
         candidate = {p for p in candidate if _has_any_tag(p, tag_prefixes)}
 
+    # Intent filter, mirroring the tag filter's OR-of-prefixes shape but matching
+    # by typology descendancy. An explicit intent: include filters the candidate
+    # set by descendancy (injected DEFAULT_INTENT_SCOPE never prunes);
+    # intent:* / intent:all are vacuous and do not filter. A probe that declares
+    # no intent (an IntentProbe, by convention) is pruned only when its
+    # blocked_intent_spec covers every included code; otherwise which intents it
+    # actually serves is decided later by IntentService. A lone -intent:
+    # (no include) also prunes: any already-selected probe whose own declared
+    # intent descends from an excluded code drops out, compared against the
+    # resolved candidate set rather than the injected default scope.
+    intent_includes = [s.value for s in spec.include if s.kind == "intent"]
+    intent_excludes = [s.value for s in spec.exclude if s.kind == "intent"]
+    # Malformed codes must not drive pruning: they stay in ``rejected`` (raised
+    # below unless ``skip_unknown``), so a bad code never silently narrows the
+    # preview to IntentProbe-only under ``--list_probes``.
+    intent_filter = [
+        c
+        for c in intent_includes
+        if c.lower() not in ("*", "all") and _spec.validate_intent_specifier(c)
+    ]
+    intent_exclude_filter = [
+        c
+        for c in intent_excludes
+        if c.lower() not in ("*", "all") and _spec.validate_intent_specifier(c)
+    ]
+    if intent_filter:
+        candidate = {
+            p
+            for p in candidate
+            if _intent_keeps(p, intent_filter, intent_exclude_filter)
+        }
+    elif intent_exclude_filter:
+        candidate = {
+            p for p in candidate if not _intent_excluded(p, intent_exclude_filter)
+        }
+
     # Buffs: union of buffs.* includes (no implicit default)
     buff_includes = [
         s for s in spec.include if s.kind == "plugin_path" and s.category == "buffs"
@@ -172,8 +274,6 @@ def resolve_spec(spec: _spec.Spec, skip_unknown: bool = False) -> _spec.Resoluti
     # typology membership + expansion + detectorless filtering happen later in
     # IntentService. When no intent: selector is given, inject the default scope
     # (_spec.DEFAULT_INTENT_SCOPE) so the intent scope survives a run.spec override.
-    intent_includes = [s.value for s in spec.include if s.kind == "intent"]
-    intent_excludes = [s.value for s in spec.exclude if s.kind == "intent"]
     for code in intent_includes + intent_excludes:
         # ``*`` / ``all`` select every intent (IntentService expands the vacuous
         # sentinel); other codes must match the typology specifier format.

garak/cli.py

@@ -550,9 +550,11 @@ def worker_count_validation(workers):
 
             selected_probes = None
             if _config.run.spec:
-                selected_probes = _selection.resolve_spec(
+                resolved = _selection.resolve_spec(
                     parse_spec_file(_config.run.spec), skip_unknown=True
-                ).probes
+                )
+                command.warn_rejected_selectors(resolved.rejected, "probes")
+                selected_probes = resolved.probes
             command.print_probes(selected_probes, verbose=_config.system.verbose)
 
         elif args.list_detectors:
@@ -570,9 +572,11 @@ def worker_count_validation(workers):
 
             selected_buffs = None
             if _config.run.spec:
-                selected_buffs = _selection.resolve_spec(
+                resolved = _selection.resolve_spec(
                     parse_spec_file(_config.run.spec), skip_unknown=True
-                ).buffs
+                )
+                command.warn_rejected_selectors(resolved.rejected, "buffs")
+                selected_buffs = resolved.buffs
             command.print_buffs(selected_buffs)
 
         elif args.list_generators:

garak/command.py

@@ -353,23 +353,43 @@ def _selection_has_intent_probe(probe_names) -> bool:
 
 
 def warn_unconsumed_intents(probe_names) -> None:
-    """Warn once when ``intent:`` was given explicitly but no IntentProbe is in the
-    selection to consume it. The intent axis does not select probes, so without an
-    IntentProbe the intents are never exercised."""
-    from garak import _config
+    """Warn once when an ``intent:`` selector was given explicitly but neither
+    an IntentProbe nor a probe with its own declared ``intent`` is in the
+    selection. A probe with its own ``intent`` (e.g. ``dan.AutoDANCached``)
+    already consumes the axis by being selected for it, even though it
+    derives no stub-based prompts."""
+    from garak import _config, _plugins
 
     if not getattr(_config.transient, "intents_explicit", False):
         return
     if _selection_has_intent_probe(probe_names):
         return
+    if any(
+        _plugins.plugin_info(name).get("intent") is not None
+        for name in probe_names
+        if name.startswith("probes.")
+    ):
+        return
     msg = (
-        "intent: selector(s) given but no IntentProbe is selected; intents will "
-        "not be exercised (select an IntentProbe, e.g. probes.grandma.GrandmaIntent)"
+        "intent: selector(s) given but no IntentProbe is selected, so no "
+        "intent-derived prompts will be generated (add an IntentProbe, "
+        "e.g. probes.grandma.GrandmaIntent)"
     )
     logging.warning(msg)
     print(f"⚠️  {msg}")
 
 
+def warn_rejected_selectors(rejected, namespace: str) -> None:
+    """Warn when ``run.spec`` selectors (e.g. a malformed ``intent:`` code) were
+    rejected and silently dropped from a preview such as ``--list_probes``,
+    mirroring the reporting the run path already does via ``_check_selection``."""
+    if not rejected:
+        return
+    msg = f"unusable {namespace} selector(s), skipped: {', '.join(rejected)}"
+    logging.warning(msg)
+    print(f"⚠️  {msg}")
+
+
 # do a run
 def probewise_run(generator, probe_names, evaluator, buffs):
     import garak.harnesses.probewise

tests/cas/test_intent_run_spec.py

@@ -88,11 +88,20 @@ def test_empty_axis_yields_no_prompts():
 def test_warn_unconsumed_intents_fires_without_intent_probe(capsys):
     import garak.command as command
 
+    # probes.base.Probe declares no intent of its own (unlike e.g. dan.* probes,
+    # which default to T009ignore) -- nothing here engages the intent axis at all.
     garak._config.transient.intents_explicit = True
-    command.warn_unconsumed_intents(["probes.dan.DanInTheWild"])
+    command.warn_unconsumed_intents(["probes.base.Probe"])
+    out = capsys.readouterr().out
     assert (
-        "no IntentProbe is selected" in capsys.readouterr().out
+        "no IntentProbe is selected" in out
     ), "explicit intent: with no IntentProbe in the selection must warn"
+    assert (
+        "no intent-derived prompts will be generated" in out
+    ), "the warning states no intent-derived prompts result without an IntentProbe"
+    # the message must not assert narrowing, since intent:*/intent:all and some
+    # exclude-only specs never prune the probe set
+    assert "narrowed" not in out, "the warning must not claim narrowing"
 
 
 def test_warn_unconsumed_intents_silent_with_mixed_selection(capsys):
@@ -113,3 +122,40 @@ def test_warn_unconsumed_intents_silent_when_default(capsys):
     garak._config.transient.intents_explicit = False
     command.warn_unconsumed_intents(["probes.dan.DanInTheWild"])
     assert capsys.readouterr().out == "", "the injected default (not explicit) must not warn"
+
+
+def test_warn_rejected_selectors_reports_each(capsys):
+    # generic reporter for any run.spec selector dropped under skip_unknown=True
+    # (e.g. a malformed intent: code in a --list_probes/--list_buffs preview).
+    import garak.command as command
+
+    command.warn_rejected_selectors(["intent:zzz", "probes.nonexistent"], "probes")
+    out = capsys.readouterr().out
+    assert "intent:zzz" in out, "each rejected selector must be named in the warning"
+    assert (
+        "probes.nonexistent" in out
+    ), "each rejected selector must be named in the warning"
+
+
+def test_warn_rejected_selectors_silent_when_empty(capsys):
+    import garak.command as command
+
+    command.warn_rejected_selectors([], "buffs")
+    assert capsys.readouterr().out == "", "no rejected selectors must print nothing"
+
+
+def test_probe_pruning_does_not_change_active_intent_set():
+    # the probe-selection filter and the IntentService active set are independent
+    # axes: filtering the probe set must not change which intents become active.
+    from garak._selection import resolve_spec
+    from garak._spec import parse_spec_string
+
+    res = resolve_spec(parse_spec_string("probes.*,intent:S005hate"))
+    assert res.intents == [
+        "S005hate"
+    ], "the intent axis carries the requested code regardless of probe pruning"
+    active = _load("S005hate")
+    assert (
+        "S005hate" in active
+    ), "the active intent set derives from the code, not from surviving probes"
+