Skip to content

detectors: type-validate agent_breaker judge verdicts - #2121

Open
VaggelisGian wants to merge 1 commit into
NVIDIA:mainfrom
VaggelisGian:fix-agent-breaker-verdict-types
Open

detectors: type-validate agent_breaker judge verdicts#2121
VaggelisGian wants to merge 1 commit into
NVIDIA:mainfrom
VaggelisGian:fix-agent-breaker-verdict-types

Conversation

@VaggelisGian

Copy link
Copy Markdown

Fixes #2120

What this change does

AgentBreakerResult._verify_once called .upper() on the judge verdict's success field without checking its type. A judge returning valid JSON with "success": true, false, null, a number, or a list raised AttributeError, which is outside the caught (json.JSONDecodeError, ValueError, TypeError) tuple, so it escaped _verify_once, skipped the verify_attempts retry loop entirely, and propagated through the unprotected probe call site, aborting the whole scan. The same input reaching the detect() fallback instead scored [None] through that path's handler, so identical judge output behaved differently depending on which path ran.

This change type-validates the two scoring-relevant verdict fields inside the existing guarded block:

  • a non-string success raises TypeError naming the offending JSON type;
  • a boolean confidence raises TypeError (float(True) would otherwise silently score 1.0 and inflate ASR);
  • case normalization of YES/PARTIAL is preserved, with a regression test pinning lowercase handling;
  • malformed values now follow the documented contract on every path: warn, consume a retry up to verify_attempts, then score unknown (None) rather than crashing or counting as a silent miss.

Deliberately not changed: the two pre-existing broad except Exception handlers in this module stay as they are. They sit at a plugin boundary where any generator family can raise anything; narrowing them would widen scope beyond this issue. (The issue text mentioned narrowing as part of a planned fix; after review I kept them, for the reason above.)

Why this is not a duplicate

No existing issue or PR covers this value-type crash axis. Issues #2043/#2044 and #2069 hardened the malformed-shape axis of judge parsing and are merged; searches before filing returned nothing overlapping:

  • gh search issues --repo NVIDIA/garak "agent_breaker" --state open
  • gh pr list --repo NVIDIA/garak --state open --search "agent_breaker"
  • gh pr list --repo NVIDIA/garak --state open --search "408" / "retry" / "NIM" (nearest PRs target NIM HTTP retry under a different issue)

Verification

Environment: garak 0.16.1.pre1 main, Python 3.12.6, Windows 11 Pro.

New regression tests fail before the fix, pass after (watched both):

python -m pytest tests/detectors/test_detectors_agent_breaker.py::TestMalformedVerdictValues -q
  before fix -> 8 failed (AttributeError escaping verify(), boolean confidence accepted as 1.0)
  after fix  -> 10 passed

Full affected suites:

python -m pytest tests/detectors/test_detectors_agent_breaker.py tests/probes/test_agent_breaker.py -q
  -> 111 passed in 17.57s
python -m pytest tests/test_docs.py -q
  -> 680 passed in 11.97s
python -m black --config pyproject.toml --check garak/detectors/agent_breaker.py tests/detectors/test_detectors_agent_breaker.py
  -> 2 files would be left unchanged

Crash repro from the issue, run against the fixed code (fixture-style construction, mocked judge):

verify() #1: None                       <- success: true degrades to unknown
verify() #2: (True, 0.9, 'exploited')   <- next valid verdict accepted
generate.call_count: 2                  <- retry consumed properly

Transparency note on the wider suite: tests/detectors/test_detectors.py currently shows 6 failures on this machine (ModuleNotFoundError: Could not import module 'TextClassificationPipeline' via broken torchvision), all in unrelated detector plugins (misleading, mitigation ModernBERTRefusal, unsafe_content). Verified pre-existing by stashing this change and re-running on clean main: same failure. Not touched by this PR.

Checklist notes:

  • Supporting configuration such as generator configuration file: none needed, no params added or changed
  • garak -t <target_type> -n <model_name>: not run end to end; the probe path needs a live judge endpoint and no API key is available here. Coverage instead comes from the detector/probe unit suites above, including the mocked-judge repro.
  • Run the tests: commands and literal results above
  • Verify the thing does what it should: malformed verdict values score unknown after retries on both code paths
  • Verify the thing does not do what it should not: well-formed verdicts (including lowercase yes/partial, numeric-string confidence) behave exactly as before, pinned by tests
  • Documentation: no new plugin class or module; behavior matches the existing docstring contract ("unknown, not safe")

Statement on AI assistance

AI assistance was used for this contribution: an AI coding agent drafted the change and tests under my direction; I reviewed every changed line, ran all quoted commands myself, and submit this PR as its author. It addresses #2120, which was filed with a minimal reproducer, and is not a duplicate per the section above.

A judge response containing valid JSON whose success value is not a
string (true/false/null/number/list) raised AttributeError in
_verify_once, outside the caught exception tuple. The error escaped the
verify_attempts retry loop and propagated through the unprotected probe
call site, aborting the whole scan. The detect() fallback masked the
same input as an unknown score through its own handler, so the two
paths disagreed on identical input.

The verdict fields are now type-checked before use: a non-string
success or a boolean confidence raises TypeError inside the guarded
block, which the existing handler converts into the documented
unknown-after-retry behavior on every path. Case normalization of
YES/PARTIAL verdicts is preserved and pinned by test. Boolean
confidence is rejected because float(True) would silently score 1.0
and inflate ASR.

Test Plan:
  python -m pytest tests/detectors/test_detectors_agent_breaker.py -q
    -> 51 passed
  python -m pytest tests/probes/test_agent_breaker.py -q
    -> 60 passed
  python -m pytest tests/test_docs.py -q
    -> 680 passed
  python -m black --config pyproject.toml --check     garak/detectors/agent_breaker.py tests/detectors/test_detectors_agent_breaker.py
    -> 2 files would be left unchanged

Signed-off-by: Vaggelis <baggelis100@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent_breaker detector: non-string success value in judge JSON crashes the whole scan

1 participant