Skip to content

feat: audio overlay and suffix tool-risk injection probes - #11

Open
codingshreyash wants to merge 4 commits into
pr/audio-2-probesfrom
pr/audio-3-injection
Open

feat: audio overlay and suffix tool-risk injection probes#11
codingshreyash wants to merge 4 commits into
pr/audio-2-probesfrom
pr/audio-3-injection

Conversation

@codingshreyash

@codingshreyash codingshreyash commented Jul 26, 2026

Copy link
Copy Markdown
Owner

What this changes

Adds two audio injection probes on top of the probe and detector infrastructure in #10.

AudioOverlayInjection mixes a spoken tool-risk instruction underneath a benign carrier utterance at reduced gain. It follows the overlay threat model from AudioJailbreak (arXiv:2505.14103) and sweeps the payload's gain and position.

AudioSuffixInjection appends a spoken tool-risk instruction after a benign carrier utterance. Optional speed and noise transforms are applied only to the suffix, leaving the carrier unchanged. This models an attacker who can control the appended clip but not the original audio.

Both probes support foreground_only and background_only. They also run the intelligibility check before scoring, so an unintelligible composite is not counted as a successful result.

Why

A plain spoken-request probe tests how the target responds to a directly audible instruction. These probes cover two additional audio-specific cases: placing an instruction underneath a benign utterance, and appending it as a separate audio segment.

Those cases are not represented by text-only probes.

Verification

Confirmed by hand that both conditions produce the expected composite audio and that the intelligibility gate drops composites the ASR check cannot transcribe. A live target run is not run in CI:

garak -t <target_type> -n <model_name> -p audio_overlay,audio_suffix

@codingshreyash codingshreyash changed the title Audio 3/3 — injection: hidden-overlay and spoken-suffix tool-risk feat: audio overlay and suffix tool-risk injection probes Jul 27, 2026
@codingshreyash
codingshreyash force-pushed the pr/audio-2-probes branch 2 times, most recently from 08cbd8c to da5a856 Compare July 27, 2026 14:52
…l-risk

Stacks the injection family on the probe/detector core:
- probes/audio_overlay: malicious tool-risk instruction mixed under a benign
  carrier at reduced gain (AudioJailbreak overlay threat)
- probes/audio_suffix: spoken tool-risk instruction appended after a benign
  carrier

Third of a 3-PR stack.

Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
Share validated carrier and payload records between suffix and overlay probes through user-overridable JSON. Preserve advanced sweeps as configurable class options while keeping them out of the normal default surface.

Assisted-by: OpenAI Codex
Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
Bring the probe data-path revision into the injection stack and apply the same garak.data.path resolution to suffix and overlay sources.

Assisted-by: OpenAI Codex

Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant