feat: audio overlay and suffix tool-risk injection probes - #11
Open
codingshreyash wants to merge 4 commits into
Open
feat: audio overlay and suffix tool-risk injection probes#11codingshreyash wants to merge 4 commits into
codingshreyash wants to merge 4 commits into
Conversation
This was referenced Jul 26, 2026
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
from
July 27, 2026 00:15
81f5f3f to
f38d8b8
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 00:15
6088c18 to
4a3b2c6
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 01:08
4a3b2c6 to
02f71f8
Compare
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
from
July 27, 2026 01:08
f38d8b8 to
aaf7371
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 02:21
02f71f8 to
c950a63
Compare
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
from
July 27, 2026 02:21
aaf7371 to
5c6d7a5
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 04:28
c950a63 to
55395e5
Compare
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
2 times, most recently
from
July 27, 2026 14:52
08cbd8c to
da5a856
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 14:52
55395e5 to
60131f2
Compare
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
from
July 27, 2026 17:18
da5a856 to
6f197e8
Compare
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 17:18
60131f2 to
5655e2b
Compare
…l-risk Stacks the injection family on the probe/detector core: - probes/audio_overlay: malicious tool-risk instruction mixed under a benign carrier at reduced gain (AudioJailbreak overlay threat) - probes/audio_suffix: spoken tool-risk instruction appended after a benign carrier Third of a 3-PR stack. Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
codingshreyash
force-pushed
the
pr/audio-3-injection
branch
from
July 27, 2026 17:39
5655e2b to
fbef410
Compare
codingshreyash
force-pushed
the
pr/audio-2-probes
branch
from
July 27, 2026 17:39
6f197e8 to
af07df1
Compare
Share validated carrier and payload records between suffix and overlay probes through user-overridable JSON. Preserve advanced sweeps as configurable class options while keeping them out of the normal default surface. Assisted-by: OpenAI Codex Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
Bring the probe data-path revision into the injection stack and apply the same garak.data.path resolution to suffix and overlay sources. Assisted-by: OpenAI Codex Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Adds two audio injection probes on top of the probe and detector infrastructure in #10.
AudioOverlayInjectionmixes a spoken tool-risk instruction underneath a benign carrier utterance at reduced gain. It follows the overlay threat model from AudioJailbreak (arXiv:2505.14103) and sweeps the payload's gain and position.AudioSuffixInjectionappends a spoken tool-risk instruction after a benign carrier utterance. Optional speed and noise transforms are applied only to the suffix, leaving the carrier unchanged. This models an attacker who can control the appended clip but not the original audio.Both probes support
foreground_onlyandbackground_only. They also run the intelligibility check before scoring, so an unintelligible composite is not counted as a successful result.Why
A plain spoken-request probe tests how the target responds to a directly audible instruction. These probes cover two additional audio-specific cases: placing an instruction underneath a benign utterance, and appending it as a separate audio segment.
Those cases are not represented by text-only probes.
Verification
Confirmed by hand that both conditions produce the expected composite audio and that the intelligibility gate drops composites the ASR check cannot transcribe. A live target run is not run in CI: