feat: audio synthesis, transforms, and NVVoiceChat target for audio probing by codingshreyash · Pull Request #9 · codingshreyash/garak

codingshreyash · 2026-07-26T23:38:12Z

What this changes

Adds shared audio infrastructure for the follow-up probe PRs. This PR does not add probes.

In resources/audio/:

A provider-neutral TTS synthesis interface with a Transformers implementation
WAV transforms for gain, speed, noise, echo, band-pass filtering, concatenation, overlay, channel handling, and resampling
Attack recipe identity, provenance, grouped result summaries, and confidence intervals
Fail-closed WAV and independent-ASR validation for generated candidates

In generators/:

OpenAI-compatible audio input payload support
NVVoiceChat, a NIM speech-to-speech target that sends WAV data in an input_audio block and reads the returned text transcript

NVVoiceChat is audio-in/text-out. It requires --target_name and does not select a default target.

First-class tool-call storage on Attempt or Message is deliberately out of scope. That data-model work is being explored in NVIDIA#2058; this PR does not introduce a competing representation.

Why

garak does not currently have a reusable audio-input path for generated adversarial candidates. This PR adds the shared path for synthesizing and transforming audio, validating candidate quality, sending it to an audio-capable target, and passing returned text to existing detectors.

The audio probes in #10 and #11 build on this infrastructure.

Duplicate-work check

This does not duplicate an open upstream PR. Searches for audio synthesis NVVoiceChat, audio input, and speech input found no open NVIDIA/garak PR implementing this foundation. NVIDIA#2058 covers the separate Attempt/Message tool-call model and is intentionally not duplicated here.

Tests

python -m pytest -q tests/test_attempt.py tests/generators/test_generators_base.py tests/generators/test_openai_compatible.py tests/generators/test_nim_voicechat.py tests/resources/test_audio_attack.py tests/resources/test_audio_synthesis.py tests/resources/test_audio_transforms.py tests/resources/test_audio_validation.py — 83 passed
python -m pytest -q tests/test_docs.py — 682 passed
python -m pytest -q tests/generators/test_generators.py — 120 passed, 2 skipped
python -m black --check <changed Python files> — passed
git diff --check — passed
Full local suite attempt — 5165 passed, 116 skipped; 113 failures and 22 errors were confined to unavailable network/model downloads, sandbox-blocked local sockets, and writes outside the workspace. GitHub CI is the cross-platform gate for the pushed commit.

Notes for review

This branch is based on feature/technique_intent because follow-up probes use IntentProbe, which is not on main yet.

This is the first of three related PRs:

feat: audio synthesis, transforms, and NVVoiceChat target for audio probing #9 adds shared synthesis, transformation, validation, provenance, and target infrastructure.
feat: spoken tool-risk and reliability audio probes + detectors #10 adds the base audio probes and detectors.
feat: audio overlay and suffix tool-risk injection probes #11 adds overlay and suffix-injection probes.

#10 and #11 depend on this PR. PR #9 can land independently.

AI assistance

OpenAI Codex assisted with review analysis, implementation, and test execution. The human submitter is responsible for reviewing every changed line, understanding the design, and validating the commands above before upstream submission.

…oiceChat target Infrastructure for the audio probe family (no probes yet): - resources/audio: pluggable TTS synthesis, signal transforms, attack provenance/recipes, audio validation, spoken-reliability case data - generators: audio-format interface (base.py supported_formats, openai.py), OpenAI-compatible NVVoiceChat audio target (nim.py) Based on feature/technique_intent. First of a 3-PR stack (foundation -> probes+detectors -> injection). Assisted-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

jmartin-tech

Some initial thoughts while I work on getting a test target up and running.

Still working thru the resources/audio utility here to understand what is being provided. Will likely need a pass to understand the contracts of the methods and another to evaluate the code inside them.

I also suspect much of the _call_model() code here can be reworked to utilized the parent method and do pre and post processing as most of it looks to be the shared.

jmartin-tech · 2026-08-05T16:05:20Z

+    pipeline, but any returned audio is ignored.
+    """
+
+    ENV_VAR = "NIM_API_KEY"


Not needed already inherited.

jmartin-tech · 2026-08-05T16:10:33Z

+        "request_timeout": 120,  # S2S endpoints can be slow
+        "request_retries": 2,


The openai.client request parameter would pass a timeout configured with each request, I see this is registered on the client creation but is not really needed.

_call_model() should implement backoff expectations the request_retries here diverges from garak generator common behavior and is probably not needed.

jmartin-tech · 2026-08-05T16:13:28Z

+    def _load_unsafe(self):
+        # voice shims may not implement /models, so don't enumerate on empty name
+        self.client = openai.OpenAI(
+            base_url=self.uri,
+            api_key=self.api_key,
+            timeout=getattr(self, "request_timeout", 120),
+            max_retries=getattr(self, "request_retries", 2),
+        )
+        if self.name in ("", None):
+            raise ValueError(
+                f"{self.generator_family_name} requires model name to be set, "
+                "e.g. --target_name <model-served-by-the-shim>"
+            )
+        self.generator = self.client.chat.completions


This again should not be needed as noted timeout should just be exposed as a default param and request_retries should be managed by _call_model() backoff configuration.

jmartin-tech · 2026-08-05T16:16:55Z

+                "spelled correctly and does the shim accept input_audio?"
+            )
+            logging.critical(msg, exc_info=e)
+            raise GarakException(f"\U0001f6d1 {msg}") from e


Do not place control characters in exception messages.

jmartin-tech · 2026-08-05T16:20:01Z

+        if tool_calls:
+            text = self._serialise_tool_calls(tool_calls, content)
+        else:
+            text = content if isinstance(content, str) else ""


This is not a sustainable way to represent response content or tool calls. Serializing the response into a string and stuffing into a text field is not viable to release as an accepted pattern.

This would tightly couple the any paired detector to data this specific generator implements without any formal structure while also placing the data in a location other detectors would attempt to process it.

jmartin-tech · 2026-08-05T16:33:15Z

+                        # this test drives a text prompt; skip generators that
+                        # require non-text input (e.g. audio-only S2S targets)
+                        text_conv = Conversation(
+                            [Turn("user", Message("first testing string"))]
+                        )
+                        try:
+                            class_instance._prepare_prompt(text_conv)
+                        except Exception:
+                            continue


This should test attributes of the class not execution to filter incompatible generators.

jmartin-tech · 2026-08-05T16:36:49Z

+def test_openai_compatible_reports_supported_audio_formats():
+    assert OpenAICompatible.supported_formats("audio") == {
+        "wav",
+        "mp3",
+    }, "reports audio formats through the generator format interface"
+    assert (
+        OpenAICompatible.supported_formats("image") == set()
+    ), "reports no image formats by default"
+
+
+def test_openai_audio_compatible_declares_audio_modality():
+    assert OpenAICompatible.modality["in"] == {
+        "text"
+    }, "generic compatible targets remain text-only at the harness boundary"
+    assert OpenAIAudioCompatible.modality["in"] == {
+        "text",
+        "audio",
+    }, "explicit audio-compatible targets accept text plus audio"
+    assert OpenAIAudioCompatible.supported_formats("audio") == {
+        "wav",
+        "mp3",
+    }, "audio-compatible targets inherit supported wire formats"


These test checks static metadata of the generator. Test should validate the contract of the unit not the static implementation details.

Also hardcoded values should not be asserted additions to the class list should not require expansion of the test unless it was due to some explicit restriction.

jmartin-tech · 2026-08-05T19:46:53Z

+    @staticmethod
+    def _append_wav_silence(wav_bytes: bytes, silence_ms: int) -> bytes:
+        """Return wav_bytes with silence_ms milliseconds of silence appended."""
+        with wave.open(io.BytesIO(wav_bytes)) as wf:
+            params = wf.getparams()
+            original_frames = wf.readframes(wf.getnframes())
+
+        silence_frames = int(params.framerate * silence_ms / 1000)
+        silence_bytes = b"\x00" * silence_frames * params.nchannels * params.sampwidth
+
+        buf = io.BytesIO()
+        with wave.open(buf, "wb") as wf_out:
+            wf_out.setparams(params)
+            wf_out.writeframes(original_frames + silence_bytes)
+        return buf.getvalue()


This seems like something that belongs in the resource utility.

jmartin-tech · 2026-08-05T19:48:09Z

+            "n",
+            "frequency_penalty",
+            "presence_penalty",
+            "timeout",


Seems odd to both inject timeout at the client level from a different mapped param and suppress it here.

jmartin-tech · 2026-08-05T19:50:43Z

+    @staticmethod
+    def _serialise_tool_calls(tool_calls, content) -> str:
+        serialised = []
+        for tc in tool_calls:
+            if hasattr(tc, "model_dump"):
+                serialised.append(tc.model_dump())
+            elif isinstance(tc, dict):
+                serialised.append(tc)
+            else:
+                serialised.append(str(tc))
+        payload = {"tool_calls": serialised}
+        if isinstance(content, str) and content:
+            payload["content"] = content
+        return json.dumps(payload, sort_keys=True, default=str)


This is not a sustainable way to represent response content. Serializing the response into an string and stuffing into a text field is not viable to release as an accepted pattern.

Reuse the shared OpenAI request path, preserve tool calls as structured message metadata, and move WAV silence padding into audio resources. Update the affected tests to exercise behavior contracts and class metadata. Assisted-by: OpenAI Codex Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

jmartin-tech

Think of many of these comments as design questions.

The revisions here that attempt to introduce first calls tool calls to Message objects are likely somewhat controversial. There is an open PR in the upstream repo exploring how Attempt and Message should add this support, it also links to an example of how another fork has been exploring this.

Similar to the last review recommendation are focused on existing garak patterns, the audio resources library code still needs a review pass targeted at what the code enables.

jmartin-tech · 2026-08-14T14:35:23Z

+            if tool_call is not None:
+                normalised.append(tool_call)
+        self.notes["tool_calls"] = normalised
+


Why is this here? simply adding a tool_calls member to the data class would be sufficient. We should not build some sort of backwards compatible layer for new functionality that is introduced.

tool_calls: Optional[list[ToolCall]] = field(default_factory=list)

jmartin-tech · 2026-08-14T14:39:17Z

+def test_supported_formats_reports_declared_modality_contract():
+    formats = _FormatAwareGenerator.supported_formats("audio")
+
+    assert formats == {"test-audio"}, "reports formats declared for the modality"
+    formats.add("caller-only")
+    assert _FormatAwareGenerator.audio_formats == {
+        "test-audio"
+    }, "returns a copy so callers cannot mutate generator metadata"


Not sure what this is testing or why it is a expected to be a contract of the Generator class.

Python interpreter features are not unit test validation canidates.

jmartin-tech · 2026-08-14T14:43:18Z

+    enumerate_models_on_missing_name = True
+    warn_on_unqualified_name = True


This is not a feature this generator needs, behavior changes the generator not in scope for the goal of the PR should not be injected.

Also note even if in scope these should be DEFAULT_PARAMS entries not class attributes.

jmartin-tech · 2026-08-14T14:44:06Z

            msg = "Model call didn't match endpoint expectations, see log"
            logging.critical(msg, exc_info=uee)
-            raise GarakException(f"🛑 {msg}") from uee
+            raise GarakException(msg) from uee


Do not change error messages not related to the PR goal.

jmartin-tech · 2026-08-14T14:44:23Z

-            msg = "NIM generation failed. Is the model name spelled correctly?"
-            logging.critical(msg, exc_info=oe)
-            raise GarakException(f"🛑 {msg}") from oe
+            raise GarakException(msg) from nfe


Do not change error messages not related to the PR goal.

jmartin-tech · 2026-08-14T14:58:17Z

+        if audio_msg.data_path is not None:
+            audio_path = Path(audio_msg.data_path)
+            if not audio_path.is_file():
+                raise GarakException(
+                    f"{self.__class__.__name__} audio file not found: {audio_path}"
+                )
+            fmt = audio_path.suffix.lower().lstrip(".")
+            if fmt not in self.audio_formats:
+                raise GarakException(
+                    f"{self.__class__.__name__} expected one of "
+                    f"{sorted(self.audio_formats)} audio formats: {audio_path}"
+                )
+            raw = audio_path.read_bytes()
+        else:


Determining a file type by extension is not portable if you have a file then it should be inspected for type, there is already built in support for this used in the else branch below related to the data_type tuple.

I suspect this code may have been intended as a runtime memory saving action, however this file extension check is too brittle and should avoided. The branch below is all that is needed.

jmartin-tech · 2026-08-14T15:01:24Z

+                )
+            raw = audio_path.read_bytes()
+        else:
+            raw = audio_msg.data


This does not need to hold the raw data in variable, only ensure it has been accessed once to be sure data_type is populated.

audio_msg.data # access the data member to ensure it was loaded by the Message

In theory this should not be required and the Message object could be enhanced to always ensure if data_path or data was used to set the binary content that data_type will return a populated value.

jmartin-tech · 2026-08-14T15:07:36Z

+        configured_extra_body = getattr(self, "extra_body", None)
+        extra_body = (
+            dict(configured_extra_body)
+            if isinstance(configured_extra_body, dict)
+            else {}
+        )
+        if self.generate_audio:
+            extra_body.setdefault("generate_audio", True)
+        self.extra_body = extra_body


Why is this here? manipulating self.extra_body every time a prompt is processed is not really a desired side-effect this method should be introducing.

jmartin-tech · 2026-08-14T15:15:02Z

+    ) -> list[ToolCall]:
+        """Return provider-neutral tool calls from an OpenAI-style response."""
+
+        del content


Why accept a parameter to immediately mark for garbage collection?

I suspect this is due to how the nim generator serializes the tool calls in a response. The base class does not need that implementation detail and the extending class should be structured to handle that without introducing responsibilities to the base class.

jmartin-tech · 2026-08-14T15:20:31Z

+    @classmethod
+    def from_value(cls, value: object) -> Optional["ToolCall"]:
+        """Normalise a structured call object or dictionary."""
+
+        if isinstance(value, cls):
+            return value if value.name.strip() else None
+        if not isinstance(value, dict):
+            return None
+
+        function = value.get("function")
+        payload = function if isinstance(function, dict) else value
+        name = payload.get("name")
+        if not isinstance(name, str) or not name.strip():
+            return None


That is this really meant to do? This looks related to the preemptive backwards compatible notes storage for tool calls that was noted is likely unnecessary.

Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

jmartin-tech · 2026-08-18T15:42:34Z

Test failures for dd4a870 are a dependency introduced issue with python 3.10.

Avoid the LiteLLM 1.97.0 Pydantic regression on Python 3.10 while leaving newer Python versions unconstrained. Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

codingshreyash force-pushed the pr/audio-1-foundation branch from 9b4ad8c to f09f8b6 Compare July 27, 2026 00:15

codingshreyash changed the title ~~Audio 1/3 — foundation: TTS synthesis, transforms, NVVoiceChat target~~ feat: audio synthesis, transforms, and NVVoiceChat target for audio probing Jul 27, 2026

codingshreyash force-pushed the pr/audio-1-foundation branch from f09f8b6 to 5d1eb3e Compare July 27, 2026 01:08

codingshreyash force-pushed the pr/audio-1-foundation branch from 5d1eb3e to 8176bcb Compare July 27, 2026 04:28

jmartin-tech reviewed Aug 5, 2026

View reviewed changes

jmartin-tech reviewed Aug 14, 2026

View reviewed changes

Address PR 9 review feedback

dd4a870

Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

codingshreyash requested a review from jmartin-tech August 19, 2026 15:30

Constrain LiteLLM on Python 3.10

7c8b118

Avoid the LiteLLM 1.97.0 Pydantic regression on Python 3.10 while leaving newer Python versions unconstrained. Assisted-by: OpenAI Codex <noreply@openai.com> Signed-off-by: Shreyash Ranjan <shrranjan@nvidia.com>

		"request_timeout": 120, # S2S endpoints can be slow
		"request_retries": 2,

		enumerate_models_on_missing_name = True
		warn_on_unqualified_name = True

Conversation

codingshreyash commented Jul 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this changes

Why

Duplicate-work check

Tests

Notes for review

AI assistance

Uh oh!

jmartin-tech left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmartin-tech left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmartin-tech commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codingshreyash commented Jul 26, 2026 •

edited

Loading