Skip to content

Audio: pluggable PETTS synthesis and provenance - #2

Closed
codingshreyash wants to merge 1 commit into
base/tts-techniquefrom
pr/audio-resources
Closed

Audio: pluggable PETTS synthesis and provenance#2
codingshreyash wants to merge 1 commit into
base/tts-techniquefrom
pr/audio-resources

Conversation

@codingshreyash

@codingshreyash codingshreyash commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Why

PETTS needs a stable synthesis boundary before additional audio probes can build on it. This PR separates text-to-speech synthesis from probe logic, keeps modality conversion in the attack layer, and records enough provenance to reproduce generated audio.

What changed

  • adds a provider-neutral SynthesisProvider contract and request/result types
  • adds a lazily loaded Transformers provider with public default checkpoint suno/bark-small
  • keeps synthesis in probes.audio.PETTS; generators remain responsible only for target communication
  • adds bounded PCM WAV transforms for follow-up audio probes
  • records requested and effective synthesis settings, provider identity, checkpoint revision, transform recipe, and streaming SHA-256 file hashes
  • fails closed when the target does not advertise a compatible audio input format
  • documents configuration, architecture boundaries, and licensing considerations

No mandatory project dependency is added. Audio-specific imports are deferred until PETTS is invoked.

Configuration

PETTS uses garak's standard plugin configuration:

  • tts_model_name
  • tts_model_revision
  • tts_voice
  • tts_sample_rate
  • tts_audio_format
  • tts_audio_subtype
  • tts_audio_stereo

The public default checkpoint is distributed under the MIT licence. Its local download and inference requirements can be substantial; users remain responsible for the terms of any alternative checkpoint or remote provider they configure.

Relationship to upstream and duplicate check

The duplicate-work search found NVIDIA#1770. This PR intentionally builds on that work at commit cf8648ea; it does not duplicate its IntentProbe, initial PETTS implementation, or generator-format support. No other open upstream PR matched the synthesis-provider and audio-resource scope.

This PR intentionally does not add a target adapter, automatic speech recognition, or security-specific audio probes.

Verification

Run locally with Python 3.12:

XDG_CACHE_HOME=/private/tmp/garak-pr-audit-cache \
XDG_DATA_HOME=/private/tmp/garak-pr-audit-data \
python -m pytest -q \
  tests/resources/test_audio_attack.py \
  tests/resources/test_audio_synthesis.py \
  tests/resources/test_audio_transforms.py \
  tests/probes/test_probes_audio.py \
  tests/test_docs.py

Result: 706 passed.

python -m black --check <changed Python files>
git diff --check

Result: clean.

Tests cover provider selection, lazy loading, provenance, file hashing, WAV transforms, requested-versus-effective settings, unsupported formats, and malformed or oversized input. No live hosted-service result is claimed.

Reviewer focus

The main review questions are whether the provider contract is narrow enough for local and remote synthesis implementations, whether PETTS owns the modality shift at the correct layer, and whether the provenance is sufficient without coupling the probe to a specific checkpoint.

AI assistance

ChatGPT was used to review this change.

@codingshreyash codingshreyash changed the title Audio: resource library (garak.resources.audio) Audio: pluggable PETTS synthesis and provenance Jul 23, 2026
@codingshreyash
codingshreyash changed the base branch from base/main to base/tts-technique July 23, 2026 21:08
Add provider-neutral synthesis, bounded audio transforms, and provenance helpers while keeping modality conversion in the probe layer.

AI-Review: ChatGPT
@codingshreyash

Copy link
Copy Markdown
Owner Author

Superseded by the rebased, cleaned 3-PR stack based on current feature/technique_intent: #9 (foundation), #10 (probes+detectors), #11 (injection). Content preserved there; closing to consolidate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant