Repository navigation
feat(agent): speech through a ComfyUI server's Qwen3-TTS nodes, with cloning and clean-then-clone - #98
Merged
Conversation
…Qwen3-TTS nodes @av/comfyui-speech is a small ComfyUI client (queue, wait on the history, upload an input, download an output) and four workflows kept as API-format templates: a preset speaker, a cloned reference, a voice saved on the server, and cleaning a reference (cut, voice separation) before cloning it. pva_synthesize_speech takes provider "comfyui-qwen": voice is a speaker or saved:<name>, referenceAudioPath clones a recording of the workspace, and cleanReference cleans it first. It is free, never chosen on its own, and writes the WAV and its synthesis record like the other providers. The server's address is AVG_COMFYUI_URL or the library's .render/comfyui.json. pva_capability_status says whether the server answers and what it can run; pva_voices lists its speakers and saved voices. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…fyUI server, without cloning @av/comfyui-speech gains cleanVoice and workflows/clean.json: LoadAudio, an optional TrimAudioDuration (bypassed when no cut is asked for), AudioSeparation's Vocals, SaveAudio, with the clean-clone chain's nodes and inputs. The new pva_clean_voice keeps the voice as a new WAV under the workspace's assets/, with <file>.clean.json beside it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d honest remote provenance
phoebus-84
force-pushed
the
feat/comfyui-qwen-speech
branch
from
November 16, 2026 17:11
1673fff to
57ddce9
Compare
phoebus-84
changed the base branch from
feat/elevenlabs-speech-avatar
to
main
November 16, 2026 17:11
Owner
|
Integration is now rebased/transplanted onto main after #99 → #95, without #96. Original head is preserved locally as refs/backup/pr98-before-integration (1673fff). Full local |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Optional ComfyUI Qwen3-TTS speech behind the existing
pva_synthesize_speechtool. Integrated after #99 and #95; the local Qwen worker (MLX or Torch) remains the default and failures never switch provider.Originally stacked on #96. This branch now targets main, carrying only the two ComfyUI commits and integration fixes: #96's ElevenLabs/avatar features are not included.
Features
@av/comfyui-speech: explicit server configuration, queue/poll/upload/download client, node/readiness probing and five API-format JSON workflows: preset speaker, reference clone, saved voice, clean-then-clone and clean-only.provider: "comfyui-qwen"uses the same episode/source/film/scene/shot workspace addressing and synthesis → register → align → verify flow. No fallback; no change to the worker's durable attempt/mastering contracts or sealed series profiles.pva_clean_voiceseparates vocals from an optional range into a new workspace asset, without cloning.voiceConsentPath, an operator-created JSON authorization record bound to the reference SHA-256 or exactsaved:<name>. Checked before upload/execution; removing the record or settingconfirmed: falseblocks subsequent calls. The agent must not grant consent for the operator.runtimeVerified: falseexplicitly acknowledges that ComfyUI does not attest actual checkpoint hashes/resolved runtime versions. Saved voice names are likewise not immutable waveform identities.Validation
npm run checkpassed: 23 workspace test runs, zero failures.npm run docs:buildpassed, full profile composition passed; agent typecheck passed.Offline faster-whisper large-v3 transcribed both clone samples with zero normalized word errors. The preset differed only in spelling of the product name (
ComfyUI→ConfiUi, 1/9 words). This is a technical/content smoke, not a listening or voice-similarity approval or a controlled comparison against #95.Reports, logs and synthetic audio remain under
~/ai/video-studio-pr98-smoke/, never git. The smoke wrote only its ownstudio-pr98-smokeComfyUI input/output namespace; it did not save a new speaker.Configuration and limits
Set
AVG_COMFYUI_URL, orurlinlibrary/.render/comfyui.json(AVG_COMFYUI_CONFIGcan name another file). For loopback-only servers, open your own SSH tunnel. Node settings includedevice,precision,attentionandunloadModelAfterGenerate(default true).No word timings, VoiceDesign, dialogue/fine-tuning nodes or automatic tunnel management. Saved-voice execution/save-as remain fake-server-tested only; no existing saved voice was exercised live. Full consent-record format and provenance limits are in
docs/running.md.