fix(demo): free GPU tensors between batches to prevent VRAM accumulation

[SuperMarioYL]

Problem

When --audio_files contains multiple files, transcribe_with_batching() loops over
batches and calls transcribe_batch() for each. Inside transcribe_batch(), two large
GPU tensors are created:

• inputs — the padded batch of speech features moved to device
• output_ids — the generated token ids from model.generate()

Neither is explicitly freed before the function returns. Python's reference-count GC
does not guarantee immediate CUDA memory deallocation, so the allocator cannot reclaim
VRAM between iterations. After 2–3 batches of 25-min audio on a 24 GB GPU the process
OOMs even though each individual batch fits comfortably (issue #368).

Fix

Three targeted changes, no logic modifications:

1. transcribe_batch() — del inputs, output_ids immediately after the decode loop,
   then flush the CUDA allocator via torch.cuda.empty_cache().

2. _release_device_cache() helper — 2-line private method (empty_cache + gc.collect)
   called from the three callsites that need it; avoids repeating the pair inline.

3. close() method — explicit teardown for callers who want to reclaim all VRAM after
   processing (e.g. before loading a second model in the same process). The main() entry
   point wraps inference in try/finally to call it on both normal exit and exception.

Verification

• No change to any transcription output or model weights.
• Argument signatures of transcribe_batch and transcribe_with_batching are unchanged.
• All existing demo usage (python demo/vibevoice_asr_inference_from_file.py ...) works as before.

Fixes #368

[SuperMarioYL]

@microsoft-github-policy-service agree

[SuperMarioYL]

@microsoft-github-policy-service agree

[rickthomasjr]

Review: Approve — GPU VRAM memory leak fix

Problem addressed: VRAM accumulation across consecutive batch processing. This is a real issue (#368) where torch holds onto allocated device memory even after tensors are no longer referenced.

Changes are well-structured:

1. del inputs, output_ids after each batch — explicit tensor release
2. New _release_device_cache() — calls torch.cuda.empty_cache() and gc.collect()
3. New close() method — ensures model/processor are freed when done
4. try/finally: asr.close() in main — ensures cleanup even on error

Potential concern: torch.cuda.empty_cache() is an expensive operation. In tight batch loops it could add measurable overhead. However, since this is for processing "many files" (not intra-batch), the trade-off is appropriate. The alternative would be to only call empty_cache() every N batches.

Verdict: Approve. Addresses a genuine VRAM leak with a clean API (close pattern). The try/finally ensures no resource leak.

[SuperMarioYL]

This has been approved and green for a few weeks now — is there anything blocking the merge, or anything else you'd like from me? Happy to rebase onto the latest base if that helps.