mirror of
https://github.com/Nighthawk42/Qwen3-TTS-streaming.git
synced 2026-08-30 09:52:26 +00:00
Kedara Studios
117ec6192a
Introduce batch streaming methods that process multiple text inputs in a single batched pass through the transformer, enabling parallel audio generation with shared KV cache and lockstep frame advancement. Core model layer (modeling_qwen3_tts.py): - Add `batch_stream_generate_pcm()` to Qwen3TTSForConditionalGeneration - Batched prefill and single-step decode with shared KV cache across items - Per-item state management for codec buffers, crossfade tails, repetition penalty tracking, ref_code ICL contexts, and independent EOS detection - Two-phase streaming support: aggressive first-chunk emission followed by stable-phase parameters for balancing latency vs quality - Windowed decode with per-item Hann fade-in/out and overlap crossfade - Flush pass to decode remaining frames per item after generation completes High-level API (qwen3_tts_model.py): - Add `batch_stream_generate_voice_clone()` to Qwen3TTSModel - Handles broadcasting of language and voice_clone_prompt to batch size - Input validation, tokenization, and ref_ids construction per item - Filters and merges generation kwargs before delegating to core method Example script (examples/test_batch_streaming.py): - Demonstrates batch streaming with 3 texts using a shared voice prompt - Benchmarks batch vs sequential single-item streaming for comparison - Saves per-item WAV outputs with timing and size reporting