Document the new batch_stream_generate_voice_clone() method including
a usage example showing voice prompt broadcasting, per-item chunk
accumulation, and WAV output. Add entry to the "Added in this fork" list.
Introduce batch streaming methods that process multiple text inputs in a
single batched pass through the transformer, enabling parallel audio
generation with shared KV cache and lockstep frame advancement.
Core model layer (modeling_qwen3_tts.py):
- Add `batch_stream_generate_pcm()` to Qwen3TTSForConditionalGeneration
- Batched prefill and single-step decode with shared KV cache across items
- Per-item state management for codec buffers, crossfade tails, repetition
penalty tracking, ref_code ICL contexts, and independent EOS detection
- Two-phase streaming support: aggressive first-chunk emission followed by
stable-phase parameters for balancing latency vs quality
- Windowed decode with per-item Hann fade-in/out and overlap crossfade
- Flush pass to decode remaining frames per item after generation completes
High-level API (qwen3_tts_model.py):
- Add `batch_stream_generate_voice_clone()` to Qwen3TTSModel
- Handles broadcasting of language and voice_clone_prompt to batch size
- Input validation, tokenization, and ref_ids construction per item
- Filters and merges generation kwargs before delegating to core method
Example script (examples/test_batch_streaming.py):
- Demonstrates batch streaming with 3 texts using a shared voice prompt
- Benchmarks batch vs sequential single-item streaming for comparison
- Saves per-item WAV outputs with timing and size reporting
fix: add repetition penalty to streaming and sync upstream finetuning
Streaming inference (voice clone) - repetition penalty:
Without repetition penalty, the model can fall into a degenerate state where it keeps sampling the same codec tokens over and over. This manifests as:
Looping audio: the same syllable or sound fragment repeats endlessly
Extremely long generation: instead of reaching EOS in ~200-500 frames, it runs for thousands of frames (up to max_frames=10000)
Apparent "slowness": a response that should take ~1s of audio takes 10-30s to generate
The fix works by tracking previously generated first-codebook token IDs and penalizing them before sampling:
Tokens with positive logits get divided by repetition_penalty (lowering their probability)
Tokens with negative logits get multiplied by it (pushing them further down)
This nudges the model away from re-selecting the same tokens, so it progresses through the text naturally and reaches EOS in a reasonable number of steps rather than looping. Default is 1.0 (disabled) and is exposed through the supported_params whitelist in
stream_generate_voice_clone() so it can be set via generate_config or user kwargs.
Upstream sync (QwenLM/Qwen3-TTS):
Bump version 0.0.4 -> 0.1.1 to match upstream release.
finetuning/sft_12hz.py: weight sub-talker loss by 0.3 factor to prevent the code predictor gradient from dominating the main talker loss during SFT.
finetuning/sft_12hz.py: remove sub-codebook embedding accumulation loop (codec groups 1-15) from input embeddings, unnecessary and harmful for finetuning convergence (upstream PR Resolve each training epoch resulting in progressively faster output QwenLM/Qwen3-TTS#178).
finetuning/README.md: update recommended hyperparameters to batch_size=32, lr=2e-6, num_epochs=10 for more stable training.
Streaming inference (voice clone) - repetition penalty:
Without repetition penalty, the model can fall into a degenerate state
where it keeps sampling the same codec tokens over and over. This
manifests as:
- Looping audio: the same syllable or sound fragment repeats endlessly
- Extremely long generation: instead of reaching EOS in ~200-500 frames,
it runs for thousands of frames (up to max_frames=10000)
- Apparent "slowness": a response that should take ~1s of audio takes
10-30s to generate
The fix works by tracking previously generated first-codebook token IDs
and penalizing them before sampling:
- Tokens with positive logits get divided by repetition_penalty (lowering
their probability)
- Tokens with negative logits get multiplied by it (pushing them further
down)
This nudges the model away from re-selecting the same tokens, so it
progresses through the text naturally and reaches EOS in a reasonable
number of steps rather than looping. Default is 1.0 (disabled) and is
exposed through the supported_params whitelist in
stream_generate_voice_clone() so it can be set via generate_config or
user kwargs.
Upstream sync (QwenLM/Qwen3-TTS):
- Bump version 0.0.4 -> 0.1.1 to match upstream release.
- finetuning/sft_12hz.py: weight sub-talker loss by 0.3 factor to
prevent the code predictor gradient from dominating the main talker
loss during SFT.
- finetuning/sft_12hz.py: remove sub-codebook embedding accumulation
loop (codec groups 1-15) from input embeddings, unnecessary and
harmful for finetuning convergence (upstream PR #178).
- finetuning/README.md: update recommended hyperparameters to
batch_size=32, lr=2e-6, num_epochs=10 for more stable training.
fix: handle multiple EOS tokens for generation termination
Add support for multiple EOS tokens that can terminate TTS generation, rather than just checking for a single codec EOS token. This handles different EOS tokens the model might emit (codec EOS, TTS special tokens, endoftext, etc.) in order to avoid the issues with the 0.6B model.
Allow users to set any overlap_samples value including 0 to disable
crossfade blending entirely. Renamed MIN_BLEND_SAMPLES to
DEFAULT_BLEND_SAMPLES and removed the enforced minimum.
- Replace linear crossfade with Hann window for smoother transitions
- Add MIN_BLEND_SAMPLES (512) as floor for blend region
- Add Hann fade-in on first chunk to prevent startup pop
- Add Hann fade-out on final chunk to prevent ending pop
- Trim chunk tails before emission to prevent echo artifacts
- Update README with audio quality fixes documentation
Kedara Studios