mirror of
https://github.com/Nighthawk42/Qwen3-TTS-streaming.git
synced 2026-08-30 10:12:26 +00:00
rekuenkdr
1a067a0cc6
fix: add repetition penalty to streaming and sync upstream finetuning
Streaming inference (voice clone) - repetition penalty:
Without repetition penalty, the model can fall into a degenerate state where it keeps sampling the same codec tokens over and over. This manifests as:
Looping audio: the same syllable or sound fragment repeats endlessly
Extremely long generation: instead of reaching EOS in ~200-500 frames, it runs for thousands of frames (up to max_frames=10000)
Apparent "slowness": a response that should take ~1s of audio takes 10-30s to generate
The fix works by tracking previously generated first-codebook token IDs and penalizing them before sampling:
Tokens with positive logits get divided by repetition_penalty (lowering their probability)
Tokens with negative logits get multiplied by it (pushing them further down)
This nudges the model away from re-selecting the same tokens, so it progresses through the text naturally and reaches EOS in a reasonable number of steps rather than looping. Default is 1.0 (disabled) and is exposed through the supported_params whitelist in
stream_generate_voice_clone() so it can be set via generate_config or user kwargs.
Upstream sync (QwenLM/Qwen3-TTS):
Bump version 0.0.4 -> 0.1.1 to match upstream release.
finetuning/sft_12hz.py: weight sub-talker loss by 0.3 factor to prevent the code predictor gradient from dominating the main talker loss during SFT.
finetuning/sft_12hz.py: remove sub-codebook embedding accumulation loop (codec groups 1-15) from input embeddings, unnecessary and harmful for finetuning convergence (upstream PR Resolve each training epoch resulting in progressively faster output QwenLM/Qwen3-TTS#178).
finetuning/README.md: update recommended hyperparameters to batch_size=32, lr=2e-6, num_epochs=10 for more stable training.