mirror of
https://github.com/Nighthawk42/Qwen3-TTS-streaming.git
synced 2026-08-30 09:02:26 +00:00
revert: restore codec levels 1-15 embedding summing in finetuning
Reverts a change we applied from open upstream PR #178 (commit f83f184),
which removed the sub-codebook embedding loop arguing the talker shouldn't
see levels 1-15 during training. Investigation of the inference code
(modeling_qwen3_tts.py:1894-1920) shows the sub-talker generates levels
1-15 before the talker's next step, and all 16 levels are summed into
the input. Omitting them from training creates a train/inference mismatch.
This commit is contained in:
@@ -92,6 +92,11 @@ def train():
|
||||
|
||||
input_embeddings = input_text_embedding + input_codec_embedding
|
||||
|
||||
for i in range(1, 16):
|
||||
codec_i_embedding = model.talker.code_predictor.get_input_embeddings()[i - 1](codec_ids[:, :, i])
|
||||
codec_i_embedding = codec_i_embedding * codec_mask.unsqueeze(-1)
|
||||
input_embeddings = input_embeddings + codec_i_embedding
|
||||
|
||||
outputs = model.talker(
|
||||
inputs_embeds=input_embeddings,
|
||||
attention_mask=attention_mask,
|
||||
|
||||
Reference in New Issue
Block a user