Qwen3-TTS-streaming/api/README.md

Qwen3-TTS OpenAI-like API

A FastAPI-based OpenAI-compatible API for Qwen3 Text-to-Speech with streaming support and voice cloning capabilities.

Features

• 🎵 Real-time Streaming - Stream audio chunks as they're generated
• 🎭 Voice Cloning - Clone voices using reference audio
• 🌍 Multi-language Support - English, Russian, Chinese, Japanese, Korean, and more
• 📡 OpenAI-like API - Familiar API structure for easy integration
• 🚀 High Performance - Optimized streaming with CUDA support
• 📚 Interactive Docs - Swagger UI at /docs

Installation

Prerequisites

• Python >= 3.9
• CUDA 11.8+ (for GPU support) or CPU mode
• 8GB+ VRAM recommended for GPU

Setup

1. Install dependencies

   pip install fastapi uvicorn gradio requests soundfile librosa numpy torch transformers
   

2. Install Qwen3-TTS package

   pip install -e .
   

Quick Start

Starting the API Server

# Run the API server
python -m api.main

# Or with uvicorn directly
uvicorn api.main:app --host 0.0.0.0 --port 8000

The API will be available at http://localhost:8000

Running the Gradio Demo

In a separate terminal:

python -m api.gradio_demo

The demo will be available at http://localhost:7860

API Endpoints

Health Check

GET /v1/health

Check API status and model readiness.

Response:

{
  "status": "healthy",
  "model": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
  "device": "cuda",
  "ready": true
}

List Models

GET /v1/models

Get available TTS models.

Response:

{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
      "object": "model",
      "owned_by": "Alibaba",
      "supported_languages": ["English", "Russian", "Chinese", "Japanese", "Korean"],
      "supports_streaming": true,
      "supports_voice_clone": true
    }
  ]
}

Text-to-Speech (Standard)

POST /v1/audio/speech

Generate audio from text.

Request:

{
  "text": "Hello, how are you?",
  "language": "English",
  "model": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
  "voice_clone_mode": "disabled",
  "speed": 1.0,
  "pitch": 1.0
}

Response:

{
  "id": "req_abc123def456",
  "object": "audio",
  "created": 1708284000,
  "model": "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
  "audio_base64": "UklGRi...",
  "duration": 2.5,
  "sample_rate": 24000,
  "language": "English"
}

Text-to-Speech (Streaming)

POST /v1/audio/speech/stream

Stream audio chunks as they're generated.

Request:

{
  "text": "Hello, this is a streaming test.",
  "language": "English",
  "voice_clone_mode": "disabled",
  "stream_options": {
    "emit_every_frames": 8,
    "decode_window_frames": 80,
    "overlap_samples": 512
  }
}

Response:

• Binary stream of WAV frame chunks
• Each chunk is prefixed with a 4-byte length (big-endian)
• Format: [length (4 bytes)][WAV frame data]...

Voice Cloning

Setup

Voice cloning requires two files in assets/voice_cloning/:

1. ref_audio.wav - Reference audio file

   • Format: WAV
   • Duration: 5-30 seconds
   • Clear, natural speech

2. ref_text.txt - Transcription of reference audio

   • Plain text UTF-8
   • Must exactly match the audio content

Using Voice Cloning

# 1. Place reference files
cp your_reference.wav assets/voice_cloning/ref_audio.wav
echo "Reference text here..." > assets/voice_cloning/ref_text.txt

# 2. Start the API (it will load voice cloning automatically)
python -m api.main

# 3. Use voice cloning in requests
curl -X POST "http://localhost:8000/v1/audio/speech" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "New text to synthesize",
    "language": "English",
    "voice_clone_mode": "reference_audio"
  }'

Python Integration

Using the API from Python

import requests
import base64
from io import BytesIO
import soundfile as sf

# Generate speech
response = requests.post(
    "http://localhost:8000/v1/audio/speech",
    json={
        "text": "Hello, world!",
        "language": "English",
        "voice_clone_mode": "disabled"
    }
)

data = response.json()
audio_bytes = base64.b64decode(data["audio_base64"])
audio, sr = sf.read(BytesIO(audio_bytes))

# Save to file
sf.write("output.wav", audio, sr)

Streaming Example

import requests

response = requests.post(
    "http://localhost:8000/v1/audio/speech/stream",
    json={
        "text": "This is a streaming test.",
        "language": "English",
    },
    stream=True
)

chunks = []
for chunk in response.iter_content(chunk_size=65536):
    # Parse frame length (4 bytes)
    frame_len = int.from_bytes(chunk[:4], byteorder='big')
    # Extract WAV frame data
    frame_data = chunk[4:4 + frame_len]
    chunks.append(frame_data)

Configuration

Model Selection

Currently supports:

• Qwen/Qwen3-TTS-12Hz-1.7B-Base (default)

More models can be added to TTSModel enum in api/models.py.

Device Selection

The API automatically selects the best available device:

• CUDA GPU (if available and with sufficient VRAM)
• CPU (fallback)

To force a specific device, modify api/main.py:

# In ModelState.__init__
self.device = "cpu"  # Force CPU mode

Streaming Options

Adjust streaming performance in api/models.py:

class StreamOptions(BaseModel):
    emit_every_frames: int = 8        # Lower = more chunks, lower latency
    decode_window_frames: int = 80    # Higher = better quality, higher latency
    overlap_samples: int = 512        # Overlap for smooth transitions

Performance Tips

1. GPU Optimization

   • Use CUDA 11.8+ for better performance
   • flash_attention_2 is automatically enabled when available

2. Streaming Optimization

   • Lower emit_every_frames for faster first chunk
   • Increase decode_window_frames for better audio quality

3. Production Deployment

   • Use a production ASGI server like Gunicorn or Hypercorn
   • Enable response caching for identical requests
   • Monitor GPU memory usage

Troubleshooting

API won't start

error: No module named 'transformers'
→ pip install transformers accelerate

Model loading fails

error: CUDA out of memory
→ Use CPU mode: modify device to "cpu" in api/main.py
→ Or reduce batch size

Voice cloning not working

• Verify ref_audio.wav and ref_text.txt exist in assets/voice_cloning/
• Check file permissions
• Ensure reference text matches audio content exactly

Slow responses

• Check GPU/CPU utilization
• Reduce decode_window_frames for faster (but lower quality) generation
• Ensure sufficient system memory available

API Documentation

Once the server is running, visit:

• Interactive Docs (Swagger UI): http://localhost:8000/docs
• ReDoc Documentation: http://localhost:8000/redoc

Examples

See the examples/ directory for more usage examples:

• test_streaming.py - Streaming generation example
• test_model_12hz_base.py - Base model usage
• test_model_12hz_custom_voice.py - Voice cloning example

License

This API is licensed under the Apache 2.0 License. See LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit pull requests.

Support

For issues, questions, or suggestions:

1. Check the troubleshooting section
2. Review the API documentation at /docs
3. Check logs for detailed error messages