thx

README.md

@@ -39,7 +39,7 @@ language:
   <a href="https://github.com/OpenMOSS/MOSS-TTS/tree/main"><img src="https://img.shields.io/badge/Project%20Page-GitHub-blue"></a>
   <a href="https://modelscope.cn/collections/OpenMOSS-Team/MOSS-TTS"><img src="https://img.shields.io/badge/ModelScope-Models-lightgrey?logo=modelscope&amp"></a>
   <a href="https://mosi.cn/#models"><img src="https://img.shields.io/badge/Blog-View-blue?logo=internet-explorer&amp"></a>
-  <a href="https://arxiv.org/abs/2603.18090"><img src="https://img.shields.io/badge/Arxiv-2603.18090-red?logo=Arxiv&amp"></a>
+  <a href="https://github.com/OpenMOSS/MOSS-TTS"><img src="https://img.shields.io/badge/Arxiv-Coming%20soon-red?logo=arxiv&amp"></a>
 
   <a href="https://studio.mosi.cn"><img src="https://img.shields.io/badge/AIStudio-Try-green?logo=internet-explorer&amp"></a>
   <a href="https://studio.mosi.cn/docs/moss-tts"><img src="https://img.shields.io/badge/API-Docs-00A3FF?logo=fastapi&amp"></a>
@@ -47,7 +47,6 @@ language:
   <a href="https://discord.gg/fvm5TaWjU3"><img src="https://img.shields.io/badge/Discord-Join-5865F2?logo=discord&amp"></a>
 </div>
 
-
 ## Overview
 MOSS‑TTS Family is an open‑source **speech and sound generation model family** from [MOSI.AI](https://mosi.cn/#hero) and the [OpenMOSS team](https://www.open-moss.com/). It is designed for **high‑fidelity**, **high‑expressiveness**, and **complex real‑world scenarios**, covering stable long‑form speech, multi‑speaker dialogue, voice/character design, environmental sound effects, and real‑time streaming TTS.
 
@@ -61,37 +60,23 @@ MOSS‑TTS Family is an open‑source **speech and sound generation model family
 
 When a single piece of audio needs to **sound like a real person**, **pronounce every word accurately**, **switch speaking styles across content**, **remain stable over tens of minutes**, and **support dialogue, role‑play, and real‑time interaction**, a single TTS model is often not enough. The **MOSS‑TTS Family** breaks the workflow into five production‑ready models that can be used independently or composed into a complete pipeline.
 
-- **MOSS‑TTS**: The flagship production model featuring high fidelity and optimal zero-shot voice cloning. It supports **long-speech generation**, **fine-grained control over Pinyin, phonemes, and duration**, as well as **multilingual/code-switched synthesis**.
-- **MOSS‑TTSD**: A spoken dialogue generation model for expressive, multi-speaker, and ultra-long dialogues. The new **v1.0 version** achieves **industry-leading performance on objective metrics** and **outperformed top closed-source models like Doubao and Gemini 2.5-pro** in subjective evaluations. You can visit the [MOSS-TTSD repository](https://github.com/OpenMOSS/MOSS-TTSD) for details.
-- **MOSS‑VoiceGenerator**: An open-source voice design model capable of generating diverse voices and styles directly from text prompts, **without any reference speech**. It unifies voice design, style control, and synthesis, functioning independently or as a design layer for downstream TTS. Its performance **surpasses other top-tier voice design models in arena ratings**.
-- **MOSS‑TTS‑Realtime**: A multi-turn context-aware model for real-time voice agents. It uses incremental synthesis to ensure natural and coherent replies, making it **ideal for building low-latency voice agents when paired with text models**. The TTFB (Time To First Byte) of MOSS-TTS-Realtime reaches 180 ms, and the $T_{\text{LLM-first-sentence}} + T_{\text{MOSS-TTS-Realtime-TTFB}}$ is 377 ms.
-- **MOSS‑SoundEffect**: A content creation model specialized in **sound effect generation** with wide category coverage and controllable duration. It generates audio for natural environments, urban scenes, biological sounds, human actions, and musical fragments, suitable for film, games, and interactive experiences.
-
-
-## Model Architecture
-
-We train **MossTTSDelay** and **MossTTSLocal** as complementary baselines under one training/evaluation setup: **Delay** emphasizes long-context stability, inference speed, and production readiness, while **Local** emphasizes lightweight flexibility and strong objective performance for streaming-oriented systems. Together they provide reproducible references for deployment and research.
-
-**MossTTSRealtime** is not a third comparison baseline; it is a capability-driven design for voice agents. By modeling multi-turn context from both prior text and user acoustics, it delivers low-latency streaming speech that stays coherent and voice-consistent across turns.
+- **MOSS‑TTS**: MOSS-TTS is the flagship production TTS foundation model, centered on high-fidelity zero-shot voice cloning with controllable long-form synthesis, pronunciation, and multilingual/code-switched speech. It serves as the core engine for scalable narration, dubbing, and voice-driven products.
+- **MOSS‑TTSD**: MOSS-TTSD is a production long-form dialogue model for expressive multi-speaker conversational audio at scale. It supports long-duration continuity, turn-taking control, and zero-shot voice cloning from short references for podcasts, audiobooks, commentary, dubbing, and entertainment dialogue.
+- **MOSS‑VoiceGenerator**: MOSS-VoiceGenerator is an open-source voice design model that creates speaker timbres directly from free-form text, without reference audio. It unifies timbre design, style control, and content synthesis, and can be used standalone or as a voice-design layer for downstream TTS.
+- **MOSS‑SoundEffect**: MOSS-SoundEffect is a high-fidelity text-to-sound model with broad category coverage and controllable duration for real content production. It generates stable audio from prompts across ambience, urban scenes, creatures, human actions, and music-like clips for film, games, interactive media, and data synthesis.
+- **MOSS‑TTS‑Realtime**: MOSS-TTS-Realtime is a context-aware, multi-turn streaming TTS model for real-time voice agents. By conditioning on dialogue history across both text and prior user acoustics, it delivers low-latency synthesis with coherent, consistent voice responses across turns.
 
 
-| Architecture  | Core Mechanism | Arch Details |
-|---|---|---|
-| `MossTTSDelay` |  Multi‑head parallel RVQ prediction with delay‑pattern scheduling | [![Arch Details](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_tts_delay/README.md) |
-| `MossTTSLocal` | Time‑synchronous RVQ blocks with a depth transformer | [![Arch Details](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_tts_local/README.md) |
-| `MossTTSRealtime` | Hierarchical text–audio inputs for realtime synthesis | [![Arch Details](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/moss_tts_realtime/README.md) |
-
 ## Released Models
 
-
-| Model | Architecture | Size | Model Card | Hugging Face | ModelScope |
-|---|---|---:|---|---|---|
-| **MOSS-TTS** | `MossTTSDelay` | 8B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-TTS) |
-|  | `MossTTSLocal` | 1.7B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-TTS-Local-Transformer) |
-| **MOSS‑TTSD‑V1.0** | `MossTTSDelay` | 8B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_ttsd_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-TTSD-v1.0) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-TTSD-v1.0) |
-| **MOSS‑VoiceGenerator** | `MossTTSDelay` | 1.7B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_voice_generator_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-VoiceGenerator) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-VoiceGenerator) |
-| **MOSS‑SoundEffect** | `MossTTSDelay` | 8B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_sound_effect_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-SoundEffect) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-SoundEffect) |
-| **MOSS‑TTS‑Realtime** | `MossTTSRealtime` | 1.7B | [![Model Card](https://img.shields.io/badge/Model%20Card-View-blue?logo=markdown)](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_realtime_model_card.md) | [![Hugging Face](https://img.shields.io/badge/Huggingface-Model-orange?logo=huggingface)](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime) | [![ModelScope](https://img.shields.io/badge/ModelScope-Model-lightgrey?logo=modelscope)](https://modelscope.cn/models/openmoss/MOSS-TTS-Realtime) |
+| Model | Architecture | Size | Model Card | Hugging Face |
+|---|---|---:|---|---|
+| **MOSS-TTS** | MossTTSDelay | 8B | [moss_tts_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS) |
+|  | MossTTSLocal | 1.7B | [moss_tts_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer) |
+| **MOSS‑TTSD‑V1.0** | MossTTSDelay | 8B | [moss_ttsd_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_ttsd_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTSD-v1.0) |
+| **MOSS‑VoiceGenerator** | MossTTSDelay | 1.7B | [moss_voice_generator_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_voice_generator_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-Voice-Generator) |
+| **MOSS‑SoundEffect** | MossTTSDelay | 8B | [moss_sound_effect_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_sound_effect_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-SoundEffect) |
+| **MOSS‑TTS‑Realtime** | MossTTSRealtime | 1.7B | [moss_tts_realtime_model_card.md](https://github.com/OpenMOSS/MOSS-TTS/blob/main/docs/moss_tts_realtime_model_card.md) | 🤗 [Huggingface](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Realtime) |
 
 ## Supported Languages
 
@@ -101,12 +86,13 @@ MOSS-TTS, MOSS-TTSD and MOSS-TTS-Realtime currently supports **20 languages**:
 |---|---|---|---|---|---|---|---|---|
 | Chinese | zh | 🇨🇳 | English | en | 🇺🇸 | German | de | 🇩🇪 |
 | Spanish | es | 🇪🇸 | French | fr | 🇫🇷 | Japanese | ja | 🇯🇵 |
-| Italian | it | 🇮🇹 | Hungarian | hu | 🇭🇺 | Korean | ko | 🇰🇷 |
+| Italian | it | 🇮🇹 | Hebrew | he | 🇮🇱 | Korean | ko | 🇰🇷 |
 | Russian | ru | 🇷🇺 | Persian (Farsi) | fa | 🇮🇷 | Arabic | ar | 🇸🇦 |
 | Polish | pl | 🇵🇱 | Portuguese | pt | 🇵🇹 | Czech | cs | 🇨🇿 |
-| Danish | da | 🇩🇰 | Swedish | sv | 🇸🇪 | | | |
+| Danish | da | 🇩🇰 | Swedish | sv | 🇸🇪 | Hungarian | hu | 🇭🇺 |
 | Greek | el | 🇬🇷 | Turkish | tr | 🇹🇷 |  |  |  |
 
+
 # MOSS-TTS
 ## 1. Overview
 ### 1.1 TTS Family Positioning
@@ -245,6 +231,27 @@ torch.backends.cuda.enable_flash_sdp(True)
 torch.backends.cuda.enable_mem_efficient_sdp(True)
 torch.backends.cuda.enable_math_sdp(True)
 
+class DelayGenerationConfig(GenerationConfig):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self.layers = kwargs.get("layers", [{} for _ in range(32)])
+        self.do_samples = kwargs.get("do_samples", None)
+        self.n_vq_for_inference = 32
+        
+def initial_config(tokenizer, model_name_or_path):
+    generation_config = DelayGenerationConfig.from_pretrained(model_name_or_path)
+    generation_config.pad_token_id = tokenizer.pad_token_id
+    generation_config.eos_token_id = 151653
+    generation_config.max_new_tokens = 1000000
+    generation_config.temperature = 1.0
+    generation_config.top_p = 0.95
+    generation_config.top_k = 100
+    generation_config.repetition_penalty = 1.1
+    generation_config.use_cache = True
+    generation_config.do_sample = False
+    return generation_config
+
+
 pretrained_model_name_or_path = "OpenMOSS-Team/MOSS-TTS-Local-Transformer"
 device = "cuda" if torch.cuda.is_available() else "cpu"
 dtype = torch.bfloat16 if device == "cuda" else torch.float32
@@ -339,6 +346,25 @@ model = AutoModel.from_pretrained(
 ).to(device)
 model.eval()
 
+generation_config = initial_config(processor.tokenizer, pretrained_model_name_or_path)
+generation_config.n_vq_for_inference = model.channels - 1
+generation_config.do_samples = [True] * model.channels
+generation_config.layers = [
+    {
+        "repetition_penalty": 1.0, 
+        "temperature": 1.5, 
+        "top_p": 1.0, 
+        "top_k": 50
+    }
+] + [ 
+    {
+        "repetition_penalty": 1.1, 
+        "temperature": 1.0, 
+        "top_p": 0.95,
+        "top_k": 50
+    }
+] * (model.channels - 1) 
+
 batch_size = 1
 
 save_dir = Path(f"inference_root_moss_tts_local_transformer_generation")
@@ -354,7 +380,7 @@ with torch.no_grad():
         outputs = model.generate(
             input_ids=input_ids,
             attention_mask=attention_mask,
-            max_new_tokens=4096,
+            generation_config=generation_config
         )
 
         for message in processor.decode(outputs):
@@ -362,6 +388,7 @@ with torch.no_grad():
             out_path = save_dir / f"sample{sample_idx}.wav"
             sample_idx += 1
             torchaudio.save(out_path, audio.unsqueeze(0), processor.model_config.sampling_rate)
+
 ```
 
 ### Continuation + Voice Cloning (Prefix Audio + Text)
@@ -381,6 +408,27 @@ torch.backends.cuda.enable_flash_sdp(True)
 torch.backends.cuda.enable_mem_efficient_sdp(True)
 torch.backends.cuda.enable_math_sdp(True)
 
+class DelayGenerationConfig(GenerationConfig):
+    def __init__(self, **kwargs):
+        super().__init__(**kwargs)
+        self.layers = kwargs.get("layers", [{} for _ in range(32)])
+        self.do_samples = kwargs.get("do_samples", None)
+        self.n_vq_for_inference = 32
+        
+def initial_config(tokenizer, model_name_or_path):
+    generation_config = DelayGenerationConfig.from_pretrained(model_name_or_path)
+    generation_config.pad_token_id = tokenizer.pad_token_id
+    generation_config.eos_token_id = 151653
+    generation_config.max_new_tokens = 1000000
+    generation_config.temperature = 1.0
+    generation_config.top_p = 0.95
+    generation_config.top_k = 100
+    generation_config.repetition_penalty = 1.1
+    generation_config.use_cache = True
+    generation_config.do_sample = False
+    return generation_config
+
+
 pretrained_model_name_or_path = "OpenMOSS-Team/MOSS-TTS-Local-Transformer"
 device = "cuda" if torch.cuda.is_available() else "cpu"
 dtype = torch.bfloat16 if device == "cuda" else torch.float32
@@ -447,6 +495,25 @@ model = AutoModel.from_pretrained(
 ).to(device)
 model.eval()
 
+generation_config = initial_config(processor.tokenizer, pretrained_model_name_or_path)
+generation_config.n_vq_for_inference = model.channels - 1
+generation_config.do_samples = [True] * model.channels
+generation_config.layers = [
+    {
+        "repetition_penalty": 1.0, 
+        "temperature": 1.5, 
+        "top_p": 1.0, 
+        "top_k": 50
+    }
+] + [ 
+    {
+        "repetition_penalty": 1.1, 
+        "temperature": 1.0, 
+        "top_p": 0.95,
+        "top_k": 50
+    }
+] * (model.channels - 1) 
+
 batch_size = 1
 
 save_dir = Path("inference_root_moss_tts_local_transformer_continuation")
@@ -462,7 +529,7 @@ with torch.no_grad():
         outputs = model.generate(
             input_ids=input_ids,
             attention_mask=attention_mask,
-            max_new_tokens=4096,
+            generation_config=generation_config
         )
 
         for message in processor.decode(outputs):
@@ -470,6 +537,7 @@ with torch.no_grad():
             out_path = save_dir / f"sample{sample_idx}.wav"
             sample_idx += 1
             torchaudio.save(out_path, audio.unsqueeze(0), processor.model_config.sampling_rate)
+
 ```
 
 
@@ -572,31 +640,30 @@ print(model_input_text)
 
 ## 3. Evaluation
 MOSS-TTS achieved state-of-the-art results on the open-source zero-shot TTS benchmark Seed-TTS-eval, not only surpassing all open-source models but also rivaling the most powerful closed-source models.
-| Model | Params | Open‑source | EN WER (%) ↓ | EN SIM (%) ↑ | ZH CER (%) ↓ | ZH SIM (%) ↑ |
+
+| Model | Params | Open-source | EN WER (%) ↓ | EN SIM (%) ↑ | ZH CER (%) ↓ | ZH SIM (%) ↑ |
 |---|---:|:---:|---:|---:|---:|---:|
 | DiTAR | 0.6B | ❌ | 1.69 | 73.5 | 1.02 | 75.3 |
-| FishAudio‑S1 | 4B | ❌ | 1.72 | 62.57 | 1.22 | 72.1 |
-| CosyVoice3 | 1.5B | ❌ | 2.22 | 72 | 1.12 | 78.1 |
-| Seed‑TTS |  | ❌ | 2.25 | 76.2 | 1.12 | 79.6 |
-| MiniMax‑Speech |  | ❌ | 1.65 | 69.2 | 0.83 | 78.3 |
+| FishAudio-S1 | 4B | ❌ | 1.72 | 62.57 | 1.22 | 72.1 |
+| Seed-TTS |  | ❌ | 2.25 | 76.2 | 1.12 | 79.6 |
+| MiniMax-Speech |  | ❌ | 1.65 | 69.2 | 0.83 | 78.3 |
 |  |  |  |  |  |  |  |
 | CosyVoice | 0.3B | ✅ | 4.29 | 60.9 | 3.63 | 72.3 |
 | CosyVoice2 | 0.5B | ✅ | 3.09 | 65.9 | 1.38 | 75.7 |
 | CosyVoice3 | 0.5B | ✅ | 2.02 | 71.8 | 1.16 | 78 |
-| F5‑TTS | 0.3B | ✅ | 2 | 67 | 1.53 | 76 |
+| CosyVoice3 | 1.5B | ✅ | 2.22 | 72 | 1.12 | 78.1 |
+| F5-TTS | 0.3B | ✅ | 2 | 67 | 1.53 | 76 |
 | SparkTTS | 0.5B | ✅ | 3.14 | 57.3 | 1.54 | 66 |
 | FireRedTTS | 0.5B | ✅ | 3.82 | 46 | 1.51 | 63.5 |
-| FireRedTTS‑2 | 1.5B | ✅ | 1.95 | 66.5 | 1.14 | 73.6 |
-| Qwen2.5‑Omni | 7B | ✅ | 2.72 | 63.2 | 1.7 | 75.2 |
-| FishAudio‑S1‑mini | 0.5B | ✅ | 1.94 | 55 | 1.18 | 68.5 |
+| FireRedTTS-2 | 1.5B | ✅ | 1.95 | 66.5 | 1.14 | 73.6 |
+| Qwen2.5-Omni | 7B | ✅ | 2.72 | 63.2 | 1.7 | 75.2 |
+| FishAudio-S1-mini | 0.5B | ✅ | 1.94 | 55 | 1.18 | 68.5 |
 | IndexTTS2 | 1.5B | ✅ | 2.23 | 70.6 | 1.03 | 76.5 |
 | VibeVoice | 1.5B | ✅ | 3.04 | 68.9 | 1.16 | 74.4 |
-| HiggsAudio‑v2 | 3B | ✅ | 2.44 | 67.7 | 1.5 | 74 |
-| GLM-TTS | 1.5B | ✅ | 2.23 | 67.2 | 1.03 | 76.1 |
-| GLM-TTS-RL | 1.5B | ✅ | 1.91 | 68.1 | **0.89** | 76.4 |
-| VoxCPM | 0.5B | ✅ | 1.85 | 72.9 | 0.93 | 77.2 |
-| Qwen3‑TTS | 0.6B | ✅ | 1.68 | 70.39 | 1.23 | 76.4 |
-| Qwen3‑TTS | 1.7B | ✅ | **1.5** | 71.45 | 1.33 | 76.72 |
+| HiggsAudio-v2 | 3B | ✅ | 2.44 | 67.7 | 1.5 | 74 |
+| VoxCPM | 0.5B | ✅ | 1.85 | 72.9 | **0.93** | 77.2 |
+| Qwen3-TTS | 0.6B | ✅ | 1.68 | 70.39 | 1.23 | 76.4 |
+| Qwen3-TTS | 1.7B | ✅ | **1.5** | 71.45 | 1.33 | 76.72 |
 |  |  |  |  |  |  |  |
-| **MossTTSDelay** | **8B** | ✅ | 1.84 | 70.86 | 1.37 | 76.98 |
-| **MossTTSLocal** | **1.7B** | ✅ | 1.93 | **73.28** | 1.44 | **79.62** |
+| MossTTSDelay | 8B | ✅ | 1.79 | 71.46 | 1.32 | 77.05 |
+| MossTTSLocal | 1.7B | ✅ | 1.85 | **73.42** | 1.2 | **78.82** |

modeling_moss_tts.py

@@ -616,109 +616,6 @@ class MossTTSDelayModel(MosiTTSPretrainedModel, CustomMixin):
     def can_generate(self):
         return True
 
-    def _build_generation_config(
-        self,
-        generation_config: Optional[GenerationConfig] = None,
-        max_new_tokens: Optional[int] = None,
-        text_temperature: Optional[float] = None,
-        text_top_p: Optional[float] = None,
-        text_top_k: Optional[int] = None,
-        text_repetition_penalty: Optional[float] = None,
-        audio_temperature: Optional[float] = None,
-        audio_top_p: Optional[float] = None,
-        audio_top_k: Optional[int] = None,
-        audio_repetition_penalty: Optional[float] = None,
-        n_vq_for_inference: Optional[int] = None,
-    ) -> GenerationConfig:
-        config = copy.deepcopy(generation_config or self.generation_config)
-
-        text_temperature = 1.5 if text_temperature is None else float(text_temperature)
-        text_top_p = 1.0 if text_top_p is None else float(text_top_p)
-        text_top_k = 50 if text_top_k is None else int(text_top_k)
-        text_repetition_penalty = 1.0 if text_repetition_penalty is None else float(text_repetition_penalty)
-        audio_temperature = 1.0 if audio_temperature is None else float(audio_temperature)
-        audio_top_p = 0.95 if audio_top_p is None else float(audio_top_p)
-        audio_top_k = 50 if audio_top_k is None else int(audio_top_k)
-        audio_repetition_penalty = 1.1 if audio_repetition_penalty is None else float(audio_repetition_penalty)
-
-        text_do_sample = text_temperature > 0
-        if not text_do_sample:
-            text_temperature = 1.0
-        audio_do_sample = audio_temperature > 0
-        if not audio_do_sample:
-            audio_temperature = 1.0
-
-        if max_new_tokens is not None:
-            config.max_new_tokens = int(max_new_tokens)
-        elif getattr(config, "max_new_tokens", None) is None:
-            config.max_new_tokens = 100000 # about 2.2 hours , can be overridden by user input, you can set to a smaller value for faster generation during debugging
-
-        if getattr(config, "pad_token_id", None) is None:
-            config.pad_token_id = self.config.pad_token_id
-        config.eos_token_id = self.config.audio_end_token_id
-        config.use_cache = True
-        config.do_sample = text_do_sample or audio_do_sample
-
-        resolved_n_vq = self.channels - 1 if n_vq_for_inference is None else int(n_vq_for_inference)
-        resolved_n_vq = max(1, min(self.channels - 1, resolved_n_vq))
-        config.n_vq_for_inference = resolved_n_vq
-        config.do_samples = [text_do_sample] + [audio_do_sample] * (self.channels - 1)
-        config.layers = [
-            {
-                "repetition_penalty": text_repetition_penalty,
-                "temperature": text_temperature,
-                "top_p": text_top_p,
-                "top_k": text_top_k,
-            }
-        ] + [
-            {
-                "repetition_penalty": audio_repetition_penalty,
-                "temperature": audio_temperature,
-                "top_p": audio_top_p,
-                "top_k": audio_top_k,
-            }
-            for _ in range(self.channels - 1)
-        ]
-        return config
-
-    @torch.inference_mode()
-    def generate(
-        self,
-        input_ids: torch.LongTensor,
-        attention_mask: Optional[torch.Tensor] = None,
-        generation_config: Optional[GenerationConfig] = None,
-        max_new_tokens: Optional[int] = None,
-        text_temperature: Optional[float] = None,
-        text_top_p: Optional[float] = None,
-        text_top_k: Optional[int] = None,
-        text_repetition_penalty: Optional[int] = None,
-        audio_temperature: Optional[float] = None,
-        audio_top_p: Optional[float] = None,
-        audio_top_k: Optional[int] = None,
-        audio_repetition_penalty: Optional[float] = None,
-        n_vq_for_inference: Optional[int] = None,
-        **kwargs,
-    ):
-        resolved_generation_config = self._build_generation_config(
-            generation_config=generation_config,
-            max_new_tokens=max_new_tokens,
-            text_temperature=text_temperature,
-            text_top_p=text_top_p,
-            text_top_k=text_top_k,
-            text_repetition_penalty=text_repetition_penalty,
-            audio_temperature=audio_temperature,
-            audio_top_p=audio_top_p,
-            audio_top_k=audio_top_k,
-            audio_repetition_penalty=audio_repetition_penalty,
-            n_vq_for_inference=n_vq_for_inference,
-        )
-        return super().generate(
-            input_ids=input_ids,
-            attention_mask=attention_mask,
-            generation_config=resolved_generation_config,
-            **kwargs,
-        )
-
     # def tie_weights(self):
     #     ...
         # for i in range(self.config.channels):

processing_moss_tts.py

@@ -621,7 +621,7 @@ class MossTTSDelayProcessor(ProcessorMixin):
                 prefix_idx = audio_end_idx
 
             if truncation:
-                ...
+                raise RuntimeError("Truncation generation is not supported at present")
             else:
                 last_audio_end_idx = int(audio_end_indices[-1].item())
                 pad_codes = torch.full(