Duplicate from MisoLabs/MisoTTS
Browse filesCo-authored-by: Aoden Teo Masa Toshi <AodenT@users.noreply.huggingface.co>
- .gitattributes +35 -0
- LICENSE +27 -0
- README.md +89 -0
- model.safetensors +3 -0
- repo_banner.png +0 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Modified MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Kamino Learning, Inc.
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the “Software”), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
| 22 |
+
|
| 23 |
+
Our only modification part is that, if the Software (or any derivative works
|
| 24 |
+
thereof) is used for any of your commercial products or services that have
|
| 25 |
+
more than 50 million monthly active users, or more than 10 million US dollars
|
| 26 |
+
(or equivalent in other currencies) in monthly revenue, you shall prominently
|
| 27 |
+
display "Miso Labs" on the user interface of such product or service.
|
README.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: text-to-speech
|
| 5 |
+
tags:
|
| 6 |
+
- text-to-speech
|
| 7 |
+
- speech-synthesis
|
| 8 |
+
- voice
|
| 9 |
+
- audio
|
| 10 |
+
- sesame
|
| 11 |
+
- mimi
|
| 12 |
+
- llama
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
<div align="center">
|
| 16 |
+
|
| 17 |
+
<img src="repo_banner.png" alt="Miso TTS 8B" width="100%">
|
| 18 |
+
<p>
|
| 19 |
+
<a href="https://misolabs.ai"><img alt="Website" src="https://img.shields.io/badge/Website-misolabs.ai-black?style=for-the-badge"></a>
|
| 20 |
+
<a href="https://huggingface.co/MisoLabs/MisoTTS"><img alt="Hugging Face" src="https://img.shields.io/badge/Hugging%20Face-MisoTTS-yellow?style=for-the-badge"></a>
|
| 21 |
+
<a href="https://github.com/MisoLabsAI"><img alt="GitHub" src="https://img.shields.io/badge/GitHub-MisoLabsAI-181717?style=for-the-badge&logo=github&labelColor=555555"></a>
|
| 22 |
+
<a href="https://x.com/MisoLabsAI"><img alt="X" src="https://img.shields.io/badge/-MisoLabsAI-181717?style=for-the-badge&logo=x&labelColor=555555"></a>
|
| 23 |
+
</p>
|
| 24 |
+
|
| 25 |
+
<p>
|
| 26 |
+
<a href="#quickstart">Quickstart</a> |
|
| 27 |
+
<a href="#model-introduction">Model Introduction</a> |
|
| 28 |
+
<a href="#model-summary">Model Summary</a> |
|
| 29 |
+
<a href="#architecture">Architecture</a> |
|
| 30 |
+
<a href="#links">Links</a>
|
| 31 |
+
</p>
|
| 32 |
+
|
| 33 |
+
</div>
|
| 34 |
+
|
| 35 |
+
---
|
| 36 |
+
# Miso TTS 8B
|
| 37 |
+
## Model Introduction
|
| 38 |
+
|
| 39 |
+
Miso TTS 8B is a text-to-speech model based on the Sesame CSM architecture. It
|
| 40 |
+
generates Mimi audio codes from text and optional audio context, using a large
|
| 41 |
+
Llama 3.2-style backbone and a smaller autoregressive audio decoder.
|
| 42 |
+
|
| 43 |
+
The model is designed for high-quality conversational speech generation and
|
| 44 |
+
voice continuation from prompt audio. This repository contains the inference
|
| 45 |
+
code, model definition, and setup instructions for running Miso TTS locally.
|
| 46 |
+
|
| 47 |
+
---
|
| 48 |
+
|
| 49 |
+
## Quickstart
|
| 50 |
+
|
| 51 |
+
To run the model, use the inference code at our [public repository](https://github.com/MisoLabsAI/MisoTTS),
|
| 52 |
+
or try our demo at misolabs.ai.
|
| 53 |
+
|
| 54 |
+
## Model Summary
|
| 55 |
+
|
| 56 |
+
| Item | Value |
|
| 57 |
+
| ------------------- | ---------------- |
|
| 58 |
+
| Model | Miso TTS 8B |
|
| 59 |
+
| Organization | Miso Labs |
|
| 60 |
+
| Task | Text-to-speech |
|
| 61 |
+
| Architecture | Sesame-style CSM |
|
| 62 |
+
| Backbone | `llama-8B` |
|
| 63 |
+
| Audio decoder | `llama-300M` |
|
| 64 |
+
| Text vocabulary | `128,256` |
|
| 65 |
+
| Audio vocabulary | `2,051` |
|
| 66 |
+
| Audio codebooks | `32` |
|
| 67 |
+
| Audio tokenizer | Mimi |
|
| 68 |
+
| Max sequence length | `2,048` |
|
| 69 |
+
|
| 70 |
+
### Architecture
|
| 71 |
+
|
| 72 |
+
Miso TTS 8B uses two transformer components:
|
| 73 |
+
|
| 74 |
+
- A large backbone transformer that consumes text/audio-frame embeddings.
|
| 75 |
+
- A smaller decoder transformer that autoregressively predicts higher-order
|
| 76 |
+
audio codebooks within each frame.
|
| 77 |
+
|
| 78 |
+
Codebook 0 is
|
| 79 |
+
predicted from the backbone hidden state, while codebooks 1 through 31 are
|
| 80 |
+
predicted by the audio decoder autoregressively in codebook depth.
|
| 81 |
+
|
| 82 |
+
---
|
| 83 |
+
|
| 84 |
+
## Links
|
| 85 |
+
|
| 86 |
+
- Website: [misolabs.ai](https://misolabs.ai)
|
| 87 |
+
- Hugging Face: [MisoLabs/MisoTTS](https://huggingface.co/MisoLabs/MisoTTS)
|
| 88 |
+
- GitHub: [MisoLabsAI](https://github.com/MisoLabsAI)
|
| 89 |
+
- X: [@MisoLabsAI](https://x.com/MisoLabsAI)
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5454494e43d35dd5ec5692f90467c2db09831f8079300dc04faab15b4fb8e47a
|
| 3 |
+
size 32752529792
|
repo_banner.png
ADDED
|