Spaces:
Paused
Paused
update README demo
Browse files- README.md +164 -0
- docs/demo01.png +3 -0
- docs/demo02.png +3 -0
- docs/demo03.png +3 -0
- docs/demo04.png +3 -0
- docs/demo05_clone.png +3 -0
- docs/demo06_clone.png +3 -0
- docs/demo07_clone.png +3 -0
- docs/demo08_clone.png +3 -0
- docs/hf_spec.png +3 -0
README.md
CHANGED
|
@@ -39,6 +39,42 @@ Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gra
|
|
| 39 |
| **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
|
| 40 |
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
## Models
|
| 43 |
|
| 44 |
| Model | Role | Size |
|
|
@@ -58,6 +94,32 @@ The stock narration model and LFM Q&A model are preloaded at startup for fast re
|
|
| 58 |
- **TTS Backends:** Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
|
| 59 |
- **Q&A Backend:** LFM2.5-Audio for direct audio-in/audio-out answers with lower interaction latency
|
| 60 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
## Features
|
| 62 |
|
| 63 |
- π€ **Voice Cloning** β Record 15s, get a narrator that sounds like you
|
|
@@ -138,3 +200,105 @@ All inference runs on the Hugging Face Space GPU. Voice samples, story text, and
|
|
| 138 |
|
| 139 |
Hackathon project. Stories are public domain.
|
| 140 |
test push verification
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
| **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
|
| 40 |
| **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
|
| 41 |
|
| 42 |
+
## Application Demo
|
| 43 |
+
|
| 44 |
+
> Full step-by-step walkthrough: [docs/DEMO_WALKTHROUGH.md](docs/DEMO_WALKTHROUGH.md)
|
| 45 |
+
|
| 46 |
+
### Explore Space β Browse the Story Shelf
|
| 47 |
+
|
| 48 |
+

|
| 49 |
+
|
| 50 |
+
### My Library β Now Playing with Pause & Resume
|
| 51 |
+
|
| 52 |
+

|
| 53 |
+
|
| 54 |
+
### Voice Q&A β Ask a Question by Recording
|
| 55 |
+
|
| 56 |
+

|
| 57 |
+
|
| 58 |
+
### LFM2.5-Audio Answer β Spoken Response Generated Instantly
|
| 59 |
+
|
| 60 |
+

|
| 61 |
+
|
| 62 |
+
### Clone Voice Studio β Record Your Voice (10s minimum)
|
| 63 |
+
|
| 64 |
+

|
| 65 |
+
|
| 66 |
+
### Clone Voice Studio β Voice Embedding Extraction in Progress
|
| 67 |
+
|
| 68 |
+

|
| 69 |
+
|
| 70 |
+
### Clone Voice Studio β Cloning Complete, Preview Ready
|
| 71 |
+
|
| 72 |
+

|
| 73 |
+
|
| 74 |
+
### Narration Playing in Cloned Voice
|
| 75 |
+
|
| 76 |
+

|
| 77 |
+
|
| 78 |
## Models
|
| 79 |
|
| 80 |
| Model | Role | Size |
|
|
|
|
| 94 |
- **TTS Backends:** Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
|
| 95 |
- **Q&A Backend:** LFM2.5-Audio for direct audio-in/audio-out answers with lower interaction latency
|
| 96 |
|
| 97 |
+
## System Requirements
|
| 98 |
+
|
| 99 |
+
| GPU | VRAM | Status | Notes |
|
| 100 |
+
|---|---|---|---|
|
| 101 |
+
| Nvidia A10G (24 GB) | 24 GB | β
Recommended | All three models load comfortably; narration + Q&A run without memory pressure |
|
| 102 |
+
| Nvidia T4 (16 GB) | 16 GB | β οΈ Functional | Works but is tight β voice cloning model may OOM if loaded concurrently with LFM; restart may be needed between heavy operations |
|
| 103 |
+
| CPU only | β | β Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
|
| 104 |
+
|
| 105 |
+
**Hugging Face Spaces:**
|
| 106 |
+
- **A10G Small** β confirmed working, recommended for the hackathon demo
|
| 107 |
+
- **T4 Small** β runs with limitations; recommend disabling concurrent model loading
|
| 108 |
+
|
| 109 |
+

|
| 110 |
+
|
| 111 |
+
**Memory breakdown (approximate):**
|
| 112 |
+
|
| 113 |
+
| Model | VRAM (bfloat16) |
|
| 114 |
+
|---|---|
|
| 115 |
+
| LFM2.5-Audio-1.5B | ~3 GB |
|
| 116 |
+
| Qwen3-TTS-0.6B-CustomVoice | ~1.2 GB |
|
| 117 |
+
| Qwen3-TTS-1.7B-Base (cloning) | ~3.4 GB |
|
| 118 |
+
| KV cache + activations at inference | ~1β2 GB |
|
| 119 |
+
| **Total (all models loaded)** | **~8β9 GB** |
|
| 120 |
+
|
| 121 |
+
The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5β6 GB with only the narration and Q&A models preloaded.
|
| 122 |
+
|
| 123 |
## Features
|
| 124 |
|
| 125 |
- π€ **Voice Cloning** β Record 15s, get a narrator that sounds like you
|
|
|
|
| 200 |
|
| 201 |
Hackathon project. Stories are public domain.
|
| 202 |
test push verification
|
| 203 |
+
|
| 204 |
+
|
| 205 |
+
## Application Demo
|
| 206 |
+
|
| 207 |
+
# MomsVoice β Demo Walkthrough
|
| 208 |
+
|
| 209 |
+
Step-by-step guide through the app with screenshots.
|
| 210 |
+
|
| 211 |
+
---
|
| 212 |
+
|
| 213 |
+
## Step 1: Explore the Story Shelf
|
| 214 |
+
|
| 215 |
+

|
| 216 |
+
|
| 217 |
+
Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
|
| 218 |
+
|
| 219 |
+
The featured story highlights the current pick. All 10 stories are public domain classics sourced from Project Gutenberg.
|
| 220 |
+
|
| 221 |
+
---
|
| 222 |
+
|
| 223 |
+
## Step 2: Open My Library and Start Narration
|
| 224 |
+
|
| 225 |
+

|
| 226 |
+
|
| 227 |
+
Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
|
| 228 |
+
|
| 229 |
+
- Cover art and title
|
| 230 |
+
- Active narrator label (stock Vivian voice or your cloned voice)
|
| 231 |
+
- **Pause** / **Resume Story** buttons
|
| 232 |
+
- Story text scrolling in sync with narration
|
| 233 |
+
|
| 234 |
+
Narration starts immediately, playing one paragraph at a time using the TTS model. The player highlights the current paragraph as audio streams.
|
| 235 |
+
|
| 236 |
+
---
|
| 237 |
+
|
| 238 |
+
## Step 3: Ask a Question About the Story
|
| 239 |
+
|
| 240 |
+

|
| 241 |
+
|
| 242 |
+
Scroll down in the Library panel to find the **Ask About the Story** section. Click **β Ask a Question** β narration pauses automatically and the microphone starts recording.
|
| 243 |
+
|
| 244 |
+
Speak your question naturally (e.g. *"What did Puss in Boots do to help his master?"*). When you stop recording, the question is submitted automatically β no button press needed. You can also type a question in the text fallback field and press **Get Answer** manually.
|
| 245 |
+
|
| 246 |
+
---
|
| 247 |
+
|
| 248 |
+
## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
|
| 249 |
+
|
| 250 |
+

|
| 251 |
+
|
| 252 |
+
The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1β2 sentences, based only on the current story context.
|
| 253 |
+
|
| 254 |
+
The answer is delivered in two ways simultaneously:
|
| 255 |
+
- **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
|
| 256 |
+
- **Answer Audio** waveform auto-plays the spoken response
|
| 257 |
+
|
| 258 |
+
After the answer finishes, narration resumes from where it paused.
|
| 259 |
+
|
| 260 |
+
---
|
| 261 |
+
|
| 262 |
+
## Step 5: Open Clone Voice Studio and Record Your Voice
|
| 263 |
+
|
| 264 |
+

|
| 265 |
+
|
| 266 |
+
Go to the **Clone Voice Studio** tab. Fill in:
|
| 267 |
+
- **Voice Nickname** β a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
|
| 268 |
+
- **Voice Gender Avatar** β Female or Male
|
| 269 |
+
|
| 270 |
+
A story script is displayed to read aloud. Click the **Record** button and read the script naturally for at least 10 seconds. The waveform displays your voice in real time. Click **Stop** when done.
|
| 271 |
+
|
| 272 |
+
---
|
| 273 |
+
|
| 274 |
+
## Step 6: Analyze & Synthesize the Vocal Profile
|
| 275 |
+
|
| 276 |
+

|
| 277 |
+
|
| 278 |
+
Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
|
| 279 |
+
|
| 280 |
+
A progress bar shows synthesis percentage (e.g. *"Generating voice preview... 70.0%"*) with elapsed time. This step runs Qwen3-TTS-1.7B Base to extract the speaker embedding from your recording.
|
| 281 |
+
|
| 282 |
+
---
|
| 283 |
+
|
| 284 |
+
## Step 7: Voice Profile Ready
|
| 285 |
+
|
| 286 |
+

|
| 287 |
+
|
| 288 |
+
When synthesis completes, the Status Pipeline Visualizer shows:
|
| 289 |
+
- Your **avatar** and nickname with a **"Cloned successfully"** badge
|
| 290 |
+
- *"Synthesized today using Qwen3-TTS voice cloning"*
|
| 291 |
+
- A **preview audio player** β tap Play to hear a short sample in your cloned voice
|
| 292 |
+
- **"Synced successfully! Bind Companion Voice to Inventory"** confirmation
|
| 293 |
+
|
| 294 |
+
The profile is saved to disk and will persist across Space restarts.
|
| 295 |
+
|
| 296 |
+
---
|
| 297 |
+
|
| 298 |
+
## Step 8: Story Narration in Your Cloned Voice
|
| 299 |
+
|
| 300 |
+

|
| 301 |
+
|
| 302 |
+
Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β confirming the cloned voice is active.
|
| 303 |
+
|
| 304 |
+
Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β no audio leaves the server.
|
docs/demo01.png
ADDED
|
Git LFS Details
|
docs/demo02.png
ADDED
|
Git LFS Details
|
docs/demo03.png
ADDED
|
Git LFS Details
|
docs/demo04.png
ADDED
|
Git LFS Details
|
docs/demo05_clone.png
ADDED
|
Git LFS Details
|
docs/demo06_clone.png
ADDED
|
Git LFS Details
|
docs/demo07_clone.png
ADDED
|
Git LFS Details
|
docs/demo08_clone.png
ADDED
|
Git LFS Details
|
docs/hf_spec.png
ADDED
|
Git LFS Details
|