sshtel commited on
Commit
04f7e9c
Β·
1 Parent(s): 7257098

update README demo

Browse files
README.md CHANGED
@@ -39,6 +39,42 @@ Built for the Hugging Face Hackathon. Runs entirely on local models inside a Gra
39
  | **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
40
  | **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
41
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
  ## Models
43
 
44
  | Model | Role | Size |
@@ -58,6 +94,32 @@ The stock narration model and LFM Q&A model are preloaded at startup for fast re
58
  - **TTS Backends:** Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
59
  - **Q&A Backend:** LFM2.5-Audio for direct audio-in/audio-out answers with lower interaction latency
60
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
  ## Features
62
 
63
  - 🎀 **Voice Cloning** β€” Record 15s, get a narrator that sounds like you
@@ -138,3 +200,105 @@ All inference runs on the Hugging Face Space GPU. Voice samples, story text, and
138
 
139
  Hackathon project. Stories are public domain.
140
  test push verification
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  | **Answer** | A short grounded answer is generated from the story context and spoken by the low-latency Q&A audio model. It does not use the cloned narrator voice. | LFM2.5-Audio-1.5B |
40
  | **Resume** | Story continues from where it left off. Play also pauses on tab switch. | Cached chunks |
41
 
42
+ ## Application Demo
43
+
44
+ > Full step-by-step walkthrough: [docs/DEMO_WALKTHROUGH.md](docs/DEMO_WALKTHROUGH.md)
45
+
46
+ ### Explore Space β€” Browse the Story Shelf
47
+
48
+ ![Explore Space](docs/demo01.png)
49
+
50
+ ### My Library β€” Now Playing with Pause & Resume
51
+
52
+ ![My Library Now Playing](docs/demo02.png)
53
+
54
+ ### Voice Q&A β€” Ask a Question by Recording
55
+
56
+ ![Voice Q&A Panel](docs/demo03.png)
57
+
58
+ ### LFM2.5-Audio Answer β€” Spoken Response Generated Instantly
59
+
60
+ ![LFM Answer Audio](docs/demo04.png)
61
+
62
+ ### Clone Voice Studio β€” Record Your Voice (10s minimum)
63
+
64
+ ![Clone Studio Recording](docs/demo05_clone.png)
65
+
66
+ ### Clone Voice Studio β€” Voice Embedding Extraction in Progress
67
+
68
+ ![Clone Studio Processing](docs/demo06_clone.png)
69
+
70
+ ### Clone Voice Studio β€” Cloning Complete, Preview Ready
71
+
72
+ ![Clone Studio Complete](docs/demo07_clone.png)
73
+
74
+ ### Narration Playing in Cloned Voice
75
+
76
+ ![Narration with Cloned Voice](docs/demo08_clone.png)
77
+
78
  ## Models
79
 
80
  | Model | Role | Size |
 
94
  - **TTS Backends:** Qwen3-TTS Base (cloned voice) + CustomVoice 0.6B (stock fallback)
95
  - **Q&A Backend:** LFM2.5-Audio for direct audio-in/audio-out answers with lower interaction latency
96
 
97
+ ## System Requirements
98
+
99
+ | GPU | VRAM | Status | Notes |
100
+ |---|---|---|---|
101
+ | Nvidia A10G (24 GB) | 24 GB | βœ… Recommended | All three models load comfortably; narration + Q&A run without memory pressure |
102
+ | Nvidia T4 (16 GB) | 16 GB | ⚠️ Functional | Works but is tight β€” voice cloning model may OOM if loaded concurrently with LFM; restart may be needed between heavy operations |
103
+ | CPU only | β€” | ❌ Not viable | LFM2.5-Audio and Qwen3-TTS inference is too slow for real-time use |
104
+
105
+ **Hugging Face Spaces:**
106
+ - **A10G Small** β€” confirmed working, recommended for the hackathon demo
107
+ - **T4 Small** β€” runs with limitations; recommend disabling concurrent model loading
108
+
109
+ ![HuggingFace Space Hardware Options](docs/hf_spec.png)
110
+
111
+ **Memory breakdown (approximate):**
112
+
113
+ | Model | VRAM (bfloat16) |
114
+ |---|---|
115
+ | LFM2.5-Audio-1.5B | ~3 GB |
116
+ | Qwen3-TTS-0.6B-CustomVoice | ~1.2 GB |
117
+ | Qwen3-TTS-1.7B-Base (cloning) | ~3.4 GB |
118
+ | KV cache + activations at inference | ~1–2 GB |
119
+ | **Total (all models loaded)** | **~8–9 GB** |
120
+
121
+ The 1.7B cloning model is loaded on demand (not at startup), so idle VRAM usage is ~5–6 GB with only the narration and Q&A models preloaded.
122
+
123
  ## Features
124
 
125
  - 🎀 **Voice Cloning** β€” Record 15s, get a narrator that sounds like you
 
200
 
201
  Hackathon project. Stories are public domain.
202
  test push verification
203
+
204
+
205
+ ## Application Demo
206
+
207
+ # MomsVoice β€” Demo Walkthrough
208
+
209
+ Step-by-step guide through the app with screenshots.
210
+
211
+ ---
212
+
213
+ ## Step 1: Explore the Story Shelf
214
+
215
+ ![Explore Space](docs/demo01.png)
216
+
217
+ Open the **Explore Space** tab to browse available stories displayed as warm bookshelf cards. Each card shows the cover art, title, author, and a short excerpt. Click **"Tap to Play"** on any card to open it in My Library, or click **"More Info"** for the full story description.
218
+
219
+ The featured story highlights the current pick. All 10 stories are public domain classics sourced from Project Gutenberg.
220
+
221
+ ---
222
+
223
+ ## Step 2: Open My Library and Start Narration
224
+
225
+ ![My Library β€” Now Playing](docs/demo02.png)
226
+
227
+ Switch to the **My Library** tab. Click any story card to load it into the **Now Playing** panel on the right. The panel shows:
228
+
229
+ - Cover art and title
230
+ - Active narrator label (stock Vivian voice or your cloned voice)
231
+ - **Pause** / **Resume Story** buttons
232
+ - Story text scrolling in sync with narration
233
+
234
+ Narration starts immediately, playing one paragraph at a time using the TTS model. The player highlights the current paragraph as audio streams.
235
+
236
+ ---
237
+
238
+ ## Step 3: Ask a Question About the Story
239
+
240
+ ![Voice Q&A Panel](docs/demo03.png)
241
+
242
+ Scroll down in the Library panel to find the **Ask About the Story** section. Click **❓ Ask a Question** β€” narration pauses automatically and the microphone starts recording.
243
+
244
+ Speak your question naturally (e.g. *"What did Puss in Boots do to help his master?"*). When you stop recording, the question is submitted automatically β€” no button press needed. You can also type a question in the text fallback field and press **Get Answer** manually.
245
+
246
+ ---
247
+
248
+ ## Step 4: Hear the Spoken Answer (LFM2.5-Audio)
249
+
250
+ ![LFM Answer Audio](docs/demo04.png)
251
+
252
+ The recorded question waveform appears in the audio widget. Below it, the **LFM2.5-Audio** model generates a grounded answer in 1–2 sentences, based only on the current story context.
253
+
254
+ The answer is delivered in two ways simultaneously:
255
+ - **Answer text** shown inline (e.g. *"Puss in Boots, a poor man, bargained for boots and a bag, and Grimalkin, ailient rabbit, rescued him from a king's wrath."*)
256
+ - **Answer Audio** waveform auto-plays the spoken response
257
+
258
+ After the answer finishes, narration resumes from where it paused.
259
+
260
+ ---
261
+
262
+ ## Step 5: Open Clone Voice Studio and Record Your Voice
263
+
264
+ ![Clone Studio β€” Recording](docs/demo05_clone.png)
265
+
266
+ Go to the **Clone Voice Studio** tab. Fill in:
267
+ - **Voice Nickname** β€” a name for this profile (e.g. *"Mom"*, *"Grandpa Joe"*)
268
+ - **Voice Gender Avatar** β€” Female or Male
269
+
270
+ A story script is displayed to read aloud. Click the **Record** button and read the script naturally for at least 10 seconds. The waveform displays your voice in real time. Click **Stop** when done.
271
+
272
+ ---
273
+
274
+ ## Step 6: Analyze & Synthesize the Vocal Profile
275
+
276
+ ![Clone Studio β€” Processing](docs/demo06_clone.png)
277
+
278
+ Click **Analyze & Synthesize Customized Vocal Profile**. The **Status Pipeline Visualizer** on the right tracks progress through extraction, de-noising, and pitch tuning stages.
279
+
280
+ A progress bar shows synthesis percentage (e.g. *"Generating voice preview... 70.0%"*) with elapsed time. This step runs Qwen3-TTS-1.7B Base to extract the speaker embedding from your recording.
281
+
282
+ ---
283
+
284
+ ## Step 7: Voice Profile Ready
285
+
286
+ ![Clone Studio β€” Complete](docs/demo07_clone.png)
287
+
288
+ When synthesis completes, the Status Pipeline Visualizer shows:
289
+ - Your **avatar** and nickname with a **"Cloned successfully"** badge
290
+ - *"Synthesized today using Qwen3-TTS voice cloning"*
291
+ - A **preview audio player** β€” tap Play to hear a short sample in your cloned voice
292
+ - **"Synced successfully! Bind Companion Voice to Inventory"** confirmation
293
+
294
+ The profile is saved to disk and will persist across Space restarts.
295
+
296
+ ---
297
+
298
+ ## Step 8: Story Narration in Your Cloned Voice
299
+
300
+ ![Narration with Cloned Voice](docs/demo08_clone.png)
301
+
302
+ Return to **My Library** and select a story. The Now Playing panel shows **Active Narrator: [your nickname]** β€” confirming the cloned voice is active.
303
+
304
+ Narration plays in your recorded voice, paragraph by paragraph (*"Paragraph 1 / 32"*). The story text scrolls in the reader below the player. All TTS inference runs locally on the Space GPU β€” no audio leaves the server.
docs/demo01.png ADDED

Git LFS Details

  • SHA256: 113dea5706008517952947d2af43c469dea1be56a199e98c5ea7eaf75d957b7d
  • Pointer size: 131 Bytes
  • Size of remote file: 740 kB
docs/demo02.png ADDED

Git LFS Details

  • SHA256: 51055da15434eaec327c03ec01f7d73ac7e96126efd0fcc652f1e8b7c9ec7e34
  • Pointer size: 132 Bytes
  • Size of remote file: 1.15 MB
docs/demo03.png ADDED

Git LFS Details

  • SHA256: 6bd5e50b0c8fcbbc6a83ca237ab78ba199d56a83626c1ce293a267003b67bab4
  • Pointer size: 131 Bytes
  • Size of remote file: 778 kB
docs/demo04.png ADDED

Git LFS Details

  • SHA256: eb03260e4ecc0ddb28d3ed7b1bf198837c53ed3f7715c40f9f96800ae70cc95b
  • Pointer size: 130 Bytes
  • Size of remote file: 71.8 kB
docs/demo05_clone.png ADDED

Git LFS Details

  • SHA256: 6ce2c8fd8164ec38df3b8ed9d9cf589946a6558810f09e88a4b7105ddfb1fa22
  • Pointer size: 131 Bytes
  • Size of remote file: 131 kB
docs/demo06_clone.png ADDED

Git LFS Details

  • SHA256: 1fc6ce5cc89b4e4fc2d54c567ff45a804dabfdb9ff8983c46b321a473f5c8368
  • Pointer size: 131 Bytes
  • Size of remote file: 125 kB
docs/demo07_clone.png ADDED

Git LFS Details

  • SHA256: dbf072df6465da4847b07c5f1f12786614740f3bb15f3312b50e9078a1e919b4
  • Pointer size: 131 Bytes
  • Size of remote file: 137 kB
docs/demo08_clone.png ADDED

Git LFS Details

  • SHA256: def199cccf184b6d94c037822d0a09ddce8d2e32bbc40bd955a9f06e91368b7b
  • Pointer size: 131 Bytes
  • Size of remote file: 125 kB
docs/hf_spec.png ADDED

Git LFS Details

  • SHA256: 1c1682fd39dbd73a574d7f0a230ca358e6057df13c4f7aa400f884e64e3b643c
  • Pointer size: 131 Bytes
  • Size of remote file: 296 kB