Add interruptible chunked streaming narration + mobile considerations PRD

future_mobile_app_considerations.md

@@ -0,0 +1,142 @@
+# Future Mobile App Considerations
+
+## Purpose
+
+This note captures future mobile deployment guidance for ReadBookMom. The current hackathon plan should remain a Hugging Face Spaces demo, with mobile treated as a later product direction.
+
+## Recommendation
+
+Do not plan the first mobile version as a fully on-device app running all models locally. A hybrid mobile app is more realistic for reasonable latency, battery life, interruption behavior, and device coverage.
+
+For the mobile app, use the phone as the recording, playback, interruption, and story interaction surface. Keep cloned-voice TTS and Q&A generation on a GPU backend until the models and runtime stack are proven on target devices.
+
+The mobile version should inherit the current PRD direction: interruptible chunked streaming with a tap-to-Ask flow. The first mobile product should not attempt always-listening voice barge-in.
+
+## Model Fit For Mobile
+
+| Model | Mobile Fit | Notes |
+|---|---|---|
+| Qwen2.5-3B-Instruct | Possible on high-end phones only | With 4-bit quantization, it may fit on flagship devices, but latency, RAM pressure, battery drain, and thermal throttling will vary by device and runtime. |
+| Whisper-small | Borderline | Better suited to server or desktop-class environments. For mobile latency, prefer native iOS/Android speech APIs, Whisper-tiny, or Whisper-base. |
+| QWEN-TTS-0.6B | Hardest to deploy locally | Voice cloning TTS adds audio pipeline cost beyond parameter count. On-device latency may be poor unless a mobile-optimized runtime path is available. |
+
+## Mobile Interaction Model
+
+The mobile app should behave like a smart audiobook player:
+
+```text
+Story chunk is playing
+-> child taps Ask
+-> app pauses local playback immediately
+-> question is captured as text or voice
+-> backend generates a short grounded answer
+-> answer audio plays in the cloned voice
+-> story resumes from the saved position
+```
+
+Button-based interruption is the recommended first version. True voice barge-in, where the child talks over narration and the app detects it automatically, should remain a later research item because it requires echo cancellation, voice activity detection, full-duplex audio handling, and careful privacy UX.
+
+## Mobile Architecture
+
+| Layer | Responsibility | Recommendation |
+|---|---|---|
+| Mobile app | Recording, playback, Ask interruption, local state, local cache | Native iOS/Android app or React Native/Flutter with native audio modules. |
+| Backend API | Voice profile session, story chunk generation, Q&A answer generation, answer TTS | GPU backend using the same model stack as the Space demo. |
+| Streaming/control channel | Request chunks, pause/resume, cancel queued work, receive generation status | WebSocket for bidirectional control; REST plus polling is acceptable for a prototype. |
+| Audio delivery | Stream or download generated chunks | Prefer short chunk files with signed URLs and local prefetch cache. |
+| Session state | Track story and interruption position | Store `story_id`, `voice_session_id`, `current_chunk_index`, playback state, and cache keys. |
+
+The phone should own playback responsiveness. The server can cancel or deprioritize queued generation, but the app should pause current audio locally without waiting for backend confirmation.
+
+## Deployment Options
+
+### Option 1: Cloud-Backed Mobile App
+
+The mobile app handles recording, story selection, interruption, playback, and local chunk caching. ASR, Q&A, and cloned-voice TTS run on a GPU backend.
+
+**Best for:** first production prototype, hackathon follow-up, broad device support.
+
+**Pros:** fastest path to reliable model latency, easier model updates, works across more phones.
+
+**Cons:** requires backend hosting, network connectivity, and clear privacy messaging.
+
+### Option 2: Hybrid Mobile App
+
+The app uses native or lightweight on-device ASR for the child's question, while Q&A and voice cloning TTS run on a backend. The phone manages interruptible chunk playback, local pause/resume, and cached story chunks.
+
+**Best for:** better perceived latency while keeping heavy voice generation off-device.
+
+**Pros:** practical performance, less backend work for transcription, smoother UX, and faster interruption response.
+
+**Cons:** still depends on server-side TTS and model hosting.
+
+### Option 3: Fully On-Device App
+
+The phone runs ASR, Q&A, voice cloning TTS, chunk generation, and playback locally.
+
+**Best for:** long-term privacy-first product direction on selected flagship devices.
+
+**Pros:** strongest offline and privacy story.
+
+**Cons:** highest engineering risk; likely issues with memory, heat, battery drain, app size, cold start, audio session complexity, and inconsistent latency across devices.
+
+## Suggested Mobile Stack
+
+| Capability | Near-Term Mobile Choice | Long-Term On-Device Candidate |
+|---|---|---|
+| Q&A | Backend `Qwen/Qwen2.5-3B-Instruct` | Quantized `Qwen2.5-1.5B-Instruct` or quantized `Qwen2.5-3B-Instruct` on flagship devices |
+| ASR | Native iOS/Android speech APIs or backend Whisper | Whisper-tiny/base through a mobile-optimized runtime |
+| Voice cloning TTS | Backend QWEN-TTS-0.6B | Mobile-optimized TTS/voice clone model if quality and latency are proven |
+| Story narration | Backend chunk generation with mobile playback and local chunk cache | Pre-generate and store locally after first generation |
+| Story Q&A audio | Generate short answers on demand | On-device Q&A plus server or local lightweight TTS |
+| Interruption | Tap-to-Ask pauses local playback immediately | Voice barge-in after echo cancellation and VAD are proven |
+
+## Latency Expectations
+
+| Flow | Target | Mobile Strategy |
+|---|---|---|
+| Story start | First chunk starts in 3-6 seconds on a good network | Generate the first chunk server-side, then prefetch the next chunks. |
+| Narration interruption | Local playback pauses within 500ms after Ask tap | Pause on-device immediately; backend cancellation is best effort. |
+| Child question transcription | 1-2 seconds | Prefer native mobile speech APIs or a tiny ASR model. |
+| Q&A text answer | 1-3 seconds | Use short prompts, 1-2 sentence answers, and backend Qwen2.5-3B-Instruct. |
+| Spoken Q&A answer | Starts in 4-8 seconds after question submit | Keep answer text short and generate TTS server-side. |
+| Story resume | Under 1 second when next chunk is cached | Resume from saved chunk position; fetch missing chunks in the background. |
+| Cache miss resume | 2-5 seconds | Show a lightweight loading state while the next chunk is generated or downloaded. |
+
+Mobile latency depends heavily on network quality. The app should show resilient playback states instead of assuming the backend will always meet desktop demo timing.
+
+## Mobile Constraints
+
+| Constraint | Product Impact | Recommendation |
+|---|---|---|
+| Network jitter | Chunks may arrive late or out of order. | Prefetch the next 1-2 chunks and keep a small playback buffer. |
+| Battery and thermals | Continuous local ASR or on-device LLM/TTS can drain battery quickly. | Keep heavy models server-side for v1; avoid always-on microphone. |
+| Audio session complexity | Recording while audio is playing is fragile on mobile. | Pause playback before recording the question; resume after answer playback. |
+| Echo cancellation | Open-mic interruption can capture the story audio instead of the child. | Use tap-to-Ask first; research VAD and echo cancellation later. |
+| App lifecycle | Incoming calls, lock screen, and backgrounding can interrupt playback. | Treat v1 as foreground-first; persist playback state and chunk cache. |
+| Storage | Cached voice/story audio can grow quickly. | Cache by voice session, story ID, chunk index, and model version with TTL cleanup. |
+| Privacy | Family voice data and child questions are sensitive. | Make upload, retention, deletion, and cache behavior explicit. |
+
+## Product Implications
+
+- The first mobile app should be a hybrid app, not a full local AI runtime.
+- The phone should own playback state and interruption responsiveness.
+- Privacy messaging must be explicit: explain what audio is sent to the backend, how long it is retained, and whether generated voice samples are cached.
+- If offline mode becomes a requirement, treat it as a separate research milestone, not a launch feature.
+- Device qualification matters. Fully on-device AI should start with a short list of supported flagship phones.
+- Chunk prefetching and caching are key to making bedtime story playback and resume feel instant.
+- Always-listening barge-in should not be promised until echo cancellation, VAD, and mobile privacy UX are proven.
+
+## Future Research Tasks
+
+1. Benchmark quantized Qwen2.5-1.5B-Instruct and Qwen2.5-3B-Instruct on representative iOS and Android devices.
+2. Compare native speech APIs with Whisper-tiny/base for child speech accuracy and latency.
+3. Test whether QWEN-TTS-0.6B has a viable mobile runtime path or whether server-side TTS remains required.
+4. Prototype backend chunk generation and mobile prefetch cache keyed by story ID, voice profile, chunk index, and model version.
+5. Prototype tap-to-Ask interruption with local pause, backend cancellation/deprioritization, answer playback, and story resume.
+6. Measure behavior under poor mobile networks, app backgrounding, and audio session interruptions.
+7. Define privacy and retention policy before any real family audio is stored.
+
+## Decision
+
+For the next phase, keep the model stack server-side and design the mobile app as a polished hybrid client with native playback, local chunk cache, tap-to-Ask interruption, and resume state. Revisit fully on-device deployment only after benchmarking smaller Q&A models, mobile ASR options, mobile-ready voice cloning TTS, and reliable barge-in audio handling.

mission.md

@@ -2,26 +2,32 @@
 
 ## Product Vision
 
-**ReadBookMom** — a parent reads a bedtime story in their own cloned voice, powered by a small local model. Built as a Gradio app on Hugging Face Spaces.
+**ReadBookMom** — a parent records a short voice sample, then a bedtime story and story Q&A are spoken back in that parent's cloned voice. The demo runs on local models inside a Gradio app on Hugging Face Spaces.
 
 ## Problem
 
-Parents can't always be there at bedtime. Children want to hear *their parent's voice* reading to them. Existing TTS is generic and impersonal. Voice cloning APIs are expensive and require sending private audio to third parties.
+Parents can't always be there at bedtime. Children want to hear *their parent's voice* reading to them. Existing TTS is generic and impersonal, while voice cloning APIs are expensive and require sending private family audio to third parties.
 
 ## Demo Loop
 
 ```
-Parent records 15s of voice → Pick a story → Child hears it in parent's voice
+Parent records 15s of voice → Pick a story → Story starts in parent's voice
                                     ↓
-                        Child asks a question → Hears answer in parent's voice
+                    Child taps Ask mid-story → Narration pauses
+                                    ↓
+                  Child asks a question → Hears answer in parent's voice
+                                    ↓
+                         Resume story from the same position
 ```
 
+Latency and natural interruption are part of the demo promise: the story should start quickly, pause cleanly when the child wants to ask something, answer in the cloned voice, then resume without losing the story position.
+
 ## Hackathon Goal (2 days)
 
 Ship a working Hugging Face Space that demonstrates:
 1. **Voice cloning** — parent uploads/records a short audio clip
-2. **Story narration** — QWEN-TTS-0.6B reads a children's story in the cloned voice
-3. **Voice Q&A** — child asks a question about the story, gets an answer spoken in the cloned voice
+2. **Interruptible story narration** — QWEN-TTS-0.6B streams paragraph-sized chunks in the cloned voice
+3. **Voice Q&A during narration** — child taps Ask, narration pauses, Qwen2.5-3B-Instruct answers from the story context, then QWEN-TTS-0.6B speaks the answer in the cloned voice
 
 ## Target Demo User
 
@@ -32,32 +38,59 @@ A parent at a laptop who records their voice, picks a short story, and shows the
 | Criteria | Target |
 |---|---|
 | Voice clone from ≤ 30s audio | Recognizable similarity |
-| Story narration latency | First audio chunk in ≤ 5s |
-| Q&A answer (spoken) | Response in ≤ 8s |
+| Story start latency | First streamed audio chunk in ≤ 5s |
+| Narration interruption | Ask tap pauses playback in ≤ 500ms and preserves story position |
+| Cached story replay | Starts immediately after first generation |
+| Q&A answer (spoken) | Spoken answer starts in ≤ 8s from question submit |
+| Story resume latency | Resume from paused position in ≤ 1s when next chunk is cached |
 | Works on HF Spaces | Public link, no local setup needed |
 | Demo length | 3-minute live walkthrough |
+| Privacy posture | Voice and Q&A stay on the Space runtime |
 
 ## Scope
 
 **In scope:**
 - Upload/record parent voice sample (15–30s)
 - Select from 3–5 pre-loaded short stories (public domain)
-- Play story narrated in cloned voice
-- Ask a question about the story, hear answer in cloned voice
+- Play story narrated in cloned voice with interruptible chunked streaming
+- Pause narration through an Ask button, preserve the current story chunk, answer, then resume
+- Cache generated narration per voice session and story
+- Ask a question about the story, receive a grounded 1–2 sentence Qwen2.5-3B-Instruct answer, and hear it in the cloned voice
 - Clean, child-friendly UI (Google Stitch-inspired via gr.Server)
 
 **Out of scope:**
 - User accounts, auth, database
 - Offline mode, progress tracking
 - Multiple languages
-- Real-time streaming (batch generation is fine)
+- Always-listening voice barge-in, echo cancellation, and open-mic interruption
 - COPPA compliance (demo only)
 
 ## Guardrails (Lite)
 
 | Constraint | Implementation |
 |---|---|
-| Content grounding | Q&A answers sourced from story text only (system prompt) |
+| Content grounding | Qwen2.5-3B-Instruct receives the selected story text plus a strict answer-from-story instruction |
 | Voice privacy | All inference local on HF Space GPU — no audio leaves the server |
 | Child safety | Pre-curated stories only; no user-uploaded content |
 
+## Latency Strategy
+
+| Bottleneck | Strategy |
+|---|---|
+| Voice setup | Compute and cache the voice representation immediately after recording. |
+| Story narration | Use interruptible chunked streaming: generate the first paragraph chunk, play it immediately, then continue generating queued chunks. |
+| Interruption | Add an Ask state that pauses playback, cancels or deprioritizes queued narration generation, and stores the current chunk index. |
+| Story replay | Cache full generated narration by voice session and story ID. |
+| Q&A context | Send the current story position plus the most relevant story passages to Qwen2.5-3B-Instruct instead of the full story when possible. |
+| Q&A length | Cap answers to 1–2 short child-friendly sentences before sending to TTS. |
+| Story resume | Resume from the paused chunk after the spoken answer; use cached next chunks when available. |
+| Voice question input | Load ASR only when audio questions are used; text questions bypass ASR. |
+
+## Design Review Notes
+
+| Topic | Critique | Upgrade |
+|---|---|---|
+| Privacy | An external LLM API would weaken the privacy story even if audio stayed local. | Use Qwen2.5-3B-Instruct for local story Q&A so the demo has one clear privacy narrative. |
+| Latency | A 3B-class Q&A model is a better fit for the demo loop than an 8B-class model, but TTS and interruption responsiveness remain the user-visible bottlenecks. | Cache voice setup, stream chunked narration, make playback cancellable, cap Q&A output, and use 4-bit or 8-bit loading on T4. |
+| Demo clarity | Voice cloning, narration, ASR, and Q&A can feel like too many moving parts. | Present the loop as three simple actions: record, listen, ask. |
+

sprint.md

@@ -1,7 +1,7 @@
 # Sprint Plan — 2-Day Hackathon
 
 ## Goal
-Ship a public Hugging Face Space: parent clones voice → child hears story + Q&A in that voice.
+Ship a public Hugging Face Space: parent clones voice → story streams in that voice → child can interrupt, ask, hear the answer, and resume.
 
 ---
 
@@ -13,9 +13,9 @@ Ship a public Hugging Face Space: parent clones voice → child hears story + Q&
 |---|---|---|---|
 | 1 | Set up repo: `app.py`, `requirements.txt`, `stories/` | 30m | ☐ |
 | 2 | Load QWEN-TTS-0.6B locally, test basic TTS (text → audio) | 1h | ☐ |
-| 3 | Implement voice cloning: reference audio + text → cloned speech | 1.5h | ☐ |
+| 3 | Implement voice cloning and cache voice representation after recording | 1.5h | ☐ |
 | 4 | Add 3 short stories as `.txt` files (public domain) | 30m | ☐ |
-| 5 | Wire up: pick story → generate narration in cloned voice | 30m | ☐ |
+| 5 | Wire up: pick story → stream first narration chunk, track chunk index, cache story audio | 30m | ☐ |
 
 **Checkpoint:** Can generate a story audio file in a cloned voice from CLI.
 
@@ -25,11 +25,11 @@ Ship a public Hugging Face Space: parent clones voice → child hears story + Q&
 |---|---|---|---|
 | 6 | Build Gradio app with 3 tabs (Clone, Listen, Ask) | 1h | ☐ |
 | 7 | Tab 1 (Clone): `gr.Audio` record/upload + preview button | 45m | ☐ |
-| 8 | Tab 2 (Listen): story dropdown + play narration button | 45m | ☐ |
-| 9 | Add Whisper-small for child voice input (ASR) | 30m | ☐ |
-| 10 | Tab 3 (Ask): voice/text input → GPT-4o-mini answer → TTS in cloned voice | 1h | ☐ |
+| 8 | Tab 2 (Listen): story dropdown + play/pause/resume controls for streamed chunks | 45m | ☐ |
+| 9 | Add on-demand ASR for child voice input; use lighter ASR fallback if needed | 30m | ☐ |
+| 10 | Tab 3 (Ask): interrupt narration → short grounded Qwen answer → TTS → resume story | 1h | ☐ |
 
-**Checkpoint:** Full loop works locally — clone → listen → ask. Ugly but functional.
+**Checkpoint:** Full loop works locally — clone → listen → interrupt → ask → resume. Ugly but functional.
 
 ---
 
@@ -43,7 +43,7 @@ Ship a public Hugging Face Space: parent clones voice → child hears story + Q&
 | 12 | Stitch-style CSS: color palette, card layout, rounded corners, fonts | 1h | ☐ |
 | 13 | Recording UX: waveform animation, countdown timer, status indicators | 1h | ☐ |
 | 14 | Story gallery: cover images (AI-generated or placeholder), card grid | 45m | ☐ |
-| 15 | Loading states: spinner/progress during TTS generation | 30m | ☐ |
+| 15 | Loading states: playback status, paused state, TTS progress, and background chunk generation | 30m | ☐ |
 | 16 | Mobile-responsive check (parents use laptops, demo on projector) | 15m | ☐ |
 
 **Checkpoint:** App looks polished and professional. Ready for live demo.
@@ -52,10 +52,10 @@ Ship a public Hugging Face Space: parent clones voice → child hears story + Q&
 
 | # | Task | Time | Done |
 |---|---|---|---|
-| 17 | Create HF Space (GPU T4), push code, verify model downloads | 1h | ☐ |
-| 18 | Set `OPENAI_API_KEY` in Space secrets | 5m | ☐ |
-| 19 | End-to-end test on live Space (clone → listen → ask) | 30m | ☐ |
-| 20 | Fix latency issues: cache model, preload weights at startup | 45m | ☐ |
+| 17 | Create HF Space (T4 with 4-bit/8-bit loading, or A10G for headroom), push code, verify model downloads | 1h | ☐ |
+| 18 | Confirm no external LLM API secrets are required; set `HF_TOKEN` only if gated models require it | 5m | ☐ |
+| 19 | End-to-end test on live Space (clone → listen → interrupt → ask → resume) | 30m | ☐ |
+| 20 | Fix latency issues: preload weights, cache voice/story audio, cap Q&A tokens, validate pause/resume | 45m | ☐ |
 | 21 | Add error handling: graceful failures, loading messages | 30m | ☐ |
 | 22 | Record backup demo video (in case live demo fails) | 30m | ☐ |
 | 23 | Write README.md for the Space (screenshot, description) | 15m | ☐ |
@@ -70,9 +70,12 @@ Ship a public Hugging Face Space: parent clones voice → child hears story + Q&
 | Risk | Mitigation |
 |---|---|
 | QWEN-TTS-0.6B doesn't clone well | Fall back to longer reference audio (30s+); or use CosyVoice as backup model |
-| T4 OOM with TTS + Whisper + Gradio | Load Whisper only on demand; use `whisper-tiny` if needed |
+| GPU OOM with TTS + Whisper + Qwen + Gradio | Use 4-bit/8-bit Qwen loading on T4; prefer A10G for the live demo; load Whisper only on demand; use `whisper-tiny` if needed |
 | HF Space cold start too slow | Use persistent Space (not sleep); preload model in `app.py` global scope |
-| GPT-4o-mini rate limit | Cache answers per story; answers are short, limit won't hit for demo |
+| Qwen answer latency too slow | Keep answers to 1–2 sentences; cap story prompt length; cache answers per story/question pair |
+| Story narration feels slow | Stream paragraph chunks; play the first chunk quickly; continue remaining chunks in the background; cache full narration |
+| Interrupt/resume feels brittle | Track playback state, current chunk index, queued generation jobs, and cached next chunks |
+| ASR adds too much delay | Let text questions bypass ASR; use Whisper-tiny/base or browser transcription for demo mode |
 | Live demo fails | Pre-recorded backup video ready |
 
 ---
@@ -105,7 +108,18 @@ VoiceClone/
 
 - **No database** — stateless demo, voice samples held in memory/session
 - **No auth** — open access for hackathon judges
-- **No streaming** — batch TTS is fine for short stories (< 2 min audio)
-- **GPT-4o-mini over GPT-4** — 10x cheaper, fast enough for short Q&A
-- **Stories < 4K tokens** — no RAG needed, full text in prompt
+- **Interruptible chunked streaming** — play paragraph chunks quickly, pause on Ask, answer, then resume from the saved story position
+- **Qwen2.5-3B-Instruct for Q&A** — keeps story comprehension local while improving latency and GPU fit
+- **Short grounded Q&A** — retrieve relevant story passages, cap answers to 1–2 sentences, then synthesize audio
+- **Button-based interruption first** — tap Ask to pause narration; always-listening voice barge-in is out of scope for the hackathon
 - **Single `app.py`** — keep it simple, refactor later
+
+---
+
+## Review Notes
+
+| Area | Critique | Upgrade |
+|---|---|---|
+| Model plan | Replacing an external Q&A API with Qwen2.5-3B-Instruct removes API risk while keeping latency practical. | Treat T4 with quantization as viable; use A10G when demo reliability matters more than cost. |
+| Secrets | The old plan required an external LLM API secret, which contradicts the privacy-first positioning. | Remove LLM API secrets from deployment; use only HF credentials when required for model access. |
+| Demo schedule | The original plan left little time for model loading, user-visible wait states, and interruption edge cases. | Add explicit validation around model downloads, quantization, cold-start behavior, chunked playback, pause/resume, and cache hits. |

tech_stack.md

@@ -10,15 +10,15 @@ Single Python file Gradio app. Everything runs in one process on a GPU-enabled H
 │                                                 │
 │  app.py (Gradio + gr.Server)                    │
 │    ├── UI: Stitch-styled tabs/cards             │
-│    ├── QWEN-TTS-0.6B (voice clone + narration)  │
+│    ├── QWEN-TTS-0.6B (voice clone + chunks)     │
 │    ├── Whisper-small (ASR for child questions)  │
-│    └── OpenAI GPT-4o-mini (Q&A from story text) │
+│    └── Qwen2.5-3B-Instruct (story Q&A)          │
 │                                                 │
 │  stories/ (3–5 .txt files, public domain)       │
 └─────────────────────────────────────────────────┘
 ```
 
-No database. No external storage. Stories are flat files. Audio is generated on demand.
+No database. No external storage. No external LLM API. Stories are flat files. Audio is generated as interruptible chunks, then cached in the session for replay and resume.
 
 ---
 
@@ -30,13 +30,14 @@ No database. No external storage. Stories are flat files. Audio is generated on
 | `gr.Server` | Custom CSS/JS for Stitch-style polish (animations, palette, layout) |
 | `gr.Audio` | Record/upload parent voice sample |
 | `gr.Dropdown` / `gr.Gallery` | Story selection with cover art |
-| `gr.Audio` (output) | Playback of narrated story / Q&A answer |
+| `gr.Audio` (output) | Playback of streamed story chunks / Q&A answer |
 | `gr.Textbox` + `gr.Audio` (input) | Child question via text or voice |
+| Play/Pause/Ask buttons | Manual interruption and resume without open-mic barge-in |
 
 **UI Tabs:**
 1. **🎤 Clone Voice** — record/upload 15–30s, preview clone
-2. **📖 Listen** — pick story, hear it in cloned voice
-3. **❓ Ask** — ask about the story, hear answer in cloned voice
+2. **📖 Listen** — pick story, hear streamed chunks in cloned voice
+3. **❓ Ask** — pause narration, ask about the story, hear answer, resume
 
 ---
 
@@ -46,10 +47,11 @@ No database. No external storage. Stories are flat files. Audio is generated on
 |---|---|
 | Model | `Qwen/Qwen-TTS-0.6B` from Hugging Face Hub |
 | Size | 0.6B params — fits comfortably on T4 (16GB VRAM) |
-| Capability | Zero-shot voice cloning + TTS from text |
+| Capability | Zero-shot voice cloning + TTS from text chunks |
 | Input | Reference audio (≥5s) + target text |
 | Output | WAV audio in cloned voice |
 | Latency | ~3–5s for a paragraph on T4 |
+| Optimization | Cache the voice representation after recording; generate story audio in interruptible paragraph chunks |
 
 ---
 
@@ -57,9 +59,10 @@ No database. No external storage. Stories are flat files. Audio is generated on
 
 | Choice | Detail |
 |---|---|
-| Model | `openai/whisper-small` (local, 244M params) |
+| Model | Whisper-small checkpoint via Transformers (local, 244M params) |
 | Why | Fast, accurate for short child utterances; fits in GPU alongside TTS |
-| Fallback | `gr.Audio` with browser transcription if GPU memory is tight |
+| Load strategy | Load only when the Ask tab receives audio; text questions bypass ASR |
+| Fallback | Whisper-tiny/base or browser transcription if GPU memory or latency is tight |
 
 ---
 
@@ -67,10 +70,12 @@ No database. No external storage. Stories are flat files. Audio is generated on
 
 | Choice | Detail |
 |---|---|
-| Model | OpenAI `gpt-4o-mini` (API) |
-| Why | Cheap, fast, good at grounded Q&A from short context |
-| Method | Full story text in system prompt + child question → short answer |
-| No RAG needed | Stories are short (< 4K tokens); entire text fits in context |
+| Model | `Qwen/Qwen2.5-3B-Instruct` |
+| Why | Strong small-model instruction following with lower latency and VRAM pressure than an 8B-class model |
+| Method | Current story position + relevant story passages + strict answer-from-story instruction + child question → short answer |
+| Retrieval | Simple paragraph keyword overlap is enough for the demo; full-story prompt remains a fallback |
+| Output cap | 1–2 child-friendly sentences, typically 40–80 new tokens |
+| Runtime note | Use 4-bit/8-bit loading on T4; use bf16 or 8-bit on A10G for more headroom |
 
 ---
 
@@ -93,17 +98,44 @@ Each file: title on line 1, text below. No metadata DB needed.
 |---|---|
 | Platform | Hugging Face Spaces |
 | SDK | Gradio |
-| Hardware | GPU T4 (free tier) or A10G (paid, faster) |
+| Hardware | T4 with quantized Qwen for the budget path; A10G for lower risk live demos |
 | Deploy | `git push` to HF Space repo |
-| Secrets | `OPENAI_API_KEY` in HF Space settings |
+| Secrets | None for LLM inference; `HF_TOKEN` only if any selected model requires gated access |
 | Domain | `huggingface.co/spaces/{user}/readbookmom` |
 
 ---
 
-## 7. Local Dev
+## 7. Latency Plan
+
+| Flow | Target | Implementation |
+|---|---|---|
+| Voice setup | One-time after recording | Compute and cache the voice representation before story generation. |
+| Story narration start | First streamed chunk in ≤ 5s | Split the story into paragraph chunks; synthesize and play the first chunk first. |
+| Narration interruption | Pause in ≤ 500ms after Ask tap | Stop playback, preserve current chunk index, and cancel or deprioritize queued narration jobs. |
+| Q&A interruption loop | Spoken answer starts in ≤ 8s | Use current story position, retrieve relevant passages, cap answer length, then synthesize the final answer. |
+| Story resume | ≤ 1s when next chunk is cached | Resume from the paused chunk or the next queued chunk after the answer finishes. |
+| Story replay | Immediate after first generation | Cache generated audio by voice session and story ID. |
+| Child audio transcription | 1–2s target | Load ASR only for audio questions; prefer lighter ASR fallback for demo mode. |
+| Q&A text answer | 1–3s target | Send only relevant story passages to Qwen and cap output tokens. |
+| Spoken Q&A answer | ≤ 8s total target | Synthesize only the final short answer, not intermediate reasoning or context. |
+
+## 8. Interaction State
+
+| State | Meaning | Key Data |
+|---|---|---|
+| `playing` | Story chunk is currently playing. | `story_id`, `voice_session_id`, `current_chunk_index` |
+| `paused` | Playback is paused by user action. | Current chunk, elapsed position if available |
+| `asking` | Narration is interrupted while the child asks a question. | Current chunk, relevant passages, pending ASR input |
+| `answering` | Qwen answer or answer TTS is being generated. | Question text, short answer, answer audio path |
+| `resuming` | Answer finished and story playback is restarting. | Resume chunk index, cached next chunk |
+| `finished` | Story narration completed. | Cached full-story audio |
+
+---
+
+## 9. Local Dev
 
 ```bash
-pip install gradio transformers torch openai whisper
+pip install gradio transformers torch accelerate bitsandbytes soundfile numpy
 python app.py
 # → http://localhost:7860
 ```
@@ -112,21 +144,21 @@ No Docker, no DB, no infra setup.
 
 ---
 
-## 8. Dependencies
+## 10. Dependencies
 
 ```
 gradio>=5.0
 transformers
 torch
 accelerate
-openai
+bitsandbytes
 soundfile
 numpy
 ```
 
 ---
 
-## 9. Google Stitch UI Customization (via gr.Server)
+## 11. Google Stitch UI Customization (via gr.Server)
 
 `gr.Server` injects custom HTML/CSS/JS to achieve Stitch-quality polish:
 
@@ -136,3 +168,15 @@ numpy
 - **Custom favicon + title**: Branded for demo presentation
 
 All in a `static/` folder loaded by gr.Server mount.
+
+---
+
+## 12. Review Notes
+
+| Area | Critique | Upgrade |
+|---|---|---|
+| Privacy | Using an external Q&A API would undermine the local-inference claim. | Qwen2.5-3B-Instruct keeps questions, story text, and generated answers inside the Space runtime. |
+| GPU fit | QWEN-TTS-0.6B, Whisper-small, and a 3B-class LLM are a more realistic fit for a single Space than an 8B-class LLM, but running all three hot can still pressure T4. | Quantize Qwen on T4, use A10G for demo headroom, cache voice/story artifacts, and load ASR only when the Ask tab receives audio. |
+| Latency | Full-story generation and spoken Q&A can feel slow if every step waits for complete outputs. | Use interruptible paragraph chunks, cache full narration, retrieve only relevant passages, and synthesize only short final answers. |
+| Interaction | Streaming without cancellation can still feel rigid if the child must wait for a chunk to finish. | Add explicit playback state, Ask interruption, queued job cancellation/deprioritization, and resume from the saved chunk. |
+| Dependencies | External LLM SDKs and API secrets are no longer aligned with the model choice. | Use local inference dependencies and optional HF authentication only. |