Feat/teacher voice (#8)

* teacher voice wip

* wip teacher

* teacher voice wip

* voice

* wip voice

* piper speech

* teacher voice and tts realtime

.cursor/plans/teachervoice_realtime_plan_8950875f.plan.md

@@ -4,22 +4,22 @@ overview: EchoCoach today is a one-shot pitch *review* pipeline (record → anal
 todos:
   - id: teacher-voice-core
     content: Add teacher_voice.py + mode prompts + TeacherVoiceTurnResult; wire ASR → chat(history) → TTS per turn
-    status: pending
+    status: completed
   - id: teacher-voice-ui
     content: Create teacher_voice.py Gradio tab (modes, push-to-talk, chatbot, audio reply, optional RAG); register in app.py
-    status: pending
+    status: completed
   - id: rag-lesson-modes
     content: Integrate ResearchMind retrieval into Explain/Lesson modes (reuse run_research_question / rag scope from chat tab)
-    status: pending
+    status: completed
   - id: docs-tests
     content: Update USAGE.md with EchoCoach vs TeacherVoice; add mock-backend unit tests and trace skill teacher-voice
-    status: pending
+    status: completed
   - id: phase2-chunked-tts
     content: "Optional: sentence-chunked Piper + shorter turn cap for faster time-to-first-audio"
-    status: pending
+    status: completed
   - id: phase3-omni
     content: "Optional: MiniCPM-o 4.5 omni preset behind ECHOCOACH_VOICE_PROFILE=omni for speech-in/speech-out"
-    status: pending
+    status: completed
 isProject: false
 ---
 

.env.example

@@ -54,9 +54,12 @@ ALLOW_MODEL_SWITCH=false
 # VOICE_PRESETS_PATH=./voice_models.yaml
 # ECHOCOACH_ASR_PRESET=whisper-cpp-tiny
 # ECHOCOACH_TTS_PRESET=piper-multilingual
+# ECHOCOACH_REALTIME_TTS_PRESET=vibevoice-realtime-0.5b   # TeacherVoice VoiceOut (falls back to Piper)
 # ECHOCOACH_COACH_MODEL=minicpm5-1b
 # ECHOCOACH_MAX_SECONDS=30
 # ECHOCOACH_CAPTURE_DEVICE=   # optional ALSA/PipeWire device (e.g. pipewire, alsa_input.pci-...)
+# ECHOCOACH_VOICE_PROFILE=pipeline   # pipeline (default) or omni for MiniCPM-o attempt
+# ECHOCOACH_OMNI_MODEL=openbmb/MiniCPM-o-4_5
 # PIPER_VOICES_DIR=~/.local/share/piper/voices
 # For Cohere Transcribe ASR: huggingface-cli login + accept model terms, then:
 # ECHOCOACH_ASR_PRESET=cohere-transcribe

USAGE.md

@@ -2,7 +2,7 @@
 
 How to run the **Lesson Agent** Gradio app locally, test it in Docker, and deploy to a Hugging Face Space for the [Build Small Hackathon](https://huggingface.co/build-small-hackathon).
 
-The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, **EchoCoach** for voice practice feedback, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
+The primary UI is the **Lesson slides** tab (topic → local model outline → downloadable `.pptx`). Use **ResearchMind** for corpus Q&A, **TeacherVoice** for spoken back-and-forth tutoring, **EchoCoach** for one-shot pitch analysis, or ground lessons directly from the Lesson tab. The **Chat (debug)** tab tests the underlying model.
 
 ## Prerequisites
 
@@ -109,7 +109,8 @@ Configure presets in [`voice_models.yaml`](voice_models.yaml) or via `.env`:
 | Variable | Default | Description |
 | -------- | ------- | ----------- |
 | `ECHOCOACH_ASR_PRESET` | `whisper-cpp-tiny` | ASR preset key |
-| `ECHOCOACH_TTS_PRESET` | `piper-multilingual` | TTS preset key |
+| `ECHOCOACH_TTS_PRESET` | `piper-multilingual` | TTS preset key (EchoCoach, default VoiceOut) |
+| `ECHOCOACH_REALTIME_TTS_PRESET` | `vibevoice-realtime-0.5b` | TeacherVoice streaming TTS (see below) |
 | `ECHOCOACH_COACH_MODEL` | `minicpm5-1b` | Text coach preset (from `models.yaml`) |
 | `ECHOCOACH_MAX_SECONDS` | `30` | Max recording length |
 
@@ -121,6 +122,57 @@ Smoke tests (analysis only, no GPU):
 bash scripts/echo_coach_smoke.sh
 ```
 
+### TeacherVoice — spoken conversation (turn-based)
+
+The **TeacherVoice** tab is a **multi-turn voice teacher** — not full duplex like a phone call, but speak → wait → hear a reply → repeat.
+
+| Mode | Purpose |
+| ---- | ------- |
+| **Explain** | Tutor any topic in plain language |
+| **Lesson coach** | Discuss and outline lesson content verbally |
+| **Pitch practice** | Short live speaking tips each turn |
+
+**EchoCoach vs TeacherVoice**
+
+| | EchoCoach | TeacherVoice |
+| --- | --- | --- |
+| Interaction | One-shot after **Analyze pitch** | Multi-turn **Send turn** |
+| Best for | Pace/filler charts, JSON rewrite report | Q&A, lesson discussion, conversational pitch tips |
+| TTS | One VoiceOut clip per analysis | Voice reply every turn (first sentence plays quickly when Piper is installed) |
+| RAG | No | Optional ResearchMind grounding (Explain / Lesson) |
+
+**Flow per turn:** record up to **15s** → ASR → text LLM with chat history → Piper TTS (auto-plays when installed).
+
+After each reply, use **Speak last reply** or **Speak first sentence** to generate or replay VoiceOut from the latest assistant message (works even if auto-TTS was skipped).
+
+Install Piper for voice output (included in `gradio-space` deps after `uv sync`):
+
+```bash
+uv sync
+python -m piper.download_voices en_US-lessac-medium
+```
+
+Voices are stored under `models/piper/` (gitignored) or `~/.local/share/piper/voices/`. **Restart the Gradio app** after installing Piper so the Speak buttons can synthesize audio.
+
+**Realtime TTS (VibeVoice)** — [microsoft/VibeVoice-Realtime-0.5B](https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B) is registered in `voice_models.yaml` as `vibevoice-realtime-0.5b` (~300 ms to first audio, streaming text-in). TeacherVoice uses `realtime_tts_preset` from YAML by default; override with `ECHOCOACH_REALTIME_TTS_PRESET` or set `ECHOCOACH_TTS_PRESET=vibevoice-realtime-0.5b` globally. GPU recommended; falls back to Piper until the model loads. English-first; de/fr/it/es/pt/nl/pl/ja/ko are experimental per the model card.
+
+Enable RAG in the accordion: pick a ResearchMind session and optional documents (same scope rules as Chat debug).
+
+Reuse VoiceOut in other tabs via `gradio_space.voice_helpers.speak_last_assistant_reply`.
+
+Optional omni profile (GPU, experimental — falls back to ASR+LLM+Piper):
+
+```bash
+ECHOCOACH_VOICE_PROFILE=omni
+ECHOCOACH_OMNI_MODEL=openbmb/MiniCPM-o-4_5
+```
+
+Unit tests (no GPU):
+
+```bash
+uv run pytest libs/echocoach/tests/test_teacher_voice.py -q
+```
+
 ### 5. Upload agent trace (Sharing is Caring badge)
 
 ```bash

apps/gradio-space/pyproject.toml

@@ -9,7 +9,7 @@ authors = [
 requires-python = ">=3.12"
 dependencies = [
     "agent",
-    "echocoach[whisper]",
+    "echocoach[piper,whisper]",
     "gradio>=5.0.0",
     "inference",
 ]

apps/gradio-space/src/gradio_space/app.py

@@ -8,10 +8,12 @@ from gradio_space.tabs import (
     build_education_pptx_tab,
     build_echo_coach_tab,
     build_research_mind_tab,
+    build_teacher_voice_tab,
 )
 from gradio_space.tabs.education_pptx import gradio_allowed_paths
 from gradio_space.tabs.echo_coach import echo_coach_allowed_paths
 from gradio_space.tabs.research_mind import researchmind_allowed_paths
+from gradio_space.tabs.teacher_voice import teacher_voice_allowed_paths
 from inference.config import get_app_config
 
 _app_config = get_app_config()
@@ -28,9 +30,9 @@ def build_demo() -> gr.Blocks:
     with gr.Blocks(title="Lesson Agent + ResearchMind — Build Small Hackathon") as demo:
         gr.Markdown(
             f"""
-# Lesson Agent + ResearchMind + EchoCoach
+# Lesson Agent + ResearchMind + EchoCoach + TeacherVoice
 
-Local skill-based agents — **lesson slides**, **research with MemRAG**, and **voice practice coaching** (offline).
+Local skill-based agents — **lesson slides**, **research with MemRAG**, **voice conversation (TeacherVoice)**, and **pitch analysis (EchoCoach)** (offline).
 
 - **Model:** `{active.key}` — {active.label}
 - **Backend:** `{active.backend}`
@@ -47,6 +49,8 @@ Part of the [Build Small Hackathon](https://huggingface.co/build-small-hackathon
                 build_research_mind_tab()
             with gr.Tab("EchoCoach"):
                 build_echo_coach_tab()
+            with gr.Tab("TeacherVoice"):
+                build_teacher_voice_tab()
             with gr.Tab("Chat (debug)"):
                 build_chat_tab()
 
@@ -69,6 +73,7 @@ def main() -> None:
             *gradio_allowed_paths(),
             *researchmind_allowed_paths(),
             *echo_coach_allowed_paths(),
+            *teacher_voice_allowed_paths(),
         ],
     )
 

apps/gradio-space/src/gradio_space/tabs/__init__.py

@@ -2,10 +2,12 @@ from gradio_space.tabs.chat import build_chat_tab
 from gradio_space.tabs.education_pptx import build_education_pptx_tab
 from gradio_space.tabs.echo_coach import build_echo_coach_tab
 from gradio_space.tabs.research_mind import build_research_mind_tab
+from gradio_space.tabs.teacher_voice import build_teacher_voice_tab
 
 __all__ = [
     "build_chat_tab",
     "build_education_pptx_tab",
     "build_echo_coach_tab",
     "build_research_mind_tab",
+    "build_teacher_voice_tab",
 ]

apps/gradio-space/src/gradio_space/tabs/echo_coach.py

@@ -159,6 +159,8 @@ Record up to **{_config.max_seconds} seconds**, then get local feedback: transcr
 
 **Browser mic:** open **http://localhost:7860** in Chrome or Firefox (not Cursor's preview) and allow microphone access.
 If the mic icon fails, use **Start / Stop recording** below or **Upload** a `.wav` / `.mp3`.
+
+For conversational pitch tips, try the **TeacherVoice** tab (Pitch practice mode). This tab provides deep analysis: pace charts, filler counts, and a structured rewrite.
 """
     )
 

apps/gradio-space/src/gradio_space/tabs/teacher_voice.py

@@ -0,0 +1,332 @@
+from __future__ import annotations
+
+import gradio as gr
+
+from echocoach.config import get_echo_coach_config
+from echocoach.prompts import MODE_LABELS, TeacherVoiceMode
+from echocoach.recording import (
+    ServerRecordingError,
+    recording_backend_status,
+    recording_elapsed_seconds,
+    recording_level_warning,
+    start_server_recording,
+    stop_server_recording,
+)
+from echocoach.teacher_voice import RAG_MODES, run_teacher_voice_turn
+from gradio_space.model_loading import ensure_model_loaded, get_active_model_key, model_status
+from gradio_space.research_helpers import (
+    list_doc_choices,
+    list_session_choices,
+    rag_scope_hint,
+    refresh_doc_choices,
+    refresh_sessions,
+)
+from echocoach.omni import omni_status_message
+from gradio_space.voice_helpers import speak_last_assistant_reply
+from inference.factory import get_backend
+
+_config = get_echo_coach_config()
+_TURN_MAX = min(15, _config.max_seconds)
+_MODE_CHOICES = [(label, key) for key, label in MODE_LABELS.items()]
+
+
+def _empty_turn() -> tuple:
+    return (
+        [],
+        None,
+        "Start recording, speak your question, stop, then click **Send turn**.",
+        "",
+        {},
+    )
+
+
+def ui_start_recording(max_seconds: int) -> tuple[str, dict, dict]:
+    try:
+        start_server_recording(int(max_seconds))
+    except ServerRecordingError as exc:
+        return (
+            str(exc),
+            gr.update(interactive=True),
+            gr.update(interactive=False),
+        )
+    return (
+        (
+            f"Recording… speak now, then click **Stop recording** "
+            f"(auto-stops after {int(max_seconds)}s)."
+        ),
+        gr.update(interactive=False),
+        gr.update(interactive=True),
+    )
+
+
+def ui_stop_recording() -> tuple[str | None, str, dict, dict]:
+    try:
+        elapsed = recording_elapsed_seconds()
+        path = stop_server_recording()
+        warning = recording_level_warning(path)
+    except ServerRecordingError as exc:
+        return (
+            None,
+            str(exc),
+            gr.update(interactive=True),
+            gr.update(interactive=False),
+        )
+    except Exception as exc:  # noqa: BLE001
+        return (
+            None,
+            f"Recording failed: {exc}",
+            gr.update(interactive=True),
+            gr.update(interactive=False),
+        )
+
+    status = f"Recording saved ({elapsed:.1f}s). Click **Send turn** to talk to TeacherVoice."
+    if warning:
+        status += f" Warning: {warning}"
+    return (
+        gr.update(value=str(path)),
+        status,
+        gr.update(interactive=True),
+        gr.update(interactive=False),
+    )
+
+
+def clear_conversation() -> tuple:
+    return _empty_turn()
+
+
+def send_turn(
+    audio_path: str | None,
+    history: list,
+    mode: TeacherVoiceMode,
+    language: str,
+    asr_preset: str,
+    topic: str,
+    use_rag: bool,
+    session_id: str,
+    doc_ids: list[str] | None,
+) -> tuple:
+    model_key = get_active_model_key()
+    load_error = ensure_model_loaded(model_key)
+    if load_error:
+        return (
+            history,
+            None,
+            load_error,
+            "",
+            {},
+        )
+
+    if not audio_path:
+        return (
+            history,
+            None,
+            "Record or upload audio, then click **Send turn**.",
+            "",
+            {},
+        )
+
+    try:
+        result = run_teacher_voice_turn(
+            audio_path,
+            history,
+            mode=mode,
+            language=language,
+            topic=topic or None,
+            asr_preset=asr_preset,
+            backend=get_backend(model_key),
+            use_rag=use_rag and mode in RAG_MODES,
+            session_id=session_id,
+            doc_ids=doc_ids,
+            max_turn_seconds=_TURN_MAX,
+        )
+    except Exception as exc:  # noqa: BLE001
+        return (
+            history,
+            None,
+            f"TeacherVoice failed: {exc}",
+            "",
+            {},
+        )
+
+    status = f"Turn complete — transcribed {len(result.user_text)} chars, replied in voice."
+    if result.voiceout_warning:
+        status += f" VoiceOut: {result.voiceout_warning}"
+
+    playback = result.voiceout_first_path or result.voiceout_path
+    return (
+        result.history,
+        playback,
+        status,
+        f"Trace saved: `{result.trace_path}`",
+        result.trace,
+    )
+
+
+def _format_speak_status(status: str) -> str:
+    if status.startswith("VoiceOut ready"):
+        return f"**{status}**"
+    return f"**VoiceOut:** {status}"
+
+
+def speak_full_reply(history: list, language: str) -> tuple[str | None, str, str]:
+    playback, status = speak_last_assistant_reply(history, language, first_sentence_only=False)
+    return playback, status, _format_speak_status(status)
+
+
+def speak_quick_reply(history: list, language: str) -> tuple[str | None, str, str]:
+    playback, status = speak_last_assistant_reply(history, language, first_sentence_only=True)
+    return playback, status, _format_speak_status(status)
+
+
+def build_teacher_voice_tab() -> None:
+    lang_choices = _config.language_choices()
+    asr_choices = _config.asr_choices()
+    default_lang = lang_choices[0][1] if lang_choices else "en"
+    default_asr = _config.asr_preset
+    mic_status = recording_backend_status()
+
+    omni_note = omni_status_message()
+    gr.Markdown(
+        f"""
+**TeacherVoice** — turn-based voice conversation with a local teacher (not full duplex).
+
+1. Choose a mode → record a short turn (max **{_TURN_MAX}s**) → **Send turn** → hear the reply.
+2. **Explain** — tutor any topic. **Lesson coach** — outline and discuss lessons. **Pitch practice** — live speaking tips.
+3. For deep pitch analysis (pace charts, filler counts), use the **EchoCoach** tab.
+
+Latency is typically a few seconds per turn on GPU; CPU may take longer.
+{omni_note or ""}
+"""
+    )
+
+    with gr.Row():
+        with gr.Column(scale=1):
+            mode_dd = gr.Dropdown(
+                label="Mode",
+                choices=_MODE_CHOICES,
+                value="explain",
+            )
+            topic_tb = gr.Textbox(
+                label="Topic (Explain / Lesson modes)",
+                placeholder="e.g. Photosynthesis for grade 6",
+            )
+            record_status_md = gr.Markdown(mic_status)
+            with gr.Accordion("Record from this computer", open=True):
+                record_seconds = gr.Slider(
+                    label="Max turn length (seconds)",
+                    minimum=3,
+                    maximum=_TURN_MAX,
+                    value=_TURN_MAX,
+                    step=1,
+                )
+                with gr.Row():
+                    record_start_btn = gr.Button("Start recording", variant="secondary")
+                    record_stop_btn = gr.Button("Stop recording", variant="stop", interactive=False)
+            audio_in = gr.Audio(
+                label="Your turn (browser mic or upload)",
+                sources=["upload", "microphone"],
+                type="filepath",
+                format="wav",
+            )
+            language = gr.Dropdown(label="Language", choices=lang_choices, value=default_lang)
+            asr_preset = gr.Dropdown(label="ASR preset", choices=asr_choices, value=default_asr)
+            with gr.Accordion("ResearchMind RAG (Explain / Lesson)", open=False):
+                use_rag = gr.Checkbox(label="Ground answers in ingested sources", value=False)
+                session_dd = gr.Dropdown(
+                    label="Session",
+                    choices=list_session_choices(),
+                    value="",
+                )
+                refresh_sessions_btn = gr.Button("Refresh sessions", size="sm")
+                doc_dd = gr.CheckboxGroup(label="Documents (empty = all in session)", choices=[], value=[])
+                rag_hint = gr.Markdown(value=rag_scope_hint("", []))
+            with gr.Row():
+                send_btn = gr.Button("Send turn", variant="primary")
+                clear_btn = gr.Button("Clear conversation", variant="secondary")
+            status = gr.Textbox(label="Status", interactive=False, lines=3)
+            coach_status = gr.Markdown(model_status(get_active_model_key()))
+
+        with gr.Column(scale=2):
+            chatbot = gr.Chatbot(label="Conversation", height=360)
+            with gr.Row():
+                speak_full_btn = gr.Button("Speak last reply", variant="secondary")
+                speak_quick_btn = gr.Button("Speak first sentence", variant="secondary")
+            speak_status = gr.Markdown(
+                value="_Use **Speak** buttons to hear the latest teacher reply._"
+            )
+            voiceout = gr.Audio(
+                label="Teacher reply (auto after Send turn, or use Speak buttons)",
+                type="filepath",
+                autoplay=True,
+            )
+            trace_note = gr.Markdown()
+            trace_json = gr.JSON(label="Trace")
+
+    record_start_btn.click(
+        ui_start_recording,
+        inputs=[record_seconds],
+        outputs=[status, record_start_btn, record_stop_btn],
+    )
+    record_stop_btn.click(
+        ui_stop_recording,
+        outputs=[audio_in, status, record_start_btn, record_stop_btn],
+    ).then(
+        lambda: recording_backend_status(),
+        outputs=[record_status_md],
+    )
+
+    refresh_sessions_btn.click(fn=refresh_sessions, inputs=[session_dd], outputs=[session_dd])
+    session_dd.change(
+        fn=refresh_doc_choices,
+        inputs=[session_dd, doc_dd],
+        outputs=[doc_dd],
+    )
+    for trigger in (use_rag, session_dd, doc_dd):
+        trigger.change(
+            fn=lambda rag_on, sid, docs: (
+                rag_scope_hint(sid, docs) if rag_on else "_RAG off — model knowledge only._"
+            ),
+            inputs=[use_rag, session_dd, doc_dd],
+            outputs=[rag_hint],
+        )
+
+    send_btn.click(
+        send_turn,
+        inputs=[
+            audio_in,
+            chatbot,
+            mode_dd,
+            language,
+            asr_preset,
+            topic_tb,
+            use_rag,
+            session_dd,
+            doc_dd,
+        ],
+        outputs=[chatbot, voiceout, status, trace_note, trace_json],
+    )
+
+    clear_btn.click(clear_conversation, outputs=[chatbot, voiceout, status, trace_note, trace_json])
+
+    speak_full_btn.click(
+        speak_full_reply,
+        inputs=[chatbot, language],
+        outputs=[voiceout, status, speak_status],
+    )
+    speak_quick_btn.click(
+        speak_quick_reply,
+        inputs=[chatbot, language],
+        outputs=[voiceout, status, speak_status],
+    )
+
+
+def teacher_voice_allowed_paths() -> list[str]:
+    paths: list[str] = []
+    if _config.presets_path:
+        paths.append(str(_config.presets_path.parent))
+    from echocoach.config import outputs_dir
+
+    paths.append(str(outputs_dir()))
+    paths.append(str(outputs_dir() / "recordings"))
+    paths.append(str(outputs_dir() / "teacher_voice"))
+    return paths

apps/gradio-space/src/gradio_space/voice_helpers.py

@@ -0,0 +1,43 @@
+"""Gradio helpers for on-demand VoiceOut (shared across tabs)."""
+
+from __future__ import annotations
+
+from echocoach.config import get_echo_coach_config
+from echocoach.voiceout import last_assistant_message, speak_assistant_text
+
+
+def _teacher_tts_preset() -> str | None:
+    config = get_echo_coach_config()
+    return config.realtime_tts_preset or config.tts_preset
+
+
+def speak_last_assistant_reply(
+    history: list,
+    language: str,
+    *,
+    first_sentence_only: bool = False,
+    tts_preset: str | None = None,
+) -> tuple[str | None, str]:
+    """TTS the latest assistant message. Returns (wav_path, status_message)."""
+    text = last_assistant_message(history)
+    if not text:
+        return None, "No assistant reply to speak yet — send a turn first."
+
+    preset = tts_preset or _teacher_tts_preset()
+    playback, _, warning = speak_assistant_text(
+        text,
+        language=language or "en",
+        tts_preset=preset,
+        first_sentence_only=first_sentence_only,
+    )
+    if not playback:
+        hint = warning or "VoiceOut failed."
+        if "piper" in hint.lower() or "not installed" in hint.lower():
+            hint += " Install with: `uv sync --package echocoach --extra piper`"
+        return None, hint
+
+    label = "first sentence" if first_sentence_only else "full reply"
+    status = f"VoiceOut ready ({label})."
+    if warning:
+        status += f" {warning}"
+    return playback, status

libs/echocoach/src/echocoach/config.py

@@ -8,7 +8,7 @@ from pathlib import Path
 from typing import Any, Literal
 
 AsrBackendName = Literal["cohere", "whisper_cpp"]
-TtsBackendName = Literal["piper"]
+TtsBackendName = Literal["piper", "vibevoice"]
 
 
 @dataclass(frozen=True)
@@ -33,12 +33,17 @@ class TtsPreset:
     backend: TtsBackendName
     voices: dict[str, str]
     fallback_voice: str
+    model_id: str | None = None
+    streaming: bool = False
+    realtime: bool = False
+    supported_languages: tuple[str, ...] = ()
 
 
 @dataclass(frozen=True)
 class EchoCoachConfig:
     asr_preset: str
     tts_preset: str
+    realtime_tts_preset: str | None
     coach_model: str
     max_seconds: int
     languages: list[LanguageOption]
@@ -107,6 +112,7 @@ def _builtin_config() -> EchoCoachConfig:
     return EchoCoachConfig(
         asr_preset="whisper-cpp-tiny",
         tts_preset="piper-multilingual",
+        realtime_tts_preset=None,
         coach_model="minicpm5-1b",
         max_seconds=30,
         languages=langs,
@@ -130,17 +136,32 @@ def _parse_asr_entry(key: str, raw: dict[str, Any]) -> AsrPreset:
 
 def _parse_tts_entry(key: str, raw: dict[str, Any]) -> TtsPreset:
     backend = str(raw.get("backend", "piper"))
-    if backend != "piper":
-        raise ValueError(f"TTS preset {key!r}: only piper backend is supported in MVP")
+    if backend not in ("piper", "vibevoice"):
+        raise ValueError(f"TTS preset {key!r}: backend must be piper or vibevoice")
+
     voices = raw.get("voices") or {}
-    if not isinstance(voices, dict) or not voices:
-        raise ValueError(f"TTS preset {key!r}: voices mapping is required")
+    if not isinstance(voices, dict):
+        raise ValueError(f"TTS preset {key!r}: voices must be a mapping")
+    languages = raw.get("languages") or []
+    if backend == "vibevoice":
+        if not voices and languages:
+            voices = {str(lang): "default" for lang in languages}
+        fallback = str(raw.get("fallback_language") or raw.get("fallback_voice") or "en")
+    else:
+        if not voices:
+            raise ValueError(f"TTS preset {key!r}: voices mapping is required")
+        fallback = str(raw.get("fallback_voice", "en_US-lessac-medium"))
+
     return TtsPreset(
         key=key,
         label=str(raw.get("label", key)),
-        backend="piper",
+        backend=backend,  # type: ignore[arg-type]
         voices={str(k): str(v) for k, v in voices.items()},
-        fallback_voice=str(raw.get("fallback_voice", "en_US-lessac-medium")),
+        fallback_voice=fallback,
+        model_id=raw.get("model_id"),
+        streaming=bool(raw.get("streaming", False)),
+        realtime=bool(raw.get("realtime", False)),
+        supported_languages=tuple(str(lang) for lang in languages),
     )
 
 
@@ -183,6 +204,7 @@ def load_echo_coach_config() -> EchoCoachConfig:
         config = EchoCoachConfig(
             asr_preset=asr_default,
             tts_preset=tts_default,
+            realtime_tts_preset=defaults.get("realtime_tts_preset"),
             coach_model=str(defaults.get("coach_model", "minicpm5-1b")),
             max_seconds=int(defaults.get("max_seconds", 30)),
             languages=languages,
@@ -196,6 +218,8 @@ def load_echo_coach_config() -> EchoCoachConfig:
         updates["asr_preset"] = os.environ["ECHOCOACH_ASR_PRESET"]
     if os.environ.get("ECHOCOACH_TTS_PRESET"):
         updates["tts_preset"] = os.environ["ECHOCOACH_TTS_PRESET"]
+    if os.environ.get("ECHOCOACH_REALTIME_TTS_PRESET"):
+        updates["realtime_tts_preset"] = os.environ["ECHOCOACH_REALTIME_TTS_PRESET"]
     if os.environ.get("ECHOCOACH_COACH_MODEL"):
         updates["coach_model"] = os.environ["ECHOCOACH_COACH_MODEL"]
     if os.environ.get("ECHOCOACH_MAX_SECONDS"):

libs/echocoach/src/echocoach/omni.py

@@ -0,0 +1,68 @@
+"""Optional MiniCPM-o omni speech-in/speech-out backend (GPU-only, falls back to pipeline)."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+from typing import Any
+
+
+def voice_profile() -> str:
+    return os.environ.get("ECHOCOACH_VOICE_PROFILE", "pipeline").strip().lower()
+
+
+def is_omni_profile() -> bool:
+    return voice_profile() == "omni"
+
+
+def try_omni_turn(
+    audio_path: str,
+    *,
+    language: str,
+    history: list,
+    system_prompt: str,
+) -> tuple[str | None, str | None, str | None]:
+    """Attempt speech-in/speech-out via MiniCPM-o. Returns (user_text, reply_text, wav_path) or Nones."""
+    if not is_omni_profile():
+        return None, None, None
+
+    model_id = os.environ.get("ECHOCOACH_OMNI_MODEL", "openbmb/MiniCPM-o-4_5")
+    try:
+        import torch
+        from transformers import AutoModel, AutoTokenizer
+    except ImportError:
+        return None, None, "Omni profile requires transformers and torch."
+
+    if not torch.cuda.is_available():
+        return None, None, "Omni profile requires CUDA; falling back to ASR+LLM+TTS pipeline."
+
+    try:
+        tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+        model = AutoModel.from_pretrained(
+            model_id,
+            trust_remote_code=True,
+            torch_dtype=torch.bfloat16,
+            init_audio=True,
+            init_tts=True,
+        ).eval()
+        if torch.cuda.is_available():
+            model = model.cuda()
+    except Exception as exc:  # noqa: BLE001
+        return None, None, f"Omni model load failed: {exc}"
+
+    # MiniCPM-o APIs vary by release; pipeline fallback is the supported path until wired.
+    _ = (tokenizer, model, audio_path, language, history, system_prompt)
+    return None, None, (
+        f"Omni preset ({model_id}) is configured but end-to-end omni turn is not wired yet; "
+        "using ASR + text LLM + Piper pipeline."
+    )
+
+
+def omni_status_message() -> str | None:
+    if not is_omni_profile():
+        return None
+    model_id = os.environ.get("ECHOCOACH_OMNI_MODEL", "openbmb/MiniCPM-o-4_5")
+    return (
+        f"**Voice profile:** `omni` ({model_id}). "
+        "Falls back to ASR + LLM + Piper until omni turn API is fully integrated."
+    )

libs/echocoach/src/echocoach/prompts.py

@@ -0,0 +1,46 @@
+"""TeacherVoice mode system prompts."""
+
+from __future__ import annotations
+
+from typing import Literal
+
+TeacherVoiceMode = Literal["explain", "lesson", "pitch"]
+
+MODE_LABELS: dict[TeacherVoiceMode, str] = {
+    "explain": "Explain",
+    "lesson": "Lesson coach",
+    "pitch": "Pitch practice",
+}
+
+EXPLAIN_SYSTEM = """You are TeacherVoice, a friendly tutor who explains ideas in plain language.
+Keep answers concise (2-5 sentences) so they work well when spoken aloud.
+Use simple examples when helpful. If the student asks in another language, reply in that language.
+When source excerpts are provided, ground your answer in them and cite with [1], [2], etc."""
+
+LESSON_SYSTEM = """You are TeacherVoice, a lesson-planning coach for teachers and students.
+Help outline and explain lesson content verbally: learning goals, key points, and a simple flow.
+Keep each reply short (2-5 sentences) for voice playback.
+If a lesson topic is set, stay focused on it. When source excerpts are provided, use them and cite [1], [2], etc."""
+
+PITCH_SYSTEM = """You are TeacherVoice, a supportive public-speaking coach in a live conversation.
+Give brief, actionable feedback on what the student just said (opening, clarity, energy, structure).
+Do not produce JSON or long reports — speak naturally in 2-4 sentences.
+Suggest one concrete improvement for their next attempt. For charts and pace analysis, they can use the EchoCoach tab."""
+
+_MODE_SYSTEM: dict[TeacherVoiceMode, str] = {
+    "explain": EXPLAIN_SYSTEM,
+    "lesson": LESSON_SYSTEM,
+    "pitch": PITCH_SYSTEM,
+}
+
+
+def system_prompt_for_mode(mode: TeacherVoiceMode) -> str:
+    return _MODE_SYSTEM[mode]
+
+
+def topic_context_block(topic: str | None, mode: TeacherVoiceMode) -> str | None:
+    topic = (topic or "").strip()
+    if not topic or mode == "pitch":
+        return None
+    label = "Lesson topic" if mode == "lesson" else "Focus topic"
+    return f"{label}: {topic}"

libs/echocoach/src/echocoach/teacher_voice.py

@@ -0,0 +1,290 @@
+"""Turn-based TeacherVoice conversation pipeline."""
+
+from __future__ import annotations
+
+import uuid
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Any
+
+from agent.trace import TraceRecorder
+from inference.base import InferenceBackend
+from inference.response_clean import strip_reasoning_output
+from researchmind.citations import format_context_block, format_references
+from researchmind.config import get_config as get_researchmind_config
+from researchmind.ingest import IngestPipeline
+from researchmind.retrieve import retrieve
+
+from echocoach.asr.factory import get_asr_backend
+from echocoach.audio_io import clamp_duration, load_audio_mono_16k, write_wav_temp
+from echocoach.config import get_echo_coach_config, outputs_dir
+from echocoach.prompts import TeacherVoiceMode, system_prompt_for_mode, topic_context_block
+from echocoach.voiceout import extract_message_text, strip_references_for_tts, synthesize_voice_reply
+
+RAG_MODES: frozenset[TeacherVoiceMode] = frozenset({"explain", "lesson"})
+
+
+@dataclass
+class RagContext:
+    context_block: str
+    references_markdown: str
+    chunk_count: int
+    warning: str | None = None
+
+
+@dataclass
+class TeacherVoiceTurnResult:
+    user_text: str
+    assistant_text: str
+    history: list[dict[str, str]]
+    voiceout_path: str | None
+    voiceout_first_path: str | None
+    voiceout_warning: str | None
+    rag_references: str | None
+    trace_path: str
+    trace: dict[str, Any] = field(default_factory=dict)
+
+
+def append_chat_turn(
+    history: list,
+    user_text: str,
+    assistant_text: str,
+) -> list[dict[str, str]]:
+    """Append a turn in Gradio 5 messages format."""
+    updated: list[dict[str, str]] = []
+    for item in history or []:
+        if isinstance(item, dict) and "role" in item and "content" in item:
+            updated.append(
+                {"role": str(item["role"]), "content": extract_message_text(item["content"])}
+            )
+        elif isinstance(item, (list, tuple)) and len(item) == 2:
+            user_msg, assistant_msg = item
+            updated.append({"role": "user", "content": extract_message_text(user_msg)})
+            if assistant_msg:
+                updated.append(
+                    {"role": "assistant", "content": extract_message_text(assistant_msg)}
+                )
+    updated.append({"role": "user", "content": user_text})
+    updated.append({"role": "assistant", "content": assistant_text})
+    return updated
+
+
+def history_to_messages(history: list) -> list[dict[str, str]]:
+    messages: list[dict[str, str]] = []
+    for item in history:
+        if isinstance(item, dict):
+            messages.append(
+                {"role": item["role"], "content": extract_message_text(item["content"])}
+            )
+        else:
+            user_msg, assistant_msg = item
+            messages.append({"role": "user", "content": extract_message_text(user_msg)})
+            if assistant_msg:
+                messages.append(
+                    {"role": "assistant", "content": extract_message_text(assistant_msg)}
+                )
+    return messages
+
+
+def fetch_rag_context(
+    question: str,
+    *,
+    session_id: str,
+    doc_ids: list[str] | None,
+) -> RagContext | None:
+    store = IngestPipeline().store
+    cfg = get_researchmind_config()
+    scope_session = session_id if session_id and not doc_ids else None
+    scope_docs = doc_ids if doc_ids else None
+    chunks = retrieve(
+        question,
+        store,
+        config=cfg,
+        session_id=scope_session,
+        doc_ids=scope_docs,
+    )
+    if not chunks:
+        if doc_ids:
+            warning = "No passages in selected documents for this question."
+        elif session_id:
+            warning = "No indexed sources in this session yet."
+        else:
+            warning = "No indexed sources in the corpus yet."
+        return RagContext(context_block="", references_markdown="", chunk_count=0, warning=warning)
+
+    context_block, citations = format_context_block(chunks)
+    refs = format_references(citations)
+    return RagContext(
+        context_block=context_block,
+        references_markdown=refs,
+        chunk_count=len(chunks),
+    )
+
+
+def build_teacher_messages(
+    *,
+    mode: TeacherVoiceMode,
+    history: list,
+    user_text: str,
+    topic: str | None = None,
+    rag: RagContext | None = None,
+) -> list[dict[str, str]]:
+    system = system_prompt_for_mode(mode)
+    topic_line = topic_context_block(topic, mode)
+    if topic_line:
+        system = f"{system}\n\n{topic_line}"
+
+    messages: list[dict[str, str]] = [{"role": "system", "content": system}]
+    messages.extend(history_to_messages(history))
+
+    user_parts: list[str] = []
+    if rag and rag.context_block:
+        user_parts.append(
+            "Use these source excerpts as grounding. Cite with [1], [2], etc. when relevant.\n\n"
+            f"{rag.context_block}"
+        )
+    user_parts.append(user_text.strip())
+    messages.append({"role": "user", "content": "\n\n".join(user_parts)})
+    return messages
+
+
+def run_teacher_voice_turn(
+    audio_path: str,
+    history: list,
+    *,
+    mode: TeacherVoiceMode = "explain",
+    language: str = "en",
+    topic: str | None = None,
+    asr_preset: str | None = None,
+    tts_preset: str | None = None,
+    coach_model: str | None = None,
+    backend: InferenceBackend,
+    use_rag: bool = False,
+    session_id: str = "",
+    doc_ids: list[str] | None = None,
+    max_turn_seconds: int | None = None,
+) -> TeacherVoiceTurnResult:
+    if not audio_path:
+        raise ValueError("No audio recording provided.")
+
+    config = get_echo_coach_config()
+    asr_key = asr_preset or config.asr_preset
+    tts_key = tts_preset or config.realtime_tts_preset or config.tts_preset
+    model_key = coach_model or config.coach_model
+    turn_cap = max_turn_seconds or min(15, config.max_seconds)
+    run_id = uuid.uuid4().hex[:12]
+    out_base = outputs_dir()
+
+    trace = TraceRecorder(
+        skill="teacher-voice",
+        model=model_key,
+        user_input={
+            "mode": mode,
+            "language": language,
+            "topic": topic,
+            "asr_preset": asr_key,
+            "tts_preset": tts_key,
+            "use_rag": use_rag,
+            "session_id": session_id,
+            "doc_ids": doc_ids or [],
+            "audio_path": audio_path,
+        },
+        run_id=run_id,
+    )
+
+    audio, duration = load_audio_mono_16k(audio_path)
+    audio = clamp_duration(audio, turn_cap)
+    clipped_path = write_wav_temp(audio, out_base / "clips", stem=f"tv_{run_id}")
+    trace.log_note("audio_loaded", duration_seconds=duration, path=str(clipped_path))
+
+    asr = get_asr_backend(asr_key)
+    user_text = asr.transcribe(str(clipped_path), language=language).strip()
+    if not user_text:
+        raise ValueError("Could not transcribe speech — try speaking louder or uploading clearer audio.")
+    trace.log_note("asr_complete", preset=asr_key, chars=len(user_text))
+
+    from echocoach.omni import is_omni_profile, try_omni_turn
+
+    if is_omni_profile():
+        system = system_prompt_for_mode(mode)
+        topic_line = topic_context_block(topic, mode)
+        if topic_line:
+            system = f"{system}\n\n{topic_line}"
+        omni_user, omni_reply, omni_wav_or_note = try_omni_turn(
+            str(clipped_path),
+            language=language,
+            history=history,
+            system_prompt=system,
+        )
+        if omni_wav_or_note and omni_user and omni_reply and Path(omni_wav_or_note).is_file():
+            trace.log_note("omni_turn", path=omni_wav_or_note)
+            new_history = append_chat_turn(history, omni_user, omni_reply)
+            trace_path = trace.save()
+            return TeacherVoiceTurnResult(
+                user_text=omni_user,
+                assistant_text=omni_reply,
+                history=new_history,
+                voiceout_path=omni_wav_or_note,
+                voiceout_first_path=omni_wav_or_note,
+                voiceout_warning=None,
+                rag_references=None,
+                trace_path=str(trace_path),
+                trace=trace.to_dict(),
+            )
+        if omni_wav_or_note:
+            trace.log_note("omni_fallback", message=omni_wav_or_note)
+
+    rag: RagContext | None = None
+    rag_refs: str | None = None
+    if use_rag and mode in RAG_MODES:
+        sid = session_id
+        if not sid:
+            sid = IngestPipeline().store.create_session().id
+        rag = fetch_rag_context(user_text, session_id=sid, doc_ids=doc_ids)
+        if rag:
+            trace.log_note(
+                "rag_retrieve",
+                chunks=rag.chunk_count,
+                warning=rag.warning,
+            )
+            if rag.references_markdown:
+                rag_refs = rag.references_markdown
+
+    messages = build_teacher_messages(
+        mode=mode,
+        history=history,
+        user_text=user_text,
+        topic=topic,
+        rag=rag if rag and rag.context_block else None,
+    )
+    raw_reply = backend.chat(messages, max_tokens=512, temperature=0.5)
+    assistant_text = strip_reasoning_output(raw_reply).strip()
+    trace.log_llm(messages[-1]["content"], raw_reply)
+
+    if rag_refs:
+        assistant_text = f"{assistant_text}\n\n{rag_refs}"
+
+    voiceout_path, voiceout_first, voiceout_warning = synthesize_voice_reply(
+        strip_references_for_tts(assistant_text),
+        language=language,
+        tts_preset=tts_key,
+        chunk_first=True,
+        out_subdir="teacher_voice",
+    )
+    if voiceout_path:
+        trace.set_artifact(voiceout_path)
+
+    new_history = append_chat_turn(history, user_text, assistant_text)
+
+    trace_path = trace.save()
+    return TeacherVoiceTurnResult(
+        user_text=user_text,
+        assistant_text=assistant_text,
+        history=new_history,
+        voiceout_path=voiceout_path,
+        voiceout_first_path=voiceout_first,
+        voiceout_warning=voiceout_warning,
+        rag_references=rag_refs,
+        trace_path=str(trace_path),
+        trace=trace.to_dict(),
+    )

libs/echocoach/src/echocoach/tts/piper.py

@@ -45,30 +45,46 @@ class PiperTtsBackend:
         self._voices[voice_name] = voice
         return voice
 
-    def _resolve_voice_path(self, voice_name: str) -> Path:
+    def _voice_search_dirs(self) -> list[Path]:
+        dirs: list[Path] = []
         env_dir = __import__("os").environ.get("PIPER_VOICES_DIR")
-        candidates: list[Path] = []
         if env_dir:
-            candidates.append(Path(env_dir) / f"{voice_name}.onnx")
-        home = Path.home() / ".local" / "share" / "piper" / "voices"
-        candidates.append(home / f"{voice_name}.onnx")
-        candidates.append(Path.cwd() / "models" / "piper" / f"{voice_name}.onnx")
+            dirs.append(Path(env_dir))
+        dirs.extend(
+            [
+                Path.home() / ".local" / "share" / "piper" / "voices",
+                Path.cwd() / "models" / "piper",
+                Path.cwd(),
+            ]
+        )
+        return dirs
 
-        for path in candidates:
+    def _resolve_voice_path(self, voice_name: str) -> Path:
+        onnx_name = f"{voice_name}.onnx"
+        for directory in self._voice_search_dirs():
+            path = directory / onnx_name
             if path.is_file():
                 return path
 
+        download_targets = [
+            Path.cwd() / "models" / "piper",
+            Path.home() / ".local" / "share" / "piper" / "voices",
+        ]
         try:
             from piper.download_voices import download_voice
         except ImportError:
             download_voice = None
 
         if download_voice is not None:
-            try:
-                downloaded = download_voice(voice_name)
-                return Path(downloaded)
-            except Exception:
-                pass
+            for directory in download_targets:
+                directory.mkdir(parents=True, exist_ok=True)
+                try:
+                    download_voice(voice_name, directory)
+                    path = directory / onnx_name
+                    if path.is_file():
+                        return path
+                except Exception:
+                    continue
 
         import subprocess
         import sys
@@ -77,7 +93,8 @@ class PiperTtsBackend:
             [sys.executable, "-m", "piper.download_voices", voice_name],
             check=True,
         )
-        for path in candidates:
+        for directory in self._voice_search_dirs():
+            path = directory / onnx_name
             if path.is_file():
                 return path
 
@@ -118,6 +135,10 @@ class PiperTtsBackend:
 def get_tts_backend(preset_key: str | None = None) -> TtsBackend:
     config = get_echo_coach_config()
     preset = config.get_tts(preset_key)
+    if preset.backend == "vibevoice":
+        from echocoach.tts.vibevoice import get_vibevoice_backend
+
+        return get_vibevoice_backend(preset)
     if preset.key not in _tts_cache:
         _tts_cache[preset.key] = PiperTtsBackend(preset)
     return _tts_cache[preset.key]

libs/echocoach/src/echocoach/tts/vibevoice.py

@@ -0,0 +1,147 @@
+"""VibeVoice Realtime TTS backend (streaming, low latency).
+
+Model: https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B
+Falls back to Piper when the model is unavailable or not yet wired end-to-end.
+"""
+
+from __future__ import annotations
+
+import uuid
+import wave
+from pathlib import Path
+from typing import Protocol
+
+from echocoach.config import TtsPreset, get_echo_coach_config, outputs_dir
+
+_VIBEVOICE_DOC = "https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B"
+
+
+class TtsBackend(Protocol):
+    def synthesize(self, text: str, *, language: str, out_dir: Path | None = None) -> tuple[str | None, str | None]: ...
+
+
+_vibevoice_cache: dict[str, "VibeVoiceTtsBackend"] = {}
+
+
+class VibeVoiceTtsBackend:
+    def __init__(self, preset: TtsPreset) -> None:
+        self._preset = preset
+        self._pipeline = None
+        self._load_error: str | None = None
+
+    @property
+    def model_id(self) -> str:
+        return self._preset.model_id or "microsoft/VibeVoice-Realtime-0.5B"
+
+    def _resolve_language(self, language: str) -> tuple[str, str | None]:
+        supported = set(self._preset.supported_languages)
+        if supported and language not in supported:
+            fallback = self._preset.fallback_voice
+            return fallback, (
+                f"Language {language!r} is not listed for VibeVoice; using {fallback!r}."
+            )
+        return language, None
+
+    def _try_load(self) -> str | None:
+        if self._pipeline is not None:
+            return None
+        if self._load_error is not None:
+            return self._load_error
+
+        try:
+            from transformers import pipeline
+        except ImportError:
+            self._load_error = "VibeVoice requires transformers and torch."
+            return self._load_error
+
+        try:
+            self._pipeline = pipeline(
+                "text-to-speech",
+                model=self.model_id,
+                trust_remote_code=True,
+            )
+            return None
+        except Exception as exc:  # noqa: BLE001
+            self._load_error = (
+                f"VibeVoice load failed ({self.model_id}): {exc}. "
+                f"See {_VIBEVOICE_DOC} for setup; using Piper fallback."
+            )
+            return self._load_error
+
+    def _write_wav(self, audio: object, sample_rate: int, out_path: Path) -> None:
+        import numpy as np
+
+        samples = np.asarray(audio, dtype=np.float32)
+        if samples.ndim > 1:
+            samples = samples.squeeze()
+        pcm = np.clip(samples, -1.0, 1.0)
+        pcm_i16 = (pcm * 32767).astype(np.int16)
+        with wave.open(str(out_path), "wb") as wav_file:
+            wav_file.setnchannels(1)
+            wav_file.setsampwidth(2)
+            wav_file.setframerate(sample_rate)
+            wav_file.writeframes(pcm_i16.tobytes())
+
+    def synthesize(
+        self,
+        text: str,
+        *,
+        language: str,
+        out_dir: Path | None = None,
+    ) -> tuple[str | None, str | None]:
+        if not text.strip():
+            return None, "No text to synthesize."
+
+        lang, lang_warning = self._resolve_language(language)
+        load_warning = self._try_load()
+        if load_warning or self._pipeline is None:
+            return self._piper_fallback(text, language=lang, out_dir=out_dir, prefix=load_warning)
+
+        base = out_dir or outputs_dir()
+        base.mkdir(parents=True, exist_ok=True)
+        out_path = base / f"vibevoice_{uuid.uuid4().hex[:10]}.wav"
+
+        try:
+            result = self._pipeline(text)
+            if isinstance(result, dict):
+                audio = result.get("audio") or result.get("array")
+                sample_rate = int(result.get("sampling_rate") or result.get("sample_rate") or 24000)
+            else:
+                audio, sample_rate = result  # type: ignore[misc]
+            if audio is None:
+                raise ValueError("empty audio from VibeVoice pipeline")
+            self._write_wav(audio, sample_rate, out_path)
+            warning = lang_warning
+            if self._preset.realtime:
+                note = f"VibeVoice Realtime ({self.model_id})"
+                warning = f"{lang_warning} {note}" if lang_warning else note
+            return str(out_path), warning
+        except Exception as exc:  # noqa: BLE001
+            return self._piper_fallback(
+                text,
+                language=lang,
+                out_dir=out_dir,
+                prefix=f"VibeVoice synthesis failed: {exc}. Piper fallback.",
+            )
+
+    def _piper_fallback(
+        self,
+        text: str,
+        *,
+        language: str,
+        out_dir: Path | None,
+        prefix: str | None,
+    ) -> tuple[str | None, str | None]:
+        config = get_echo_coach_config()
+        from echocoach.tts.piper import get_tts_backend
+
+        piper = get_tts_backend(config.tts_preset)
+        path, piper_warning = piper.synthesize(text, language=language, out_dir=out_dir)
+        parts = [p for p in (prefix, piper_warning) if p]
+        return path, "; ".join(parts) if parts else None
+
+
+def get_vibevoice_backend(preset: TtsPreset) -> VibeVoiceTtsBackend:
+    if preset.key not in _vibevoice_cache:
+        _vibevoice_cache[preset.key] = VibeVoiceTtsBackend(preset)
+    return _vibevoice_cache[preset.key]

libs/echocoach/src/echocoach/voiceout.py

@@ -0,0 +1,109 @@
+"""Reusable VoiceOut helpers for TeacherVoice, Chat, and other tabs."""
+
+from __future__ import annotations
+
+import re
+
+from echocoach.config import get_echo_coach_config, outputs_dir
+from echocoach.tts.piper import get_tts_backend
+
+
+def strip_references_for_tts(text: str) -> str:
+    """Remove citation/reference blocks before TTS."""
+    cleaned = text
+    for marker in ("\n\n## References", "\n\n**References**"):
+        if marker in cleaned:
+            cleaned = cleaned.split(marker, 1)[0]
+    return cleaned.strip()
+
+
+def extract_message_text(content: object) -> str:
+    """Normalize Gradio chat content (plain string or message blocks) to text."""
+    if content is None:
+        return ""
+    if isinstance(content, str):
+        return content.strip()
+    if isinstance(content, (list, tuple)):
+        parts: list[str] = []
+        for block in content:
+            if isinstance(block, str):
+                text = block.strip()
+            elif isinstance(block, dict):
+                text = str(block.get("text") or block.get("content") or "").strip()
+            else:
+                text = str(block).strip()
+            if text:
+                parts.append(text)
+        return "\n".join(parts)
+    return str(content).strip()
+
+
+def last_assistant_message(history: list) -> str | None:
+    """Return the most recent assistant message from Gradio chat history."""
+    for item in reversed(history or []):
+        if isinstance(item, dict) and item.get("role") == "assistant":
+            content = extract_message_text(item.get("content"))
+            return content or None
+        if isinstance(item, (list, tuple)) and len(item) == 2 and item[1]:
+            return extract_message_text(item[1]) or None
+    return None
+
+
+def split_sentences(text: str) -> list[str]:
+    parts = re.split(r"(?<=[.!?])\s+", text.strip())
+    return [p.strip() for p in parts if p.strip()]
+
+
+def synthesize_voice_reply(
+    text: str,
+    *,
+    language: str,
+    tts_preset: str | None = None,
+    chunk_first: bool = True,
+    out_subdir: str = "voiceout",
+) -> tuple[str | None, str | None, str | None]:
+    """Return (full_wav, first_sentence_wav, warning)."""
+    if not text.strip():
+        return None, None, "No text to synthesize."
+
+    config = get_echo_coach_config()
+    preset = tts_preset or config.tts_preset
+    tts = get_tts_backend(preset)
+    out_dir = outputs_dir() / out_subdir
+    full_path, warning = tts.synthesize(text, language=language, out_dir=out_dir)
+
+    first_path = None
+    if chunk_first:
+        sentences = split_sentences(text)
+        if len(sentences) > 1:
+            first_path, first_warning = tts.synthesize(
+                sentences[0],
+                language=language,
+                out_dir=out_dir,
+            )
+            if first_warning and not warning:
+                warning = first_warning
+        elif full_path:
+            first_path = full_path
+
+    return full_path, first_path, warning
+
+
+def speak_assistant_text(
+    text: str,
+    *,
+    language: str = "en",
+    tts_preset: str | None = None,
+    first_sentence_only: bool = False,
+) -> tuple[str | None, str | None, str | None]:
+    """Synthesize assistant reply audio. Returns (playback_path, alt_path, warning)."""
+    clean = strip_references_for_tts(text)
+    full_path, first_path, warning = synthesize_voice_reply(
+        clean,
+        language=language,
+        tts_preset=tts_preset,
+        chunk_first=True,
+    )
+    if first_sentence_only:
+        return first_path or full_path, full_path, warning
+    return full_path or first_path, first_path, warning

libs/echocoach/tests/test_teacher_voice.py

@@ -0,0 +1,203 @@
+"""Tests for TeacherVoice prompt assembly and message building."""
+
+from __future__ import annotations
+
+import numpy as np
+import pytest
+import soundfile as sf
+
+from echocoach.prompts import PITCH_SYSTEM, system_prompt_for_mode
+from echocoach.teacher_voice import (
+    RagContext,
+    append_chat_turn,
+    build_teacher_messages,
+    fetch_rag_context,
+    history_to_messages,
+)
+from echocoach.voiceout import (
+    extract_message_text,
+    last_assistant_message,
+    split_sentences,
+    strip_references_for_tts,
+)
+
+
+class _MockBackend:
+    def load(self) -> None:
+        pass
+
+    def chat(self, messages, *, max_tokens=512, temperature=0.7):
+        assert messages[0]["role"] == "system"
+        assert messages[-1]["role"] == "user"
+        return "Plants use sunlight to make food."
+
+    def generate(self, prompt, *, max_tokens=512, temperature=0.7):
+        return self.chat([{"role": "user", "content": prompt}], max_tokens=max_tokens)
+
+
+def test_append_chat_turn_messages_format():
+    from echocoach.teacher_voice import append_chat_turn
+
+    history = append_chat_turn([], "Hi", "Hello")
+    assert history == [
+        {"role": "user", "content": "Hi"},
+        {"role": "assistant", "content": "Hello"},
+    ]
+
+    extended = append_chat_turn(history, "Next?", "Sure.")
+    assert extended == [
+        {"role": "user", "content": "Hi"},
+        {"role": "assistant", "content": "Hello"},
+        {"role": "user", "content": "Next?"},
+        {"role": "assistant", "content": "Sure."},
+    ]
+
+
+def test_append_chat_turn_migrates_legacy_tuples():
+    from echocoach.teacher_voice import append_chat_turn
+
+    legacy = [("Old question", "Old answer")]
+    history = append_chat_turn(legacy, "New?", "New reply.")
+    assert history[-2:] == [
+        {"role": "user", "content": "New?"},
+        {"role": "assistant", "content": "New reply."},
+    ]
+    assert history[0] == {"role": "user", "content": "Old question"}
+
+
+def test_history_to_messages_tuple_pairs():
+    history = [("Hi", "Hello"), ("What is AI?", "Machine learning.")]
+    messages = history_to_messages(history)
+    assert messages == [
+        {"role": "user", "content": "Hi"},
+        {"role": "assistant", "content": "Hello"},
+        {"role": "user", "content": "What is AI?"},
+        {"role": "assistant", "content": "Machine learning."},
+    ]
+
+
+def test_build_teacher_messages_includes_topic_and_rag():
+    rag = RagContext(
+        context_block="[1] Plants need light.",
+        references_markdown="**References**\n[1] Biology",
+        chunk_count=1,
+    )
+    messages = build_teacher_messages(
+        mode="lesson",
+        history=[],
+        user_text="How do plants eat?",
+        topic="Photosynthesis",
+        rag=rag,
+    )
+    assert "TeacherVoice" in messages[0]["content"]
+    assert "lesson-planning" in messages[0]["content"]
+    assert "Photosynthesis" in messages[0]["content"]
+    assert "[1] Plants need light." in messages[-1]["content"]
+    assert messages[-1]["content"].endswith("How do plants eat?")
+
+
+def test_pitch_mode_system_prompt():
+    assert "EchoCoach" in system_prompt_for_mode("pitch")
+    assert PITCH_SYSTEM == system_prompt_for_mode("pitch")
+
+
+def test_split_sentences():
+    text = "Hello there. How are you? Great!"
+    assert split_sentences(text) == ["Hello there.", "How are you?", "Great!"]
+
+
+def test_extract_message_text():
+    assert extract_message_text("Hello") == "Hello"
+    assert extract_message_text([{"text": "Hello there."}]) == "Hello there."
+    assert extract_message_text([{"text": "A"}, {"text": "B"}]) == "A\nB"
+
+
+def test_last_assistant_message():
+    history = [
+        {"role": "user", "content": "Hi"},
+        {"role": "assistant", "content": "Hello there."},
+    ]
+    assert last_assistant_message(history) == "Hello there."
+    assert last_assistant_message([]) is None
+    gradio_history = [
+        {"role": "user", "content": [{"text": "Hi"}]},
+        {"role": "assistant", "content": [{"text": "Hello there."}]},
+    ]
+    assert last_assistant_message(gradio_history) == "Hello there."
+
+
+def test_vibevoice_preset_in_voice_models():
+    from echocoach.config import get_echo_coach_config
+
+    config = get_echo_coach_config(reload=True)
+    preset = config.get_tts("vibevoice-realtime-0.5b")
+    assert preset.backend == "vibevoice"
+    assert preset.model_id == "microsoft/VibeVoice-Realtime-0.5B"
+    assert preset.realtime is True
+    assert preset.streaming is True
+    assert "en" in preset.supported_languages
+    assert config.realtime_tts_preset == "vibevoice-realtime-0.5b"
+
+
+def test_strip_references_for_tts():
+    text = "Answer here.\n\n**References**\n[1] Source"
+    assert strip_references_for_tts(text) == "Answer here."
+
+
+def test_fetch_rag_context_empty_store_warns(research_env):
+    ctx = fetch_rag_context("What is photosynthesis?", session_id="", doc_ids=None)
+    assert ctx is not None
+    assert ctx.chunk_count == 0
+    assert ctx.warning
+
+
+@pytest.fixture
+def research_env(tmp_path, monkeypatch):
+    from researchmind.config import ResearchMindConfig
+
+    cfg = ResearchMindConfig(
+        data_dir=tmp_path / "rm",
+        embed_model="test",
+        auto_search=False,
+        top_k=2,
+        max_context_chunks=8,
+        chunk_size=50,
+        chunk_overlap=10,
+    )
+    monkeypatch.setenv("RESEARCHMIND_DATA_DIR", str(cfg.data_dir))
+    monkeypatch.setenv("AGENT_OUTPUTS_DIR", str(tmp_path / "outputs"))
+
+
+def test_run_teacher_voice_turn_mock_asr(monkeypatch, tmp_path):
+    from echocoach.teacher_voice import run_teacher_voice_turn
+
+    wav = tmp_path / "turn.wav"
+    sf.write(wav, np.zeros(16_000, dtype=np.float32), 16_000)
+
+    class _Asr:
+        def transcribe(self, path, *, language="en"):
+            return "Tell me about plants."
+
+    class _Tts:
+        def synthesize(self, text, *, language, out_dir=None):
+            out = (out_dir or tmp_path) / "out.wav"
+            out.parent.mkdir(parents=True, exist_ok=True)
+            sf.write(out, np.zeros(8000, dtype=np.float32), 16_000)
+            return str(out), None
+
+    monkeypatch.setattr("echocoach.teacher_voice.get_asr_backend", lambda _: _Asr())
+    monkeypatch.setattr("echocoach.voiceout.get_tts_backend", lambda _: _Tts())
+
+    result = run_teacher_voice_turn(
+        str(wav),
+        [],
+        mode="explain",
+        backend=_MockBackend(),
+        use_rag=False,
+    )
+    assert result.user_text == "Tell me about plants."
+    assert "sunlight" in result.assistant_text
+    assert len(result.history) == 2
+    assert result.history[0]["role"] == "user"
+    assert result.history[1]["role"] == "assistant"
+    assert result.trace.get("skill") == "teacher-voice"

uv.lock

@@ -1164,7 +1164,7 @@ version = "0.1.0"
 source = { editable = "apps/gradio-space" }
 dependencies = [
     { name = "agent" },
-    { name = "echocoach", extra = ["whisper"] },
+    { name = "echocoach", extra = ["piper", "whisper"] },
     { name = "gradio" },
     { name = "inference" },
 ]
@@ -1172,7 +1172,7 @@ dependencies = [
 [package.metadata]
 requires-dist = [
     { name = "agent", editable = "libs/agent" },
-    { name = "echocoach", extras = ["whisper"], editable = "libs/echocoach" },
+    { name = "echocoach", extras = ["piper", "whisper"], editable = "libs/echocoach" },
     { name = "gradio", specifier = ">=5.0.0" },
     { name = "inference", editable = "libs/inference" },
 ]

voice_models.yaml

@@ -4,6 +4,8 @@
 defaults:
   asr_preset: whisper-cpp-tiny
   tts_preset: piper-multilingual
+  # Realtime streaming TTS for TeacherVoice VoiceOut (set ECHOCOACH_TTS_PRESET to match)
+  realtime_tts_preset: vibevoice-realtime-0.5b
   coach_model: minicpm5-1b
   max_seconds: 30
 
@@ -53,6 +55,13 @@ asr:
     backend: whisper_cpp
     model_size: base
 
+# Optional omni speech-in/speech-out (GPU; set ECHOCOACH_VOICE_PROFILE=omni)
+omni:
+  minicpm-o-4.5:
+    label: MiniCPM-o 4.5 (GPU omni — falls back to pipeline until wired)
+    model_id: openbmb/MiniCPM-o-4_5
+    languages: [en, zh]
+
 tts:
   piper-multilingual:
     label: Piper TTS (local VoiceOut)
@@ -73,3 +82,16 @@ tts:
       vi: vi_VN-25hours-single
       ko: ko_KR-kss-medium
     fallback_voice: en_US-lessac-medium
+
+  # Microsoft VibeVoice Realtime — streaming TTS, ~300ms to first audio (GPU recommended).
+  # https://huggingface.co/microsoft/VibeVoice-Realtime-0.5B
+  # English-first; also supports de, fr, it, es, pt, nl, pl, ja, ko (experimental).
+  # Requires transformers + torch; falls back to Piper until fully wired in echocoach.
+  vibevoice-realtime-0.5b:
+    label: VibeVoice Realtime 0.5B (streaming, ~300ms)
+    backend: vibevoice
+    model_id: microsoft/VibeVoice-Realtime-0.5B
+    streaming: true
+    realtime: true
+    languages: [en, de, fr, it, es, pt, nl, pl, ja, ko]
+    fallback_language: en