JavaScript SDK runtime reference

Building a Reachy Mini JS app? The single source of truth is ts/APP_CREATION_GUIDE.md at the repo root. It covers scaffolding, public/icon.svg, the host shell, sdk: static deploy, mountHost() / connectToHost() API, local dev, FAQ, and the host ↔ embed contract. Pin the SDK to @pollen-robotics/reachy-mini-sdk@1.8.0 (the stable release validated against the host shell + daemon).

This file is the runtime API surface of the ReachyMini class you receive from handle.reachy once connectToHost() resolves: methods, events, properties, state machine, and the daemon-side recorded-move playback API. Bookmark it after you’ve shipped a first app from the guide.

Reachy Mini ships a browser SDK that drives a robot over WebRTC. The npm package @pollen-robotics/reachy-mini-sdk exposes:

• The ReachyMini class (the SDK runtime documented below).
• The host shell + embed adapter under the ./host* subpath exports (./host, ./host/auto, ./host/embed, ./host/protocol). See ts/APP_CREATION_GUIDE.md for the integration recipe.

Architecture

┌─────────────────────────────────┐
│  Browser                        │
│  (your app + reachy-mini-sdk.js)│
└───────┬────────────┬────────────┘
        │ SSE/HTTP   │ WebRTC (peer-to-peer)
        │ signaling  │ video + audio + data
┌───────▼──────┐     │
│  Signaling   │     │
│  Server      │     │
│  (HF Space)  │     │
└───────┬──────┘     │
        │            │
┌───────▼────────────▼────────────┐
│  Robot                          │
│  GStreamer WebRTC daemon        │
│  camera · mic · motors          │
└─────────────────────────────────┘

1. Your app is a static HTML/JS page hosted on Hugging Face Spaces.
2. The SDK handles authentication, signaling, and WebRTC negotiation.
3. The signaling server relays SDP offers/answers and ICE candidates and validates Hugging Face OAuth tokens.
4. Once the WebRTC connection is established, video, audio, and commands flow peer-to-peer; the signaling server is no longer in the path.

When you use the host shell (mountHost() + connectToHost(), documented in ts/APP_CREATION_GUIDE.md), the steps below are handled for you. The class-level API documented here is what you use after connectToHost() resolves, or what you call directly if you opted out of the host shell.

API Reference

Constructor

new ReachyMini({
    signalingUrl: "https://pollen-robotics-reachy-mini-central.hf.space",  // default
    clientId: "my-app",              // optional — OAuth client / app id
    appName: "My App",               // optional — label advertised to the robot
    videoJitterBufferTargetMs: 0,    // optional — 0 = render ASAP (teleop); omit for default browser buffering
    autoStartFromUrl: false,         // optional — auto startSession() when the URL carries a robot_peer_id hint
    // NOTE: `enableMicrophone` is deprecated and ignored. The SDK never grabs
    // the user's mic; it always wires a silent placeholder audio sender you
    // can replaceTrack() with your own audio (TTS, files, or the user's mic).
})

State Machine

'disconnected' ──connect()──▸ 'connected' ──startSession()──▸ 'streaming'
     ▴ disconnect()                ▴ stopSession()
     └─────────────────────────────┘

One-shot bring-up. autoConnect(options?) runs the whole auth → connect() → robot pick → startSession() → ensureAwake() chain and resolves to { robotId, robotName, isEmbedded }. It auto-picks when exactly one robot is free, otherwise calls your options.pickRobot(robots) callback. This is the recommended entry point when you’re not using the host shell.

Properties (read-only)

Methods

setTarget head-vs-body coupling. The head matrix is in the world frame. Sending setTarget({ body_yaw }) alone rotates the body but not the head’s commanded world yaw — the head’s gaze stays fixed in world frame, so visually it appears to counter-rotate as the body turns. For tank-style “head follows body”, add the body yaw delta to the head RPY’s yaw and ship head + body_yaw in the same setTarget call. The baseline for the head yaw must be the last commanded value you tracked yourself, not state.head from the telemetry event — telemetry lags one WebRTC RTT and cumulative deltas computed against it stall under rapid input.

Events

Use robot.addEventListener(name, handler) — the SDK extends EventTarget.

Math Utilities

import { rpyToMatrix, matrixToRpy, degToRad, radToDeg } from "@pollen-robotics/reachy-mini-sdk";

rpyToMatrix(roll, pitch, yaw)  // degrees → 4×4 rotation matrix (ZYX)
matrixToRpy(matrix)            // 4×4 matrix → { roll, pitch, yaw } in degrees

Advanced: JSON-RPC & daemon admin

Beyond the typed surface above, the runtime object exposes lower-level hooks (not part of the host-shell typed handle — reach for these only when you need them):

• rpcCall(method, params?, { timeoutMs? }) — send a JSON-RPC request over the data channel and await the correlated result (e.g. app-defined methods).
• onNotification(method, cb) — subscribe to one-way JSON-RPC notifications pushed by the robot/app (e.g. conversation.turn); returns an unsubscribe fn.
• startDaemonUpdate({ preRelease?, onProgress? }) — trigger a PyPI update of the daemon. It restarts on success (which tears the session down), so treat a successful reconnect as the “done” signal; onProgress fires with status: "failed" if the install errors first.

Debug logging

Every SDK log line is prefixed [reachy:<ns>] (reachy:sdk, reachy:session for auto-reconnect, reachy:embed / reachy:host for the host shell), so a devtools console filter on reachy: isolates the whole stack. The default level is info: a handful of lifecycle lines per session (boot phases, wake/sleep steps, reconnects). To see per-message traffic (every command, reply, and SSE event):

// From the devtools console (persists across reloads):
localStorage.setItem("reachy-log", "debug"); location.reload();

// Or from code:
import { setLogLevel } from "@pollen-robotics/reachy-mini-sdk";
setLogLevel("debug"); // "debug" | "info" | "warn" | "error" | "silent"

Debug lines use console.debug, so also enable the “Verbose” level in the devtools console filter to see them.

Daemon-side recorded-move playback

Long recorded moves (and any move with audio) should play server-side on the daemon’s local clock, not by streaming set_target frames from the browser. The browser uploads the move once over the WebRTC data channel and the daemon ticks the inner loop at the requested frequency — no per-frame round-trip, smooth on wireless robots. When audio is attached the daemon plays it on the same GStreamer pipeline, so motion and audio share a single clock (no cross-network drift).

Combined motion + audio

const result = await robot.playMove(motion, {
    audioBlob,                    // optional, 16 kHz mono PCM WAV
    audioLeadMs: -100,            // system-wide default
    description: "happy wave",
    onProgress: (p) => console.log(p.phase, p.sent, p.total),
    onStarted: ({ duration_s, has_audio }) => { /* sync anchor */ },
});
// result is { finished: true } | { cancelled: true } | { error: "..." }

// Cancel at any time from another code path:
robot.cancelMove();

motion is the shape the Python RecordedMove parser expects:

{ time: [0, 0.01, 0.02, …], set_target_data: [{ head, antennas, body_yaw }, …] }

audioLeadMs shifts audio relative to motion at the daemon:

• Positive — audio fires N ms BEFORE motion (compensates motor pickup).
• Negative — motion fires N ms BEFORE audio (compensates GStreamer playbin warmup).
• Default -100 is the empirical system-wide constant (combined motor + pipeline). Tune only after measuring.

The encoded wire form defaults to gzip+base64 (typically ~3× smaller for recorded-move JSON). Falls back to plain JSON if the browser lacks CompressionStream.

Record-time audio (sync anchor)

For recording flows that want the SAME audio pipeline at capture AND replay (so pipeline latency cancels out and one audioLeadMs works for all recordings):

// 1. During the countdown — upload the source audio.
const audioId = await robot.uploadAudio(audioBlob, { description: "song" });

// 2. At the GO! moment — kick off daemon-side playback, await the
//    started broadcast, then start motion capture.
await robot.playUploadedAudio(audioId);
const captureT0 = performance.now();
startMyMotionCapture();

// 3. On stop / cancel / restart — stop the audio.
robot.cancelAudio();

The daemon does NOT emit a finished event for standalone audio; callers know the duration from the WAV header and call cancelAudio() when done.

Audio format

Audio must be canonical 16 kHz mono 16-bit PCM WAV. Apps are responsible for normalizing before upload — the daemon does not transcode. Format mismatch is a frequent cause of “audio is silent / wrong speed” on inherited datasets.

Backpressure & cancellation

playMove and uploadAudio pace chunk sends on the data channel’s bufferedAmount so multi-megabyte uploads (a 3-min song’s WAV is ~6 MB base64) don’t degrade other channels on the same peer connection. There’s no separate pause — to stop a long upload mid-way, close the session.

Live pose stream (30 Hz)

By default the SDK refreshes robotState by polling requestState() every 500 ms. For anything that mirrors the robot in real time (a 3D view, a “wait for the move to finish” watcher), opt into the daemon’s pose stream instead: it pushes the state at ~30 Hz over a dedicated unreliable/unordered data channel, firing the same state events.

robot.subscribePose();               // start the push stream
robot.addEventListener("state", (e) => {
    // e.detail.head_joint_positions → per-motor radians (body yaw + 6 neck)
    render(e.detail);
});

// Later, when this consumer no longer needs the stream:
robot.unsubscribePose();

The subscription is refcounted: multiple consumers share one daemon-side stream, so pair every subscribePose() with exactly one unsubscribePose() — the daemon only stops pushing once the last consumer releases. Frames that arrive out of order are dropped (the channel is unordered). Against an older daemon that has no pose channel this is a no-op; fall back to requestState() polling there.

Handling backgrounded tabs

When the tab is hidden (tab switch, phone lock), the browser pauses requestAnimationFrame entirely and clamps setInterval/setTimeout to ~1 tick per second. Any robot-critical loop clocked by them - pose streaming via setTarget(), audio gain ramps, stream-health watchdogs, reconnect logic - freezes mid-motion until the tab comes back.

Split your loop in two:

• Logic (pose computation + setTarget, audio, health checks, reconnects): clock it from a Web Worker. Worker timers are not visibility-throttled, and the message events they post are delivered on the main thread even while the tab is hidden.
• Visuals (DOM updates, canvas, meters): keep them on requestAnimationFrame. Pausing invisible paints is exactly what you want, and it resumes on its own.

// pose-heartbeat.worker.js - the whole file:
setInterval(() => postMessage(0), 25); // ~40 Hz; the main thread down-samples

// app side
let worker = null;
try {
    worker = new Worker(new URL("./pose-heartbeat.worker.js", import.meta.url), { type: "module" });
    worker.onmessage = () => stepLogic(performance.now());
} catch {
    // Workers unavailable (rare): degrade to a throttled interval.
    setInterval(() => stepLogic(performance.now()), 25);
}

Two companion rules make this robust:

• Clamp your dt. After a long hidden stretch the first tick sees a huge time delta; clamp it (e.g. Math.min(dt, 100)) so filters and interpolators don’t jump.
• Resync on visibilitychange. When the tab returns, call robot.requestState() and run one logic step immediately instead of waiting for the next scheduled tick, so the UI repaints from fresh state.

Remember to worker.terminate() in your teardown. Full rationale and the host-shell variant live in ts/APP_CREATION_GUIDE.md §14.7; the pollen-robotics/sdk-js-demo-app Space meters the throttling live (Background resilience panel) and clocks its own pose editor off a worker.

Robot onboarding & management

These daemon-side commands power first-run setup and remote administration. Each resolves null when the data channel isn’t open or the daemon predates the command, so fail-open: treat null as “unsupported” and skip the gated UI.

// First wake-up wizard (robot-wide, persisted on the robot).
if ((await robot.getFirstWakeUp()) === false) {
    await runWakeUpWizard();
    await robot.setFirstWakeUp(true);
}

// Robot display name — applied live (status + central relay + mDNS), no restart.
const name = await robot.getRobotName();
await robot.setRobotName("Marvin");

// Sign the robot out of Hugging Face (deletes its stored token; the robot
// disappears from its owner's list until set up again). The session may
// tear down right after the ack — that's expected.
await robot.signOut();

Security

• Authentication goes through Hugging Face OAuth — only users logged in to HF can access the signaling server.
• By default, you can only connect to robots registered under your own HF account.
• WebRTC connections are encrypted (DTLS/SRTP).

Prerequisites

• Your robot must be running the wireless firmware and connected to the central signaling server.
• The robot must have a valid Hugging Face token configured (see Usage).
• Currently supported on wireless versions only.

Working examples

The three reference apps maintained alongside the SDK are the canonical worked examples. They all use the host shell pattern and the current SDK pin:

• pollen-robotics/reachy_mini_minimal_conversation — vanilla TS + Vite.
• pollen-robotics/reachy_mini_emotions — React 19 + MUI 7 + Vite.
• pollen-robotics/reachy_mini_telepresence — React 19 + MUI 7 + Vite with camera + media streams.

Clone the closest one and trim. See ts/APP_CREATION_GUIDE.md for the step-by-step.