Media

class reachy_mini.media.media_manager.MediaManager

( backend: MediaBackend = <MediaBackend.GSTREAMER: 'gstreamer'> log_level: str = 'INFO' use_sim: bool = False signalling_host: str = 'localhost' )

Parameters

logger (logging.Logger) — Logger instance for media-related messages.
backend (MediaBackend) — The selected media backend.
camera (Optional[CameraBase]) — Camera device instance.
audio (Optional[AudioBase]) — Audio device instance.

Media Manager for handling camera and audio devices.

This class provides a unified interface for managing both camera and audio devices across different backends. It handles initialization, configuration, and cleanup of media resources.

close

< source >

( )

Close the media manager and release resources.

This method should be called when the media manager is no longer needed to properly clean up and release all media resources. It stops any ongoing audio recording/playback and closes the camera device.

Note: After calling this method, the media manager can be reused by calling the appropriate initialization methods again, but it’s generally recommended to create a new MediaManager instance if needed.

Example:

media = MediaManager()
try:
    # Use media devices
    frame = media.get_frame()
finally:
    media.close()

get_DoA

< source >

( ) → tuple[float, bool] | None

Returns

tuple[float, bool] | None

A tuple (angle_radians, speech_detected), or None if the audio system is not available.

Get the Direction of Arrival (DoA) from the microphone array.

get_audio_sample

< source >

( ) → Optional[np.ndarray]

Returns

Optional[np.ndarray]

The recorded audio sample, or None if no data is available.

Get an audio sample from the audio device.

get_frame

< source >

( ) → Optional[npt.NDArray[np.uint8]]

Returns

Optional[npt.NDArray[np.uint8]]

The captured BGR frame as a numpy array with shape (height, width, 3), or None if the camera is not available or an error occurred.

The image is in BGR format (OpenCV convention) and can be directly used with OpenCV functions or converted to RGB if needed.

Get a frame from the camera.

Note: This method returns None if the camera is not initialized or if there’s an error capturing the frame. Always check the return value before using the frame.

Example:

frame = media.get_frame()
if frame is not None:
    # Process the frame
    cv2.imshow("Camera", frame)
    cv2.waitKey(1)

    # Convert to RGB if needed
    rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

get_input_audio_samplerate

< source >

( )

Get the input samplerate of the audio device.

get_input_channels

< source >

( )

Get the number of input channels of the audio device.

get_output_audio_samplerate

< source >

( )

Get the output samplerate of the audio device.

get_output_channels

< source >

( )

Get the number of output channels of the audio device.

play_sound

< source >

( sound_file: str )

Parameters

sound_file (str) — Path to the sound file to play.

Play a sound file.

push_audio_sample

< source >

( data: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float32]] )

Parameters

data (npt.NDArray[np.float32]) — The audio data to push to the output device (mono format).

Push audio data to the output device.

start_playing

< source >

( )

Start playing audio.

start_recording

< source >

( )

Start recording audio.

stop_playing

< source >

( )

Stop playing audio.

stop_recording

< source >

( )

Stop recording audio.

class reachy_mini.media.audio_base.AudioBase

< source >

( log_level: str = 'INFO' )

Parameters

SAMPLE_RATE (int) — Default sample rate for audio operations (16000 Hz).
CHANNELS (int) — Default number of audio channels (2 for stereo).
logger (logging.Logger) — Logger instance for audio-related messages.
_respeaker (Optional[ReSpeaker]) — ReSpeaker microphone array device handler.

Abstract class for opening and managing audio devices.

This class defines the interface that all audio implementations must follow. It provides common audio parameters and methods for managing audio devices, including microphone input and speaker output functionality.

cleanup

< source >

( )

Cleanup resources before destruction.

This method should be called to release any resources held by the audio implementation before the object is destroyed.

clear_output_buffer

< source >

( )

Clear the output buffer.

This method flushes the output buffer to prevent push samples from being played. Overwrite if necessary. It seems that set_max_output_buffers with a low value may be enough for gstreamer backend.

get_DoA

< source >

( ) → tuple

Returns

tuple

A tuple containing the DoA value as a float (radians) and the speech detection as a bool, or None if the device is not found.

Get the Direction of Arrival (DoA) value from the ReSpeaker device.

The spatial angle is given in radians: 0 radians is left, π/2 radians is front/back, π radians is right.

Note: The microphone array requires firmware version 2.1.0 or higher to support this feature. The firmware is located in src/reachy_mini/assets/firmware/*.bin. Refer to https://wiki.seeedstudio.com/respeaker_xvf3800_introduction/#update-firmware for the upgrade process.

get_audio_sample

< source >

( ) → Optional[npt.NDArray[np.float32]]

Returns

Optional[npt.NDArray[np.float32]]

A numpy array containing audio samples in float32 format, or None if no data is available or an error occurred.

The array shape is typically (num_samples,) for mono or (num_samples, num_channels) for multi-channel audio.

Read audio data from the device. Returns the data or None if error.

Note: This method should be called after start_recording() has been called. The sample rate and number of channels can be obtained via get_input_audio_samplerate() and get_input_channels() respectively.

Example:

audio.start_recording()
samples = audio.get_audio_sample()
if samples is not None:
    print(f"Got {len(samples)} audio samples")

get_input_audio_samplerate

< source >

( ) → int

Returns

int

The sample rate in Hz at which audio is being captured. Default is 16000 Hz.

Get the input samplerate of the audio device.

Note: This value represents the number of audio samples captured per second for each channel.

get_input_channels

< source >

( ) → int

Returns

int

The number of audio input channels (e.g., 1 for mono, 2 for stereo). Default is 2 channels.

Get the number of input channels of the audio device.

Note: For the ReSpeaker microphone array, this typically returns 2 channels representing the stereo microphone configuration.

get_output_audio_samplerate

< source >

( ) → int

Returns

int

The sample rate in Hz at which audio is being played back. Default is 16000 Hz.

Get the output samplerate of the audio device.

Note: This value represents the number of audio samples played per second for each channel.

get_output_channels

< source >

( ) → int

Returns

int

The number of audio output channels (e.g., 1 for mono, 2 for stereo). Default is 2 channels.

Get the number of output channels of the audio device.

Note: This determines how audio data should be formatted when passed to push_audio_sample() method.

play_sound

< source >

( sound_file: str )

Parameters

sound_file (str) — Path to the sound file to play. Supported formats depend on the specific implementation.

Play a sound file.

Note: This is a convenience method that handles the complete playback of a sound file from start to finish. For more control over audio playback, use start_playing(), push_audio_sample(), and stop_playing() methods.

Example:

audio.play_sound("/path/to/sound.wav")

push_audio_sample

< source >

( data: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float32]] )

Parameters

data (npt.NDArray[np.float32]) — Audio samples to be played. The array should contain float32 values typically in the range [-1.0, 1.0].

For mono audio: shape should be (num_samples,) For stereo audio: shape should be (num_samples, 2)

Push audio data to the output device.

Note: This method should be called after start_playing() has been called. The audio data will be played at the sample rate returned by get_output_audio_samplerate().

set_max_output_buffers

< source >

( max_buffers: int )

Parameters

max_buffers (int) — Maximum number of buffers to queue.

Set the maximum number of output buffers to queue in the player.

start_playing

< source >

( )

Raises

RuntimeError

RuntimeError — If audio playback cannot be started due to hardware or configuration issues.

Start playing audio.

This method should initialize the audio playback system and prepare it to receive audio data via push_audio_sample().

Note: Implementations should handle any necessary resource allocation and error checking. If playback cannot be started, implementations should log appropriate error messages.

start_recording

< source >

( )

Raises

RuntimeError

RuntimeError — If audio recording cannot be started due to hardware or configuration issues.

Start recording audio.

This method should initialize the audio recording system and prepare it to capture audio data. After calling this method, get_audio_sample() should be able to retrieve recorded audio data.

Note: Implementations should handle any necessary resource allocation and error checking. If recording cannot be started, implementations should log appropriate error messages.

stop_playing

< source >

( )

Stop playing audio and release resources.

This method should stop any ongoing audio playback and release all associated resources. After calling this method, push_audio_sample() calls will have no effect until start_playing() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

stop_recording

< source >

( )

Close the audio device and release resources.

This method should stop any ongoing audio recording and release all associated resources. After calling this method, get_audio_sample() should return None until start_recording() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

class reachy_mini.media.audio_gstreamer.GStreamerAudio

< source >

( log_level: str = 'INFO' )

Audio implementation using GStreamer.

clear_player

< source >

( )

Flush the player’s appsrc to drop any queued audio immediately.

get_audio_sample

< source >

( ) → Optional[npt.NDArray[np.float32]]

Returns

Optional[npt.NDArray[np.float32]]

The captured sample in raw format, or None if error.

Read a sample from the audio card. Returns the sample or None if error.

See AudioBase.get_audio_sample() for complete documentation.

get_input_audio_samplerate

< source >

( )

Get the input samplerate of the audio device.

See AudioBase.get_input_audio_samplerate() for complete documentation.

get_input_channels

< source >

( )

Get the number of input channels of the audio device.

See AudioBase.get_input_channels() for complete documentation.

get_output_audio_samplerate

< source >

( )

Get the output samplerate of the audio device.

See AudioBase.get_output_audio_samplerate() for complete documentation.

get_output_channels

< source >

( )

Get the number of output channels of the audio device.

See AudioBase.get_output_channels() for complete documentation.

play_sound

< source >

( sound_file: str )

Parameters

sound_file (str) — Path to the sound file to play.

Play a sound file.

See AudioBase.play_sound() for complete documentation.

Todo: for now this function is mean to be used on the wireless version.

push_audio_sample

< source >

( data: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float32]] )

Push audio data to the output device.

See AudioBase.push_audio_sample() for complete documentation.

start_playing

< source >

( )

Open the audio output using GStreamer.

See AudioBase.start_playing() for complete documentation.

start_recording

< source >

( )

Open the audio card using GStreamer.

See AudioBase.start_recording() for complete documentation.

stop_playing

< source >

( )

Stop playing audio and release resources.

See AudioBase.stop_playing() for complete documentation.

stop_recording

< source >

( )

Release the camera resource.

See AudioBase.stop_recording() for complete documentation.

reachy_mini.media.audio_utils.get_respeaker_card_number

< source >

( ) → int

Returns

int

The card number of the detected ReSpeaker/Reachy Mini Audio device. Returns 0 if no specific device is found (uses default sound card), or -1 if there’s an error running the detection command.

Return the card number of the ReSpeaker sound card, or 0 if not found.

Note: This function runs ‘arecord -l’ to list available audio capture devices and processes the output to find Reachy Mini Audio or ReSpeaker devices. It’s primarily used on Linux systems with ALSA audio configuration.

The function returns:

Positive integer: Card number of detected Reachy Mini Audio device
0: No Reachy Mini Audio device found, using default sound card
-1: Error occurred while trying to detect audio devices

Example:

card_num = get_respeaker_card_number()
if card_num > 0:
    print(f"Using Reachy Mini Audio card {card_num}")
elif card_num == 0:
    print("Using default sound card")
else:
    print("Error detecting audio devices")

reachy_mini.media.audio_utils.has_reachymini_asoundrc

< source >

( ) → bool

Returns

bool

True if ~/.asoundrc exists and contains the required Reachy Mini audio configuration entries, False otherwise.

Check if ~/.asoundrc exists and contains both reachymini_audio_sink and reachymini_audio_src.

Note: This function checks for the presence of the ALSA configuration file ~/.asoundrc and verifies that it contains the necessary configuration entries for Reachy Mini audio devices (reachymini_audio_sink and reachymini_audio_src). These entries are required for proper audio routing and device management.

Example:

if has_reachymini_asoundrc():
    print("Reachy Mini audio configuration is properly set up")
else:
    print("Need to configure Reachy Mini audio devices")
    write_asoundrc_to_home()  # Create the configuration

reachy_mini.media.audio_utils.check_reachymini_asoundrc

< source >

( )

Check if ~/.asoundrc exists and is correctly configured for Reachy Mini Audio.

reachy_mini.media.audio_utils.write_asoundrc_to_home

< source >

( )

Write the .asoundrc file with Reachy Mini audio configuration to the user’s home directory.

This function creates an ALSA configuration file (.asoundrc) in the user’s home directory that configures the ReSpeaker sound card for proper audio routing and multi-client support. The configuration enables simultaneous audio input and output access, which is essential for the Reachy Mini Wireless version’s audio functionality.

The generated configuration includes:

Default audio device settings pointing to the ReSpeaker sound card
dmix plugin for multi-client audio output (reachymini_audio_sink)
dsnoop plugin for multi-client audio input (reachymini_audio_src)
Proper buffer and sample rate settings for optimal performance

Note: This function automatically detects the ReSpeaker card number and creates a configuration tailored to the detected hardware. It is primarily used for the Reachy Mini Wireless version.

The configuration file will be created at ~/.asoundrc and will overwrite any existing file with the same name. Existing audio configurations should be backed up before calling this function.

class reachy_mini.media.audio_control_utils.ReSpeaker

< source >

( dev: Device )

Class to interface with the ReSpeaker XVF3800 USB device.

close

< source >

( )

Close the interface.

read

< source >

( name: str )

Read data from a specified parameter on the ReSpeaker device.

write

< source >

( name: str data_list: Any )

Write data to a specified parameter on the ReSpeaker device.

reachy_mini.media.audio_control_utils.find

< source >

( vid: int = 10374 pid: int = 26 ) → ReSpeaker | None

Parameters

vid (int) — USB Vendor ID to search for. Default: 0x2886 (XMOS).
pid (int) — USB Product ID to search for. Default: 0x001A (XMOS XVF3800).

Returns

ReSpeaker | None

A ReSpeaker object if the device is found, None otherwise.

Find and return the ReSpeaker USB device with the given Vendor ID and Product ID.

Note: This function searches for USB devices with the specified Vendor ID and Product ID using libusb backend. The default values target XMOS XVF3800 devices used in ReSpeaker microphone arrays.

Example:

from reachy_mini.media.audio_control_utils import find

# Find default ReSpeaker device
respeaker = find()
if respeaker is not None:
    print("Found ReSpeaker device")
    respeaker.close()

# Find specific device
custom_device = find(vid=0x1234, pid=0x5678)

reachy_mini.media.audio_control_utils.init_respeaker_usb

< source >

( ) → Optional[ReSpeaker]

Returns

Optional[ReSpeaker]

A ReSpeaker object if a compatible device is found, None otherwise.

Initialize the ReSpeaker USB device. Looks for both new and beta device IDs.

Note: This function attempts to initialize a ReSpeaker microphone array by searching for USB devices with known Vendor and Product IDs. It tries:

New Reachy Mini Audio firmware (0x38FB:0x1001) - preferred
Old ReSpeaker firmware (0x2886:0x001A) - with warning to update

The function handles USB backend errors gracefully and returns None if no compatible device is found or if initialization fails.

Example:

from reachy_mini.media.audio_control_utils import init_respeaker_usb

# Initialize ReSpeaker device
respeaker = init_respeaker_usb()
if respeaker is not None:
    print("ReSpeaker initialized successfully")
    # Use the device...
    doa = respeaker.read("DOA_VALUE_RADIANS")
    respeaker.close()
else:
    print("No ReSpeaker device found")

class reachy_mini.media.camera_base.CameraBase

< source >

( log_level: str = 'INFO' )

Parameters

logger (logging.Logger) — Logger instance for camera-related messages.
_resolution (Optional[CameraResolution]) — Current camera resolution setting.
camera_specs (Optional[CameraSpecs]) — Camera specifications including supported resolutions and calibration parameters.
resized_K (Optional[npt.NDArray[np.float64]]) — Camera intrinsic matrix resized to match the current resolution.

Abstract class for opening and managing a camera.

This class defines the interface that all camera implementations must follow. It provides common camera parameters and methods for managing camera devices, including image capture, resolution management, and camera calibration.

close

< source >

( )

Close the camera and release resources.

This method should stop any ongoing image capture and release all associated resources. After calling this method, read() should return None until open() is called again.

Note: Implementations should ensure proper cleanup to prevent resource leaks.

open

< source >

( )

Raises

RuntimeError

RuntimeError — If the camera cannot be opened due to hardware or configuration issues.

Open the camera.

This method should initialize the camera device and prepare it for capturing images. After calling this method, read() should be able to retrieve camera frames.

Note: Implementations should handle any necessary resource allocation, camera configuration, and error checking. If the camera cannot be opened, implementations should log appropriate error messages.

read

< source >

( ) → Optional[npt.NDArray[np.uint8]]

Returns

Optional[npt.NDArray[np.uint8]]

A numpy array containing the captured image in BGR format (OpenCV convention), or None if no image is available or an error occurred.

The array shape is (height, width, 3) where the last dimension represents the BGR color channels.

Read an image from the camera. Returns the image or None if error.

Note: This method should be called after open() has been called. The image resolution can be obtained via the resolution property.

Example:

camera.open()
frame = camera.read()
if frame is not None:
    cv2.imshow("Camera Frame", frame)
    cv2.waitKey(1)

set_resolution

< source >

( resolution: CameraResolution )

Parameters

resolution (CameraResolution) — The desired camera resolution from the CameraResolution enum.

Raises

RuntimeError or ValueError

RuntimeError — If camera specs are not set or if trying to change resolution of a Mujoco simulated camera.
ValueError — If the requested resolution is not supported by the camera.

Set the camera resolution.

Note: This method updates the camera’s resolution and automatically rescales the camera intrinsic matrix (K) to match the new resolution. The rescaling preserves the camera’s field of view and principal point position relative to the image dimensions.

Example:

from reachy_mini.media.camera_constants import CameraResolution
camera.set_resolution(CameraResolution.R1280x720at30fps)

class reachy_mini.media.camera_opencv.OpenCVCamera

< source >

( log_level: str = 'INFO' )

Parameters

Inherits all attributes from CameraBase. —
Additionally manages OpenCV VideoCapture objects and camera connections. —

Camera implementation using OpenCV.

This class implements the CameraBase interface using OpenCV, providing cross-platform camera support for Reachy Mini robots. It automatically detects and configures supported camera models.

close

< source >

( )

Release the camera resource.

See CameraBase.close() for complete documentation.

open

< source >

( udp_camera: typing.Optional[str] = None )

Open the camera using OpenCV VideoCapture.

See CameraBase.open() for complete documentation.

read

< source >

( )

Raises

RuntimeError

RuntimeError — If the camera is not opened.

Read a frame from the camera.

See CameraBase.read() for complete documentation.

set_resolution

< source >

( resolution: CameraResolution )

Set the camera resolution.

class reachy_mini.media.camera_gstreamer.GStreamerCamera

< source >

( log_level: str = 'INFO' use_sim: bool = False )

Camera implementation using GStreamer.

close

< source >

( )

Release the camera resource.

get_video_device

< source >

( )

Use Gst.DeviceMonitor to find the unix camera path /dev/videoX.

Returns the device path (e.g., ‘/dev/video2’), or ” if not found.

open

< source >

( )

Open the camera using GStreamer.

read

< source >

( ) → Optional[npt.NDArray[np.uint8]]

Returns

Optional[npt.NDArray[np.uint8]]

The captured BGR frame as a NumPy array, or None if error.

Read a frame from the camera. Returns the frame or None if error.

set_resolution

< source >

( resolution: CameraResolution )

Set the camera resolution.

reachy_mini.media.camera_utils.find_camera

< source >

( apiPreference: int = 0 no_cap: bool = False ) → Tuple[Optional[cv2.VideoCapture], Optional[CameraSpecs]]

Parameters

apiPreference (int) — Preferred API backend for the camera. Default is cv2.CAP_ANY. Options include cv2.CAP_V4L2 (Linux), cv2.CAP_DSHOW (Windows), cv2.CAP_MSMF (Windows), etc.
no_cap (bool) — If True, close the camera after finding it. Useful for testing camera detection without keeping the camera open. Default is False.

Returns

Tuple[Optional[cv2.VideoCapture], Optional[CameraSpecs]]

A tuple containing:

cv2.VideoCapture: A VideoCapture object if the camera is found and opened successfully, otherwise None.
CameraSpecs: The camera specifications for the detected camera, or None if no camera was found.

Find and return the Reachy Mini camera.

Looks for the Reachy Mini camera first, then Arducam, then older Raspberry Pi Camera. Returns None if no camera is found. Falls back to generic webcam if no specific camera is detected.

Note: This function tries to detect cameras in the following order:

Reachy Mini Lite Camera (preferred)
Older Raspberry Pi Camera
Arducam
Generic Webcam (fallback)

The function automatically sets the appropriate video codec (MJPG) for Reachy Mini and Raspberry Pi cameras to ensure compatibility.

Example:

cap, specs = find_camera()
if cap is not None:
    print(f"Found {specs.name} camera")
    # Set resolution
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
    # Capture a frame
    ret, frame = cap.read()
    cap.release()
else:
    print("No camera found")

reachy_mini.media.camera_utils.find_camera_by_vid_pid

< source >

( vid: int = 14587 pid: int = 4098 apiPreference: int = 0 ) → cv2.VideoCapture | None

Parameters

vid (int) — Vendor ID of the camera. Default is ReachyMiniLiteCamSpecs.vid (0x38FB).
pid (int) — Product ID of the camera. Default is ReachyMiniLiteCamSpecs.pid (0x1002).
apiPreference (int) — Preferred API backend for the camera. Default is cv2.CAP_ANY. On Linux, this automatically uses cv2.CAP_V4L2 for better compatibility.

Returns

cv2.VideoCapture | None

A VideoCapture object if the camera with matching VID/PID is found and opened successfully, otherwise None.

Find and return a camera with the specified VID and PID.

Note: This function uses the cv2_enumerate_cameras package to enumerate available cameras and find one with the specified USB Vendor ID and Product ID. This is useful for selecting specific camera models when multiple cameras are connected to the system.

The Arducam camera creates two /dev/videoX devices that enumerate_cameras cannot differentiate, so this function tries to open each potential device until it finds a working one.

Example:

# Find Reachy Mini Lite Camera by its default VID/PID
cap = find_camera_by_vid_pid()
if cap is not None:
    print("Found Reachy Mini Lite Camera")
    cap.release()

# Find a specific camera by custom VID/PID
cap = find_camera_by_vid_pid(vid=0x0C45, pid=0x636D)  # Arducam
if cap is not None:
    print("Found Arducam")

... cap.release()

class reachy_mini.media.camera_constants.CameraResolution

< source >

( *values )

Parameters

R1536x864at40fps — 1536x864 resolution at 40 fps
R1280x720at60fps — 1280x720 resolution at 60 fps (HD)
R1280x720at30fps — 1280x720 resolution at 30 fps (HD)
R1920x1080at30fps — 1920x1080 resolution at 30 fps (Full HD)
R1920x1080at60fps — 1920x1080 resolution at 60 fps (Full HD)
R2304x1296at30fps — 2304x1296 resolution at 30 fps
R1600x1200at30fps — 1600x1200 resolution at 30 fps
R3264x2448at30fps — 3264x2448 resolution at 30 fps
R3264x2448at10fps — 3264x2448 resolution at 10 fps
R3840x2592at30fps — 3840x2592 resolution at 30 fps
R3840x2592at10fps — 3840x2592 resolution at 10 fps
R3840x2160at30fps — 3840x2160 resolution at 30 fps (4K UHD)
R3840x2160at10fps — 3840x2160 resolution at 10 fps (4K UHD)
R3072x1728at10fps — 3072x1728 resolution at 10 fps
R4608x2592at10fps — 4608x2592 resolution at 10 fps

Base class for camera resolutions.

Enumeration of standardized camera resolutions and frame rates supported by Reachy Mini cameras. Each enum value contains a tuple of (width, height, fps).

Note: The enum values are tuples containing (width, height, frames_per_second). Not all resolutions are supported by all camera models - check the specific camera specifications for available resolutions.

Example:

from reachy_mini.media.camera_constants import CameraResolution

# Get resolution information
res = CameraResolution.R1280x720at30fps
width, height, fps = res.value
print(f"Resolution: {width}x{height}@{fps}fps")

# Check if a resolution is supported by a camera
from reachy_mini.media.camera_constants import ReachyMiniLiteCamSpecs
res = CameraResolution.R1920x1080at60fps
if res in ReachyMiniLiteCamSpecs.available_resolutions:
    print("This resolution is supported")

class reachy_mini.media.camera_constants.CameraSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Parameters

name (str) — Human-readable name of the camera model.
available_resolutions (List[CameraResolution]) — List of supported resolutions and frame rates for this camera model.
default_resolution (CameraResolution) — Default resolution used when the camera is initialized.
vid (int) — USB Vendor ID for identifying this camera model.
pid (int) — USB Product ID for identifying this camera model.
K (npt.NDArray[np.float64]) — 3x3 camera intrinsic matrix containing focal lengths and principal point coordinates.
D (npt.NDArray[np.float64]) — 5-element array containing distortion coefficients (k1, k2, p1, p2, k3) for radial and tangential distortion.

Base camera specifications.

Dataclass containing specifications for a camera model, including supported resolutions, calibration parameters, and USB identification information.

Note: The intrinsic matrix K has the format: [[fx, 0, cx], [ 0, fy, cy], [ 0, 0, 1]]

Where fx, fy are focal lengths in pixels, and cx, cy are the principal point coordinates (typically near the image center).

Example:

from reachy_mini.media.camera_constants import CameraSpecs

# Create a custom camera specification
custom_specs = CameraSpecs(
    name="custom_camera",
    available_resolutions=[CameraResolution.R1280x720at30fps],
    default_resolution=CameraResolution.R1280x720at30fps,
    vid=0x1234,
    pid=0x5678,
    K=np.array([[800, 0, 640], [0, 800, 360], [0, 0, 1]]),
    D=np.zeros(5)
)

class reachy_mini.media.camera_constants.ArducamSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Arducam camera specifications.

class reachy_mini.media.camera_constants.ReachyMiniLiteCamSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Reachy Mini Lite camera specifications.

class reachy_mini.media.camera_constants.ReachyMiniWirelessCamSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Reachy Mini Wireless camera specifications.

class reachy_mini.media.camera_constants.OlderRPiCamSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Older Raspberry Pi camera specifications. Keeping for compatibility.

class reachy_mini.media.camera_constants.MujocoCameraSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Mujoco simulated camera specifications.

class reachy_mini.media.camera_constants.GenericWebcamSpecs

< source >

( name: str = '' available_resolutions: typing.List[reachy_mini.media.camera_constants.CameraResolution] = <factory> default_resolution: CameraResolution = <CameraResolution.R1280x720at30fps: (1280, 720, 30, 1.0)> K: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> D: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float64]] = <factory> )

Generic webcam specifications (fallback for any webcam).

class reachy_mini.media.webrtc_client_gstreamer.GstWebRTCClient

< source >

( log_level: str = 'INFO' peer_id: str = '' signaling_host: str = '' signaling_port: int = 8443 )

Parameters

Inherits all attributes from CameraBase and AudioBase. —
Additionally manages GStreamer pipelines for WebRTC communication. —

GStreamer WebRTC client implementation.

This class implements a WebRTC client using GStreamer that can connect to a WebRTC server (such as the one hosted on Reachy Mini Wireless) to stream audio and video in real-time. It implements both CameraBase and AudioBase interfaces, allowing seamless integration with the media system.

close

< source >

( )

Stop the pipeline.

See CameraBase.close() for complete documentation.

get_audio_sample

< source >

( ) → Optional[npt.NDArray[np.float32]]

Returns

Optional[npt.NDArray[np.float32]]

The captured sample in raw format, or None if error.

Read a sample from the audio card. Returns the sample or None if error.

See AudioBase.get_audio_sample() for complete documentation.

open

< source >

( )

Open the video stream.

See CameraBase.open() for complete documentation.

play_sound

< source >

( sound_file: str )

Parameters

sound_file (str) — Path to the sound file to play.

Play a sound file.

See AudioBase.play_sound() for complete documentation.

push_audio_sample

< source >

( data: numpy.ndarray[tuple[typing.Any, ...], numpy.dtype[numpy.float32]] )

Push audio data to the output device via the bidirectional WebRTC connection.

read

< source >

( ) → Optional[npt.NDArray[np.uint8]]

Returns

Optional[npt.NDArray[np.uint8]]

The captured frame in BGR format, or None if error.

Read a frame from the camera. Returns the frame or None if error.

See CameraBase.read() for complete documentation.

set_resolution

< source >

( resolution: CameraResolution )

Set the camera resolution.

start_playing

< source >

( )

Open the audio output using GStreamer.

See AudioBase.start_playing() for complete documentation.

start_recording

< source >

( )

Open the audio card using GStreamer.

See AudioBase.start_recording() for complete documentation.

stop_playing

< source >

( )

Stop playing audio and release resources.

See AudioBase.stop_playing() for complete documentation.

stop_recording

< source >

( )

Release the camera resource.

See AudioBase.stop_recording() for complete documentation.

class reachy_mini.media.webrtc_daemon.GstWebRTC

< source >

( log_level: str = 'INFO' )

Parameters

_logger (logging.Logger) — Logger instance for WebRTC daemon operations.
_loop (GLib.MainLoop) — GLib main loop for handling GStreamer events.
camera_specs (CameraSpecs) — Specifications of the detected camera.
_resolution (CameraResolution) — Current streaming resolution.
resized_K (npt.NDArray[np.float64]) — Camera intrinsic matrix for current resolution.

WebRTC pipeline using GStreamer.

This class implements a WebRTC server using GStreamer that streams video and audio from the Reachy Mini robot to connected WebRTC clients. It’s designed to run as a daemon process and handle the complete WebRTC signaling and media streaming pipeline.

pause

< source >

( )

Pause the WebRTC pipeline.

send_data_message

< source >

( peer_id: typing.Optional[str] message: str )

Parameters

message — The string message to send
peer_id — If specified, send only to this peer. Otherwise broadcast to all.

Send a message to connected peers via data channel.

set_message_handler

< source >

( handler: typing.Callable[[str, str], NoneType] )

Parameters

handler — Callback function that receives (peer_id, message)

Set a callback for incoming data channel messages.

start

< source >

( )

Start the WebRTC pipeline.

stop

< source >

( )

Stop the WebRTC pipeline.

Reachy Mini

Media

Media Manager

class reachy_mini.media.media_manager.MediaManager

close

get_DoA

get_audio_sample

get_frame

get_input_audio_samplerate

get_input_channels

get_output_audio_samplerate

get_output_channels

play_sound

push_audio_sample

start_playing

start_recording

stop_playing

stop_recording

Audio

class reachy_mini.media.audio_base.AudioBase

cleanup

clear_output_buffer

get_DoA

get_audio_sample

get_input_audio_samplerate

get_input_channels

get_output_audio_samplerate

get_output_channels

play_sound

push_audio_sample

set_max_output_buffers

start_playing

start_recording

stop_playing

stop_recording

class reachy_mini.media.audio_gstreamer.GStreamerAudio

clear_player

get_audio_sample

get_input_audio_samplerate

get_input_channels

get_output_audio_samplerate

get_output_channels

play_sound

push_audio_sample

start_playing

start_recording

stop_playing

stop_recording

Audio Utils Functions

reachy_mini.media.audio_utils.get_respeaker_card_number

reachy_mini.media.audio_utils.has_reachymini_asoundrc

reachy_mini.media.audio_utils.check_reachymini_asoundrc

reachy_mini.media.audio_utils.write_asoundrc_to_home

Audio Control Utils Functions

class reachy_mini.media.audio_control_utils.ReSpeaker

close

read

write

reachy_mini.media.audio_control_utils.find

reachy_mini.media.audio_control_utils.init_respeaker_usb

Camera

class reachy_mini.media.camera_base.CameraBase

close

open

read

set_resolution

class reachy_mini.media.camera_opencv.OpenCVCamera

close

open

read

set_resolution

class reachy_mini.media.camera_gstreamer.GStreamerCamera

close

get_video_device

open

read

set_resolution

Camera Utils Functions

reachy_mini.media.camera_utils.find_camera

reachy_mini.media.camera_utils.find_camera_by_vid_pid