# BEST-RQ-2 (xares-llm encoder)

This folder contains the BEST-RQ-2 audio encoder integration for `xares-llm`.

Benchmark repository: [xiaomi-research/xares-llm](https://github.com/xiaomi-research/xares-llm)

## Setup

1. Make sure the environment to run `xares-llm` is set up properly (virtual environment initialized and the `xares-llm` package downloaded/installed).

2. Add the `BEST-RQ-2` folder to the `xares-llm` (current) directory so it is available at `./BEST-RQ-2`.

3. Before running a `xares-llm` evaluation, you must install the required packages for BEST-RQ-2:

```bash
uv pip install -r BEST-RQ-2/audio-embeddings/pyproject.toml
```

## Run an evaluation

Single task (e.g. to test that everything works):

```bash
uv run -m xares_llm.run BEST-RQ-2.BEST-RQ-2_encoder.BestRQ2Encoder <task> <task> # e.g. replace <task> with esc-50, score should be around 0.48
```

All tasks:

```bash
uv run -m xares_llm.run BEST-RQ-2.BEST-RQ-2_encoder.BestRQ2Encoder all
```

## Help

If you encounter any problems, contact: [ludovic.tuncay@irit.fr](mailto:ludovic.tuncay@irit.fr)