Confucius4-R2T2 MLX 8bit — converted with mlx_audio.convert (qwen3-asr-mlx-swift V1)

2d6d997 verified 6 days ago

2.81 kB

license: other
license_name: netease-youdao-model-use-license-agreement
license_link: https://github.com/netease-youdao/Confucius4-R2T2/blob/master/MODEL_LICENSE
base_model: netease-youdao/Confucius4-R2T2
base_model_relation: quantized
pipeline_tag: automatic-speech-recognition
library_name: mlx
tags:
  - mlx
  - mlx-audio
  - qwen3_asr
  - asr
  - streaming
  - confucius4
  - r2t2
language:
  - zh
  - en

Confucius4-R2T2 — MLX, 8-bit

netease-youdao/Confucius4-R2T2 — NetEase Youdao's low-latency, append-only streaming ASR fine-tune of Qwen3-ASR-1.7B — converted to MLX: the Qwen3 language model quantised to 8 bits, group size 64, affine; the AuT audio encoder stays bf16 (2.3 GB).

Converted with the mlx_audio.convert CLI (xocialize/mlx-audio fork @ 1792021, mlx 0.32.2), on the GPU stream of an Apple M5 Max — MLX quantisation is device-dependent, so the device is recorded:

python -m mlx_audio.convert --hf-path netease-youdao/Confucius4-R2T2 \
    --mlx-path Confucius4-R2T2-8bit --model-domain stt -q --q-bits 8 --q-group-size 64

The architecture is qwen3_asr, unchanged from Qwen/Qwen3-ASR-1.7B, so the weights load through the existing Qwen3-ASR paths of mlx-audio (Python) and mlx-audio-swift, and through qwen3-asr-mlx-swift, whose R2T2 loop and MLXEngine stt package (mlx-r2t2-stt-swift) are what this repo exists for.

Use

from mlx_audio.stt.utils import load_model, load_audio
model = load_model("mlx-community/Confucius4-R2T2-8bit")
print(model.generate(load_audio("audio.wav"), language="English").text)

Streaming (the stable-prefix protocol the model was trained for) needs the R2T2 loop: 160 ms chunks, re-feed the audio window with the committed text as the prompt, cut the output at |, roll back one token, commit the rest. See the Swift package above or netease-youdao/Confucius4-R2T2.

Licence

The weights are governed by the NetEase Youdao Model Use License Agreement (LICENSE; the Chinese text in MODEL_LICENSE_zh prevails). It is royalty-free, including commercial use, but requires a separate licence from NetEase Youdao above 100 million monthly active users or RMB 1 billion annual revenue (§2.2), forbids using the model to improve other commercial AI models (§3.4 c) and the high-risk uses it lists (§4.2), and binds downstream recipients to the same terms (§3.4 a). Read it before use.

Any modifications made to the original model in this Derivative Work are not endorsed, warranted, or guaranteed by the original right-holder of the original model, and the original right-holder disclaims all liability related to this Derivative Work.