README.md

---
tags:
- automatic-speech-recognition
- ios
- swift
- coreml
- onnxruntime
- apple-neural-engine
- on-device
language:
- en
- zh
- fr
- ja
- yue
library_name: coreml
license: apache-2.0
repository: https://github.com/AutoArk/open-audio-opd
---

<div align="center">

# Audio8-ASR-0.1B-iOS-ANE

[![GitHub](https://img.shields.io/badge/GitHub-AutoArk%2Fopen--audio--opd-blue?logo=github)](https://github.com/AutoArk/open-audio-opd)
[![arXiv](https://img.shields.io/badge/arXiv-2605.28139-b31b1b?logo=arxiv)](https://arxiv.org/abs/2605.28139)
[![License](https://img.shields.io/badge/License-Apache--2.0-green)](https://www.apache.org/licenses/LICENSE-2.0)

</div>

This repository packages an iPhone-ready, ASR-only build of
`Audio8-ASR-0.1B`. It includes a Swift SDK, a minimal iOS demo app, an optional
ANE benchmark app, and a prebuilt model asset bundle.

The ASR model is multilingual, with support for languages including English,
Chinese, Cantonese, French, and Japanese.

The on-device pipeline uses Core ML on Apple Neural Engine for the audio tower
and ONNX Runtime for the int4 language-model decoder. Audio is transcribed
locally; no network request is required by the SDK or demo app.

## Contents

| Path | Description |
| --- | --- |
| `SpeechKit/` | Swift Package exposing `SpeechKit`, `ASRKit`, and `SpeechCore` |
| `dist/ASRModels.bundle` | Prebuilt model assets: Core ML audio tower, ONNX decoder, tokenizer tables, and integrity manifest |
| `ASRDemo/` | Minimal iOS app for microphone recording and one-shot transcription |
| `ANEBench/` | Optional iOS app for Core ML / ANE latency and sustained-load checks |
| `assets/` | Screenshots and model-card media |
| `config.json` | Machine-readable package metadata and Hugging Face download-stat query file |
| `GETTING_STARTED.md` | Reproducible setup, build, signing, and device-testing guide |
| `LICENSE` | Apache License 2.0 |

## Related Repositories

- [Audio8-ASR-0.1B](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B): base model checkpoint.
- [Audio8-ASR-0.1B-onnx-runtime](https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-onnx-runtime): ONNX Runtime package.

## Packaged Model Variant and Footprint

This release packages the iPhone ANE-oriented variant of `Audio8-ASR-0.1B`:

- Audio tower/head: compiled Core ML `mlmodelc` with mixed Float16/Int8
  storage, Float16 compute/output tensors, and ANE execution.
- Decoder: ONNX Runtime CPU decoder with int4 shared language-model weights
  (`lm_shared_int4.data`) and int4 prefill/decode ONNX graphs.
- Token embedding table: Float16 (`token_embedding_fp16.bin`).

On a physical iPhone with the Core ML audio tower running on ANE, the demo is
designed to keep runtime memory footprint around 200 MB. The example below
shows a 183 MB app footprint during a microphone transcription run, with sampled
peak footprint varying by device, iOS version, cold/warm start state, and
measurement window. We position this package as one of the smallest usable ASR
model stacks for on-device iPhone transcription.

<p align="center">
  <img src="assets/iphone-asr-demo-footprint.png" alt="Audio8 ASR iPhone demo memory footprint" width="360">
</p>

## Quick Start

```bash
brew install xcodegen

cd SpeechKit
swift package resolve
swift build
swift run dev-check
cd ..

cd ASRDemo
xcodegen generate
open ASRDemo.xcodeproj
```

In Xcode, select the `ASRDemo` target, choose your signing team, change the
bundle identifier to a unique value, then run on an iPhone or iOS Simulator.

The demo uses microphone input. If you want to test a local file without
changing the app, use `asrkit-cli --file /path/to/audio.wav`.

## What Runs on ANE

The key acceleration path is the audio tower:

```text
audio -> log-mel -> Core ML audio tower on ANE -> projected audio embeddings
      -> ONNX Runtime int4 decoder on CPU -> transcript
```

The decoder intentionally stays on CPU. Its per-token workload is small enough
that ANE dispatch overhead is not beneficial for this build.

## Requirements

- macOS on Apple Silicon is recommended.
- Full Xcode, not only Command Line Tools.
- iOS 18+ / macOS 15+ for the Swift package.
- XcodeGen for regenerating the demo projects.
- An Apple Developer account for physical-device signing. A free Personal Team
  is enough for local device testing.

## Validation

```bash
cd SpeechKit
swift run dev-check
swift test

# Transcribe a local file with the bundled model assets:
swift run -c release asrkit-cli .. --file /path/to/audio.wav

# Repeat one local file to watch stability and footprint:
swift run -c release asrkit-cli .. --file /path/to/audio.wav --repeat 10
```

`dev-check` is the fastest smoke test and does not require model inference.
`asrkit-cli --file` loads `dist/ASRModels.bundle` and runs end-to-end
transcription on macOS.

Hugging Face counts model downloads through query files such as `config.json`.
If you automate downloads with `snapshot_download` or `hf_hub_download`, include
the root `config.json` in the request path so repository downloads are counted.

For memory footprint and thermal checks, run `ASRDemo` on a physical iPhone,
record one utterance, then tap `Repeat Last 10x` while watching the in-app
`System` panel. Simulator memory is useful for trends only; it is not
equivalent to iPhone memory pressure or Jetsam behavior.

## Repository Status

This staging copy is intended for review before publishing to
`https://huggingface.co/AutoArk-AI/Audio8-ASR-0.1B-iOS-ANE`.

The repository is distributed under Apache License 2.0. Confirm separately
that the model weights and tokenizer assets are intended to be released under
the same license before publishing.