File size: 2,784 Bytes
a144c29 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | ---
license: apache-2.0
tags:
- coreml
- forced-alignment
- wav2vec2
- ctc
- apple-neural-engine
library_name: coremlit
---
# alignkit-coreml — fp16-safe CoreML conversion of wav2vec2-base-960h for forced alignment
A CoreML conversion of **facebook/wav2vec2-base-960h** (CTC head, 29-class character
vocabulary) with an **fp16-survivable log-probability tail**, for word-level forced
alignment. Consumed by the `alignkit` crate of the
[coremlit](https://github.com/findit-studio/coremlit) workspace.
## Why this conversion exists
The stock conversion's tail is a decomposed `softmax → log(eps = 2^-149)`. That epsilon is
below fp16's smallest subnormal (`2^-24`), so on the Apple Neural Engine the guard rounds
to zero and `log(0)` saturates to **≈ −45440**, corrupting **16.7% of emission cells** and
shifting word timings by up to **881 ms** — silently, since the values are still finite and
negative. (A naive `torch.log_softmax` re-export reproduces the defect under coremltools
9.0.) This conversion uses an explicitly **fused `x − logsumexp(x)`** tail: the emissions
are exact log-probabilities on CPU and bounded (min > −100) on every placement.
## Contents
- `base960h_aligner.{mlpackage,mlmodelc}` — encoder weights **byte-identical** to the
established conversion (`weight.bin` sha256 `de51193f…`); only the tail graph changed.
- `CHECKSUMS.sha256` — per-file digests.
Input: 16 kHz mono f32, up to a 960,000-sample (60 s) window. Output: `[1, T, 29]`
log-probabilities, 320-sample hop.
## Placement guidance (measured)
- **CPU: exact** log-probabilities (per-frame `logsumexp ≈ 0`) and the fastest placement
at this model size — the recommended default.
- **GPU / ANE: numerically safe with this conversion** (emissions bounded, no saturation),
but the wav2vec2 encoder itself is fp16-sensitive (group-norm): expect ~98% frame-argmax
agreement vs fp32, i.e. small word-timing differences. Consumers that require exact
timings should stay on CPU; the crate's runtime guard rejects any corrupted emission
matrix regardless of placement.
## Verification (summary)
MIL-level fp16-guard audit clean (27 guard sites, every effective floor ≥ `2^-24`);
emissions min > −100 on ANE (−23.81) and GPU (−28.36) with real speech; word-timing parity
vs an independent ONNX implementation: median **0.0 ms** on unpadded 60 s audio (367/372
boundaries within one 20 ms frame). Toolchain pinned: coremltools 9.0, torch/torchaudio
2.5.1, python 3.11.15.
## Upstream provenance & licensing
Weights: [facebook/wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h)
(Apache-2.0), via torchaudio's `WAV2VEC2_ASR_BASE_960H` pipeline. This is a derivative
conversion: identical weights, corrected graph numerics.
|