mlboydaisuke/Nemotron-3-Diarization-CoreAI

Core AI is Apple's on-device ML runtime in iOS 27 / macOS 27 and the successor to Core ML: PyTorch models are exported with Apple's coreai-torch (LLMs: coreai.llm.export) into .aimodel bundles that run on the GPU or the Neural Engine.

This model has no row on DeviceMark, the on-device LLM leaderboard.

Nemotron-3-Diarization — Core AI

nvidia/Nemotron-3-Diarization (OpenMDW-1.1, 99.2M parameters) converted to Apple Core AI. It is streaming speaker diarization, "who spoke when", for up to 8 speakers at 10 ms resolution, on-device. The graph is the per-chunk network. The host keeps the state: the log-mel front end, the speaker cache and its FIFO.

It succeeds the 4-speaker Streaming-Sortformer-Diar-CoreAI. Another Core AI conversion of this checkpoint exists: smdesai/Nemotron-3-Diarization-CoreAI.

Files

file	what
n3d_streaming_float16.aimodel	the graph, fp16 weights, T = 541 packed rows; all three streaming modes; macOS (198.0 MB)
n3d_offline_float16.aimodel	the same graph at T = 684, for the offline profile; macOS (198.0 MB)
n3d_streaming_float16.h18p.aimodelc	the streaming graph compiled ahead of time for the iPhone 17 Pro GPU (h18p, 198.2 MB)
n3d_offline_float16.h18p.aimodelc	the offline graph, same target (198.3 MB)
embedder_projection.f32le	[512, 1024]: the projection of 8 stacked mel frames (a host matmul)
silence_embeds.f32le	[512]: the learned silence row of the speaker cache
mel_filters_128x257.f32le	[128, 257]: the slaney mel filterbank, bit-identical to the transformers feature extractor's
hann_window_400.f32le	[400]: torch.hann_window(400, periodic=False), bit for bit
metadata.json	the graph contract, the host constants of each profile, the mel parameters, and for every asset and bundle its sha256 (bundles also: size, target, compute preference)
host/	the four .f32le files and metadata.json again, in one directory: what a client that fetches a directory (CoreAIKit's catalog) takes
config.json, processor_config.json	the source repo's, unchanged (revision f667ed7)
LICENSE, NOTICE	the OpenMDW-1.1 agreement; the origin of the weights and what was converted
SHA256SUMS	every file above

.f32le files are raw little-endian float32 in C order.

Graph contract

inputs:  packed [1, T, 512]    float32  [speaker cache | FIFO | chunk + look-ahead] rows, left-packed in [0, L); rows [L, T) zero
         valid  [1, T]         float32  1.0 for rows < L, 0.0 after
outputs: logits [1, T*8, 8]    float32  speaker logits at 10 ms; rows [0, L*8) are real
T = 541 for the three streaming modes, 684 offline. Both fp16 bundles take and return float32.

Host

1. Log-mel: pre-emphasis 0.97 (the first sample kept), 512-point STFT with hop 160 and the 400-sample window centered, |X|², 128 slaney mel bins, log(x + 2⁻²⁴). No normalization. Each streaming chunk computes its mel from its own audio slice, as the source card's inputs_generator does.
2. Every 8 mel frames are stacked (1,024 values) and projected with embedder_projection.f32le in float32: one 512-d row per 80 ms.
3. Each step packs [cache | FIFO | chunk + look-ahead] and runs the graph. It keeps logits[:L*8] and average-pools the sigmoid 8× to the cache rate, then emits the chunk's frames.
4. The speaker cache follows transformers' Nemotron3DiarizationSpeakerCache.update / _compress line by line. The one change is precision: the compression scores are float64 (see Verification).
5. Turns: per speaker, every run of probability > 0.5. Overlapping speech stays overlapping.

profile	chunk + look-ahead (encoder frames of 80 ms)	FIFO / update period	graph
low latency	9 + 4 (0.72 s + 0.32 s)	264 / 222	T = 541
very low latency	6 + 2 (0.48 s + 0.16 s)	264 / 222	T = 541
ultra low latency	3 + 1 (0.24 s + 0.08 s)	264 / 222	T = 541
offline	340 + 40, over the whole recording's mel	40 / 300	T = 684

The speaker cache holds 264 rows in every profile. metadata.json carries all of these constants.

Verification

The reference is transformers Nemotron3DiarizationForAudioFrameClassification in fp32. Two clips were used: 97.6 s (the transformers integration-test audio, where the cache compresses 4 times in low latency) and 21.5 s. The score is speaker-activity agreement at 0.5 over every 10 ms frame × 8 speakers, with a bar of 99.9 %. Offline leaves out the last 16 frames, as the transformers integration test does.

fp16 graph on the Mac GPU, full streaming loop:

clip	profile	agreement@0.5 (differing / elements)	max |Δp|	turns: reference / ours / matched
97.6 s	low latency	99.9987 % (1 / 78,072)	0.025	37 / 38 / 36
97.6 s	very low latency	99.9424 % (45 / 78,072)	0.362	43 / 43 / 42
97.6 s	ultra low latency	99.9949 % (4 / 78,072)	0.011	61 / 60 / 59
97.6 s	offline	99.9987 % (1 / 77,952)	0.029	29 / 29 / 29
21.5 s	low / very low / ultra low	100 % (0 / 17,192 each)	≤ 0.0031	10 / 10 / 10
21.5 s	offline	99.9941 % (1 / 17,072)	0.0037	9 / 9 / 9

• The Python host and the Swift host give these numbers with bit-identical logits in all 8 runs.
• The fp32 bundle on the CPU alone, 97.6 s low latency: 100 % (max |Δp| 5.5e-6).
• The compression scores are float64 because transformers' own fp32 top-k choice sits 2 ulp from its boundary in 3 of the 16 compressions of these clips. A float32 host flips one of them in low latency.
• The Mac Neural Engine (the same fp16 graph, one ANE region) reaches 99.7746 % on 97.6 s low latency. That is below the bar, so no ANE bundle is shipped. On 21.5 s it reaches 100 %.
• iPhone 17 Pro (iOS 27.0, build 24A437, thermal state nominal throughout, 2026-09-24), the h18p GPU bundles: 97.6 s low latency 99.9987 % (1 / 78,072), 21.5 s 100 %; offline 99.9987 % and 99.9941 %. Its decisions equal the Mac GPU's on 78,070 of 78,072 elements. The h18p Neural Engine compile reaches 99.7758 % on 97.6 s low latency, the same as on the Mac, so it is not shipped.

Speed. One chunk is 0.72 s of audio in low latency; the offline graph takes 30.4 s per chunk.

device	ms per chunk, median / p90	97.6 s end to end (real-time factor)	load, first / second
iPhone 17 Pro GPU, streaming graph (h18p, iOS 27.0)	30.1 / 30.5	4.16 s (0.043)	0.68 s / 0.12 s
iPhone 17 Pro GPU, offline graph (h18p)	34.5 / 35.8	0.17 s (0.0018)	0.55 s / 0.06 s
M4 Max GPU, macOS 27.0, shared with other jobs	15.4–15.9 / 15.6–16.1	2.1–2.3 s (0.022); one run of six took 6.2 s	0.12 s warm

On the iPhone the first graph call after a load took 2.7 s (streaming) and 0.8 s (offline); the numbers above are the calls after it.

Use

The Swift host is the NemotronDiarizer package in conversion/nemotron3_diar/swift (no dependencies; it links the system CoreAI framework):

let assets = try N3DAssets(directory: repoDir)           // this repo, downloaded
let diarizer = try await N3DDiarizer(assets: assets, computeUnits: .gpu,
                                     profile: .streamingProfile(mode: .lowLatency))
let out = try await diarizer.process(samples: pcm16kMono) // out.probs: [frames, 8]
let turns = N3DDiarizer.segments(from: out.probs)        // speaker, start, end

On iOS it picks the .h18p.aimodelc beside the .aimodel. The Python host is host_loop.py in the same directory. Conversion, gates and the lessons of this port: conversion/nemotron3_diar.

License

OpenMDW-1.1, the source model's license. LICENSE (the agreement) and NOTICE (origin, revision, what was converted) ship in this repo. Keep both with any part of it you redistribute.