---
license: cc-by-4.0
library_name: parakeet.cpp
tags:
  - automatic-speech-recognition
  - asr
  - parakeet
  - gguf
  - ggml
  - cpp-inference
  - nemo
pipeline_tag: automatic-speech-recognition
base_model:
  - nvidia/parakeet-tdt_ctc-110m
  - nvidia/parakeet_realtime_eou_120m-v1
  - nvidia/parakeet-ctc-0.6b
  - nvidia/parakeet-rnnt-0.6b
  - nvidia/parakeet-tdt-0.6b-v2
  - nvidia/parakeet-tdt-0.6b-v3
  - nvidia/parakeet-ctc-1.1b
  - nvidia/parakeet-rnnt-1.1b
  - nvidia/parakeet-tdt-1.1b
  - nvidia/parakeet-tdt_ctc-1.1b
---

# Parakeet GGUF — models for parakeet.cpp

GGUF-format weights for [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++/ggml port of NVIDIA NeMo Parakeet that matches the upstream PyTorch models on CPU. This single repo collects **every supported model × quantization** as a flat set of `.gguf` files — download just the one you need.

**F16 is the recommended default** — same accuracy as F32, ~1.7× smaller, and typically the fastest on modern CPUs via ggml's F32×F16 matmul fast path.

## Models

### tdt_ctc-110m

Source: [nvidia/parakeet-tdt_ctc-110m](https://huggingface.co/nvidia/parakeet-tdt_ctc-110m) · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt_ctc-110m-f16.gguf` ← **recommended** | F16 | 267.5 MB | 0.0000 |
| `tdt_ctc-110m-q8_0.gguf` | Q8_0 | 177.8 MB | 0.0000 |
| `tdt_ctc-110m-q6_k.gguf` | Q6_K | 155.9 MB | not measured |
| `tdt_ctc-110m-q5_k.gguf` | Q5_K | 143.3 MB | not measured |
| `tdt_ctc-110m-q4_k.gguf` | Q4_K | 131.4 MB | 0.0000 |

### realtime_eou_120m-v1

Source: [nvidia/parakeet_realtime_eou_120m-v1](https://huggingface.co/nvidia/parakeet_realtime_eou_120m-v1) · Cache-aware streaming RNNT (FastConformer, EOU/EOB) · heads: RNNT (streaming)

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `realtime_eou_120m-v1-f16.gguf` ← **recommended** | F16 | 266.5 MB | not measured |
| `realtime_eou_120m-v1-q8_0.gguf` | Q8_0 | 176.0 MB | not measured |
| `realtime_eou_120m-v1-q6_k.gguf` | Q6_K | 153.9 MB | not measured |
| `realtime_eou_120m-v1-q5_k.gguf` | Q5_K | 141.2 MB | not measured |
| `realtime_eou_120m-v1-q4_k.gguf` | Q4_K | 129.1 MB | not measured |

### ctc-0.6b

Source: [nvidia/parakeet-ctc-0.6b](https://huggingface.co/nvidia/parakeet-ctc-0.6b) · CTC (FastConformer) · heads: CTC

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `ctc-0.6b-f16.gguf` ← **recommended** | F16 | 1373.4 MB | 0.0000 |
| `ctc-0.6b-q8_0.gguf` | Q8_0 | 875.4 MB | 0.0000 |
| `ctc-0.6b-q6_k.gguf` | Q6_K | 746.8 MB | not measured |
| `ctc-0.6b-q5_k.gguf` | Q5_K | 676.3 MB | not measured |
| `ctc-0.6b-q4_k.gguf` | Q4_K | 609.9 MB | not measured |

### rnnt-0.6b

Source: [nvidia/parakeet-rnnt-0.6b](https://huggingface.co/nvidia/parakeet-rnnt-0.6b) · RNNT transducer (FastConformer) · heads: RNNT

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `rnnt-0.6b-f16.gguf` ← **recommended** | F16 | 1402.8 MB | 0.0000 |
| `rnnt-0.6b-q8_0.gguf` | Q8_0 | 903.9 MB | 0.0000 |
| `rnnt-0.6b-q6_k.gguf` | Q6_K | 776.3 MB | not measured |
| `rnnt-0.6b-q5_k.gguf` | Q5_K | 705.7 MB | not measured |
| `rnnt-0.6b-q4_k.gguf` | Q4_K | 639.2 MB | not measured |

### tdt-0.6b-v2

Source: [nvidia/parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2) · TDT transducer (FastConformer) · heads: TDT

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-0.6b-v2-f16.gguf` ← **recommended** | F16 | 1404.2 MB | 0.0000 |
| `tdt-0.6b-v2-q8_0.gguf` | Q8_0 | 903.8 MB | 0.0000 |
| `tdt-0.6b-v2-q6_k.gguf` | Q6_K | 775.9 MB | not measured |
| `tdt-0.6b-v2-q5_k.gguf` | Q5_K | 705.0 MB | not measured |
| `tdt-0.6b-v2-q4_k.gguf` | Q4_K | 638.4 MB | not measured |

### tdt-0.6b-v3

Source: [nvidia/parakeet-tdt-0.6b-v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) · TDT transducer (FastConformer) · heads: TDT

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-0.6b-v3-f16.gguf` ← **recommended** | F16 | 1441.0 MB | 0.0000 |
| `tdt-0.6b-v3-q8_0.gguf` | Q8_0 | 940.7 MB | 0.0000 |
| `tdt-0.6b-v3-q6_k.gguf` | Q6_K | 812.7 MB | not measured |
| `tdt-0.6b-v3-q5_k.gguf` | Q5_K | 741.9 MB | not measured |
| `tdt-0.6b-v3-q4_k.gguf` | Q4_K | 675.2 MB | not measured |

### ctc-1.1b

Source: [nvidia/parakeet-ctc-1.1b](https://huggingface.co/nvidia/parakeet-ctc-1.1b) · CTC (FastConformer) · heads: CTC

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `ctc-1.1b-f16.gguf` ← **recommended** | F16 | 2395.8 MB | 0.0000 |
| `ctc-1.1b-q8_0.gguf` | Q8_0 | 1526.3 MB | 0.0000 |
| `ctc-1.1b-q6_k.gguf` | Q6_K | 1301.7 MB | not measured |
| `ctc-1.1b-q5_k.gguf` | Q5_K | 1178.5 MB | not measured |
| `ctc-1.1b-q4_k.gguf` | Q4_K | 1062.6 MB | not measured |

### rnnt-1.1b

Source: [nvidia/parakeet-rnnt-1.1b](https://huggingface.co/nvidia/parakeet-rnnt-1.1b) · RNNT transducer (FastConformer) · heads: RNNT

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `rnnt-1.1b-f16.gguf` ← **recommended** | F16 | 2425.2 MB | 0.0000 |
| `rnnt-1.1b-q8_0.gguf` | Q8_0 | 1554.7 MB | 0.0000 |
| `rnnt-1.1b-q6_k.gguf` | Q6_K | 1331.2 MB | not measured |
| `rnnt-1.1b-q5_k.gguf` | Q5_K | 1207.9 MB | not measured |
| `rnnt-1.1b-q4_k.gguf` | Q4_K | 1091.9 MB | not measured |

### tdt-1.1b

Source: [nvidia/parakeet-tdt-1.1b](https://huggingface.co/nvidia/parakeet-tdt-1.1b) · TDT transducer (FastConformer) · heads: TDT

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt-1.1b-f16.gguf` ← **recommended** | F16 | 2425.3 MB | 0.0000 |
| `tdt-1.1b-q8_0.gguf` | Q8_0 | 1554.8 MB | 0.0000 |
| `tdt-1.1b-q6_k.gguf` | Q6_K | 1331.2 MB | not measured |
| `tdt-1.1b-q5_k.gguf` | Q5_K | 1207.9 MB | not measured |
| `tdt-1.1b-q4_k.gguf` | Q4_K | 1091.9 MB | not measured |

### tdt_ctc-1.1b

Source: [nvidia/parakeet-tdt_ctc-1.1b](https://huggingface.co/nvidia/parakeet-tdt_ctc-1.1b) · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

| File | Variant | Size | WER vs NeMo |
|---|---|---:|---:|
| `tdt_ctc-1.1b-f16.gguf` ← **recommended** | F16 | 2429.5 MB | 0.0000 |
| `tdt_ctc-1.1b-q8_0.gguf` | Q8_0 | 1559.0 MB | 0.0000 |
| `tdt_ctc-1.1b-q6_k.gguf` | Q6_K | 1335.4 MB | not measured |
| `tdt_ctc-1.1b-q5_k.gguf` | Q5_K | 1212.1 MB | not measured |
| `tdt_ctc-1.1b-q4_k.gguf` | Q4_K | 1096.1 MB | not measured |

> WER (word error rate) is computed against the upstream NeMo reference on `tests/fixtures/speech.wav` (LibriSpeech `2086-149220-0033`, ~7.4 s, English). 0.0 = byte-for-byte identical transcript. See [parity.md](https://github.com/mudler/parakeet.cpp/blob/main/docs/parity.md) and [quantization.md](https://github.com/mudler/parakeet.cpp/blob/main/docs/quantization.md).

## Quantization notes

Quantization is applied **only** to the large linear weights fed directly into `ggml_mul_mat` (encoder FFN + attention projections, subsampling output projection, joint enc/pred projections). All other tensors (mel filterbank, LSTM prediction net, conv kernels, batch_norm stats, norms, biases, embeddings) stay F32.

## Usage

```bash
# 1. Clone + build parakeet.cpp
git clone https://github.com/mudler/parakeet.cpp
cd parakeet.cpp
cmake -B build -DPARAKEET_BUILD_CLI=ON && cmake --build build -j

# 2. Download one quant (F16 recommended)
huggingface-cli download mudler/parakeet-cpp-gguf tdt_ctc-110m-f16.gguf --local-dir models/

# 3. Transcribe
build/examples/cli/parakeet-cli transcribe \
    --model models/tdt_ctc-110m-f16.gguf \
    --input audio.wav
```

## License

The GGUF weights are derived from the NVIDIA NeMo Parakeet checkpoints, released under the [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/) license. The parakeet.cpp runtime is MIT-licensed.