Sauti TTS — Dia-1.6B Swahili (full SFT)

Open Swahili text-to-speech from Msingi-AI: a full supervised fine-tune of nari-labs/Dia-1.6B on a 500-hour Swahili speech corpus pooled from 15 openly-licensed public datasets.

This is a single-voice model — it takes text and speaks it in one learned voice. It does not clone voices and needs no reference audio.

It handles code-switching, English embedded in Swahili sentences, which is how a great deal of Swahili is actually spoken and where TTS models commonly break down: code-switch CER 0.050 against a multilingual ASR judge.

	Plain Swahili	Code-switched
CER	0.011	0.050
WER	0.054	0.286

Plain Swahili is scored by a Swahili-tuned ASR judge, code-switch by a multilingual one — for a reason. Full tables in Evaluation.

Read this before quoting the numbers. The figures above come from a curated 48-sentence set, generated with ASR-gated retries (up to 4 attempts). On a harder 500-sentence benchmark of unseen news text, plain-Swahili CER is 0.021 with retries and 0.060 single-shot. Retries matter a lot — see Release benchmark for the full picture and for why you should use the retry loop in production.

---

Inference and usage

Use the maintained Msingi-AI/dia-infer runtime. It vendors the compatible Dia inference code and includes the required Swahili mapping: [sw] is mapped to byte 8, exactly as it was during training. No stlohrey/dia-finetuning checkout, source patch, manual model construction, or audio post-processing is required.

Pass bare Swahili text to the runtime. It normalizes the text and adds the [sw] marker itself. Do not use Dia dialogue markers such as [S1] or [S2]; they were not used during training.

Modal (recommended)

An A10G is sufficient for realtime, streaming use. Modal avoids requiring a local GPU and keeps downloaded weights in a persistent Volume:

git clone https://github.com/Msingi-AI/dia-infer.git
cd dia-infer
uv sync --python 3.11 --extra modal
uv run modal setup
uv run modal serve modal_app.py

Modal prints an HTTPS endpoint. Generate a WAV with:

export DIA_URL='https://your-modal-endpoint.modal.run'

curl -fsS "$DIA_URL/generate" \
  -H 'Content-Type: application/json' \
  -d '{
    "text": "Habari za asubuhi, karibu katika matangazo yetu ya leo.",
    "temperature": 1.3,
    "seed": 37
  }' \
  -o output.wav

The same endpoint streams 44.1 kHz mono PCM16 over WebSocket at wss://your-modal-endpoint.modal.run/tts. Send the same JSON request; binary audio chunks are followed by an end JSON event. Run uv run modal deploy modal_app.py when you need a persistent endpoint.

The first container startup downloads the checkpoint and compiles Dia. Keep a container warm while iterating; later requests reuse both the model Volume and the compiled runtime.

Local generation

Local generation is useful for smoke tests; an A10-class CUDA GPU is recommended for realtime performance.

git clone https://github.com/Msingi-AI/dia-infer.git
cd dia-infer
uv sync --python 3.11
uv run python -m dia_infer.download

uv run python infer.py \
  "Habari za asubuhi, karibu katika matangazo yetu ya leo." \
  --output output.wav \
  --no-compile

output.wav is 44.1 kHz mono. Remove --no-compile after the first local test to enable the warmed CUDA path.

Python API

from dia_infer import DiaEngine, GenerationConfig

engine = DiaEngine.load("models/dia", compile=True)
generation = GenerationConfig(temperature=1.3, seed=37)

with open("output.pcm", "wb") as audio:
    for pcm16 in engine.stream_pcm16(
        "Habari za asubuhi, karibu katika matangazo yetu ya leo.",
        generation_config=generation,
    ):
        audio.write(pcm16)

stream_pcm16() yields little-endian signed 16-bit audio; stream() yields NumPy float32 waveform chunks.

Voice reference and generation settings

The runtime loads its repository-managed reference by default. The selected voice, its audio, and its exact transcript live together in references/default.json; select another bundled voice by changing only the top-level id. The transcript must match the reference audio exactly.

Reference conditioning keeps the selected house voice more consistent across requests, but it is not a guaranteed zero-shot voice-cloning solution. It can also make some short outputs sound hurried or unstable. The runtime applies a small pitch-preserving tempo adjustment to reference-conditioned audio by default (conditioned_tempo=0.9); set it to 1.0 to disable it. Treat this as a limited mitigation, not a replacement for a model trained with explicit speaker conditioning.

Start with the validated defaults: temperature=1.3, cfg_scale=3.0, top_p=0.95, and cfg_filter_top_k=45. A fixed seed makes sampling repeatable but does not itself lock a voice. Sampling remains sensitive: occasional empty or poor takes should be retried. For batch-quality workflows, ASR-gated retries are a useful optional check; realtime serving normally uses a single attempt and retries only an empty result.

Long input is segmented automatically while retaining the selected reference. For conversational responses, keep individual generations short; Dia's upstream guidance notes that very short and very long utterances may sound less natural.

---

Evaluation

48-sentence held-out set, stratified across general / named-entities / numbers-dates / code-switch. Two ASR judges: zero-shot openai/whisper-large-v3 and Swahili-tuned Jacaranda-Health/ASR-STT.

Released checkpoint (epoch 10):

Scope	whisper-large-v3 WER / CER	Jacaranda ASR WER / CER
overall_plain_sw (headline)	0.243 / 0.050	0.054 / 0.011
overall (incl. code-switch)	0.254 / 0.050	0.111 / 0.030
general	0.215 / 0.043	0.081 / 0.018
named_entities	0.249 / 0.040	0.056 / 0.010
numbers_dates	0.267 / 0.067	0.025 / 0.006
code_switch	0.286 / 0.050	0.281 / 0.088

Release benchmark: 500 unseen sentences, single-shot vs retried

The table above is our curated 48-clip set with ASR-gated retries. Because that is both small and curated, we also ran a 500-sentence benchmark on Swahili news text (MasakhaNEWS) that neither this model nor its training corpus has seen — each sentence checked against the actual training text and dropped on exact, near-duplicate or shared-5-gram match.

We report it both ways, because the difference is large and you should plan for it:

Scope (Swahili judge, WER / CER)	Single-shot	ASR-gated (≤4 attempts)
overall_plain_sw	0.154 / 0.060	0.095 / 0.021
general	0.081 / 0.015	0.083 / 0.015
named_entities	0.285 / 0.133	0.115 / 0.025
numbers_dates	0.097 / 0.033	0.088 / 0.025
code_switch	0.112 / 0.036	0.112 / 0.036

What this means in practice:

1. Use the retry loop in production. It cuts plain-Swahili CER by ~65% (0.060 → 0.021). Generation is sampling-sensitive; a failed take is usually fixed by re-rolling, and one check-ASR pass is far cheaper than shipping bad audio.
2. Named entities are the weak spot. Single-shot they degrade 5× (CER 0.133 vs 0.025 gated) — proper nouns are rare tokens and destabilise sampling. If your text is name-heavy (news, directories, announcements), retries are not optional.
3. Code-switch is stable either way — identical with and without retries, so its quality comes from the model rather than from re-rolling.
4. Numbers on this set are higher than on the curated 48-clip set (0.021 vs 0.011 gated). News prose is harder and carries source typos and quote-splitting artefacts that count against the model. Use the 48-clip figure only for comparison against our other models on that same set; use this table to predict real-world behaviour.

Read code-switch with the multilingual judge only

Jacaranda-Health/ASR-STT is Swahili-only and cannot transcribe English words, so it scores code-switch backwards. It rated epoch 9 (0.108) worse than epoch 4 (0.101); the multilingual whisper-large-v3 rated epoch 9 better (0.067 vs 0.074) — and the native listener agreed with the multilingual judge. For any code-switch decision, use whisper-large-v3.

Epoch sweep (all 10 epochs)

Plain CER improves monotonically while code-switch does not — which is why the released checkpoint is epoch 10, not the best-plain-CER epoch 9.

Epoch	Plain CER (SW judge)	Code-switch CER (multilingual)
1	0.460	0.126
2	0.022	0.093
3	0.021	0.068
4	0.015	0.074
5	0.013	0.051
6	0.010	0.051
7	0.011	0.063
8	0.012	0.048
9	0.008	0.067
10 (released)	0.011	0.050

Epochs 8, 9 and 10 were auditioned by a native Swahili listener, who approved 9 and 10. Epoch 10 was released for its better code-switch score at statistically indistinguishable plain-Swahili quality.

UTMOS was deliberately not run: it is English-trained and unreliable for Swahili.

How it compares in our internal bake-off

Same 48-clip set, same judges, so these are directly comparable. The other rows are internal reference points and are not part of this release.

Model	Plain SW CER	Code-switch CER (multilingual)	Voice cloning
VoxCPM2 full SFT (not released)	0.007	0.047	✅ zero-shot
this model (Dia full SFT e10)	0.011	0.050	❌ single voice
VoxCPM2 LoRA, 300 steps (not released)	0.012	—	✅
Chatterbox LoRA e30 (not released)	0.022	—	prompt-based
CosyVoice3 e1 (not released)	0.029	—	✅

We publish the comparison rather than only our best number: on plain Swahili a VoxCPM2 fine-tune scores better than this model. This model is competitive on plain Swahili, stronger on code-switching, and needs no reference audio to manage — which is why it is the one we release.

---

Training

Base model	nari-labs/Dia-1.6B
Method	Full supervised fine-tune (all parameters, fp32 + autocast, AdamW8bit)
Epochs	10 (3 825 steps/epoch/rank), ~2 h/epoch, ~21 h total
Hardware	4 × A100-64GB (CINECA Leonardo)
Optimiser	lr 1e-5, 500 warmup steps, grad-clip 1.0
Batching	batch size 2 × grad-accum 4 × 4 GPUs = effective batch 32
Audio	44.1 kHz output (DAC codec)

Effective batch and learning rate deliberately match our VoxCPM2 full SFT (1 × 8 × 4 = 32 at lr 1e-5) so the two runs are comparable.

Training ran in two segments: epochs 1–4, then a warm-start resume for epochs 5–10 after a Leonardo filesystem incident killed the first job mid-epoch-5. The resume restores weights only — optimiser state is not persisted by the trainer, so the LR schedule restarted with a fresh 500-step warmup at epoch 5.

Data

Same pooled corpus as our VoxCPM2 model (sw_voxcpm_corpus_v1: WAXAL swa_tts, OpenSLR-25 Swahili, AfriVoice Swahili subsets, FLEURS), but heavily filtered by Dia's architecture:

	Clips	Hours
Full corpus	99 495	500.00
Usable by Dia (≤ 17.5 s)	30 692	126.11
Dropped (too long)	68 803	373.89

Dia's config.json sets data.audio_length = 1536 DAC frames ≈ 17.9 s; longer clips would be truncated mid-utterance with the full transcript still attached, destroying text/audio alignment. This model therefore saw only 25 % of the corpus that trained VoxCPM2 — re-segmenting the long clips is the single biggest known lever for improving it.

---

Limitations

• Trained on 126 h, not 500 h (see above). The most promising future work.
• Single voice, no cloning. If you need a specific speaker, use VoxCPM2.
• Plain Swahili is behind VoxCPM2 (0.011 vs 0.007 CER).
• Short-form only. Eval is single sentences (~4–9 s); long-form and multi-sentence synthesis are unvalidated.
• Sampling-sensitive — budget for retries. Single-shot plain-Swahili CER is 0.060 vs 0.021 with ASR-gated retries. Named entities are worst affected (0.133 vs 0.025). Occasional takes come back as instant-EOS or babble and are fixed by re-rolling.
• Interior transient sharpness rises with training (median max sample-jump 0.307 at epoch 2 → 0.415 at epoch 10). A jump-threshold detector counts these as "clicks", but it cannot distinguish a click from a plosive, and listening did not confirm degradation. Treat raw click counts as a prompt to listen, not as evidence.
• No formal listening study. Quality confirmed by one native listener, not a MOS panel.

Responsible use

Carried over from the Dia-1.6B disclaimer, and it applies here too:

• Do not use this model to impersonate a real person without their explicit consent.
• Do not use it to generate deceptive or misleading content (fake news, fraudulent audio, misrepresentation).
• Do not use it for illegal or harmful purposes.

By using this model you accept responsibility for upholding the relevant legal and ethical standards in your jurisdiction. The voice in this model is learned from a pooled multi-speaker corpus and is not intended to represent any identifiable individual.

Licence and attribution

This model is released under CC-BY-4.0. You may use it commercially, modify it, and redistribute it, provided you give attribution.

CC-BY-4.0 is chosen because the training data carries CC-BY attribution requirements which must be passed on; the base model is Apache-2.0, which is compatible.

Base model

Model	Licence
nari-labs/Dia-1.6B	Apache-2.0

Training data

All sources are openly licensed. Attribution below satisfies CC-BY-4.0; please carry it forward if you redistribute derivatives.

Source	Dataset	Licence
WAXAL swa_tts	google/WaxalNLP	CC-BY-4.0
FLEURS Swahili (KE)	google/fleurs	CC-BY-4.0
AfriVoice Swahili (agriculture, education, financial, government, health)	DigitalUmuganda/Afrivoice_Swahili	CC-BY-4.0
Swahili Speech 400h	badrex/swahili-speech-400hr	CC-BY-4.0
YodaLingua Swahili	Thomcles/YodaLingua-Swahili	CC-BY-4.0
Kiswahili TTS	Bateesa/kiswahili-tts-dataset	CC-BY-4.0
Swahili TTS	jacksonwambali/swahili-tts-dataset	CC-BY-4.0
Kenyan Swahili (non-standard)	cdli/kenyan_swahili_nonstandard_speech_v1.0	CC-BY-4.0
Swahili words parallel	michsethowusu/swahili-words-speech-text-parallel	CC-BY-4.0 (audio originally published by the International Bible Association)
STEM Swahili speech	stem-content-ai-project/swahili-speech	MIT
OpenSLR-25 (ALFFA)	openslr.org/25	MIT
Common Voice 17 Swahili	mozilla-foundation/common_voice_17_0	CC0-1.0 at time of corpus build; Mozilla moved Common Voice to the Mozilla Data Collective in Oct 2025 — check current terms for your use case

Our thanks to every dataset author above. Swahili speech technology exists because people chose to release this data openly.

Files

File	Purpose
model.pth	fine-tuned Dia-1.6B weights, fp32 (6.0 GB)
config.json	Dia model config (must match the training config)
samples/	48 eval-set generations from this checkpoint
eval/results.json	full per-clip eval output
eval/results.md	eval summary tables

Optimiser/scheduler state is intentionally excluded (not needed for inference).