Orze-ASR-3Way

Orze-ASR-3Way is a three-checkpoint composite (ensemble) for English ASR. It is not a standalone checkpoint. It uses bosonai/Qwen3-ASR-1.7B-hf-orze as the anchor and two separately published, frozen voters: HojoAI/Hojo-ASR-V1 and OpenMOSS-Team/MOSS-Transcribe-preview-2B.

The anchor transcript is retained unless both voters agree on the same aligned word edit. A substitution or deletion is accepted only when both voters agree; an insertion is accepted only when both voters produce the same insertion at the same anchor boundary. The official English normalizer is applied before word alignment. The same rule and decoding parameters are used on every dataset.

Leaderboard eligibility

This repository is a research and reproduction artifact, not an eligible Open ASR Leaderboard submission. Orze-ASR-3Way combines three checkpoints, and the leaderboard does not allow composite/ensemble systems. Submission PR #206 was withdrawn and closed on 2026-08-23. The system must not be assigned or presented with an official leaderboard placement or rank.

The results below are self-evaluated, benchmark-fitted development measurements only. They were not verified by the leaderboard maintainers and do not establish performance on the default private scripted or private conversational datasets. Neither the Appen nor DataOcean private leaderboard datasets was accessed during this work.

Benchmark-fitting disclosure

The Open ASR public test-set scores, the public AppTek and MInDS-14 proxy scores, and other public proxy evaluations were used during candidate, decoding, and consensus-rule selection. Consequently, the numbers on this card are benchmark-fitted development results; they are not an independent estimate of generalization.

The Orze anchor's fine-tuning data excluded the Open ASR public test audio and the selected AppTek and MInDS-14 proxy audio. See the anchor card for its exact training-data and checkpoint-construction disclosure. The two voters are third-party published checkpoints; their training disclosures are controlled by their respective authors.

Immutable components

Role	Checkpoint	Revision
Anchor	bosonai/Qwen3-ASR-1.7B-hf-orze	bce834cee3d106a9f0ec1ba98e96d995f13f59de
Voter 1	HojoAI/Hojo-ASR-V1	a22c381896ce5f4b70038982bfb888fe9969ed99
Voter 2	OpenMOSS-Team/MOSS-Transcribe-preview-2B	c4b3988677df13c14e79d9db59f356ed761db366

The reproducible Docker evaluator is hosted at erik-at-boson/open-asr-leaderboard-orze-ensemble. It runs the checkpoints sequentially and includes the sum of all three inference times in RTFx.

Self-evaluated public development results

These results use the evaluator and normalizer from the Open ASR Leaderboard repository at commit d1e99b25524814332d6868a5645e568670834cfb.

Dataset	WER (%)
AMI Cleaned	6.93
Earnings22 Cleaned AA (chunked)	5.24
GigaSpeech Cleaned	6.68
LibriSpeech test-clean	1.02
LibriSpeech test-other	2.41
SPGISpeech	1.58
VoxPopuli Cleaned AA	2.78
Mean	3.81

Aggregate component throughput is 29.66 RTFx, calculated from the sum of all three component inference times. The evaluator also records consensus alignment overhead when it creates fresh manifests.

Integrity recheck on 2026-08-23: the pinned scorer reproduced every displayed WER and the 3.81 macro mean; regenerating all seven manifests from the recorded anchor/voter manifests reproduced all 72,341 rows exactly (zero differing rows). This is a reproducibility statement, not an official verification.

Public proxy checks used during development

AppTek Call-Center Dialogues is a public CC-BY-SA-4.0 dataset and is unrelated to the leaderboard's private Appen datasets. Three deterministic 200-segment selections (seeds 8262026, 8262027, and 8262028) used different calls and had zero pairwise segment overlap. They were excluded from the Orze anchor's fine-tuning data. Lower WER is better.

Selection	Anchor overall	Consensus overall	Anchor accent macro	Consensus accent macro
1	10.23	9.88	11.274	10.786
2	7.24	6.99	7.024	6.762
3	9.46	9.15	9.216	8.954

Both individual voters were worse than the anchor on all three selections. This supports the behavior of the unanimity gate on these selected public proxies; it does not establish performance on the private leaderboard families.

A deterministic, balanced 300-utterance selection from a stable 20% MInDS-14 partition spanning en-AU, en-GB, and en-US was also used during development. This partition was excluded from anchor fine-tuning. Overall WER changed from 31.24 to 31.04, and locale-macro WER from 31.32 to 31.13. The consensus changed 87 normalized transcripts: 38 improved, 34 worsened, and 15 were error-neutral, for 10 fewer word errors. Hojo and MOSS individually scored 52.37 and 39.18 WER on this selection.

Scope and limitations

This is a 9.64B-parameter composite, using evaluator-reported component counts rounded to two decimals (2.04B Orze Qwen, 5.18B Hojo, and 2.42B MOSS). It trades throughput and total storage for accuracy. Components run sequentially, so peak inference memory is the maximum of the components rather than their sum. The public results were actively used in system selection and should not be interpreted as a blind-test comparison.

License

The consensus code and all three checkpoint repositories at the pinned revisions declare Apache-2.0. The public proxy datasets retain their own licenses; no proxy audio is redistributed in this repository.