failed09/bashkir-pair-scorer

Bashkir-Russian Pair Scorer

Compact ONNX models for estimating whether a Bashkir-Russian sentence pair is parallel and suitable for corpus filtering. The release is designed for fast, repeatable scoring of large bilingual corpora and contains three model sizes: nano, mini and medium.

This is an alignment-quality scorer, not a translation model. It returns a compatibility score for a pair of texts and can be used to build a filtering cascade for corpus preparation.

Configurations

nano

The smallest and fastest model for a first-pass filter. Use it to remove clear non-parallel pairs before a more expensive verification stage.

mini

The recommended default for most batch jobs. It provides a good balance between throughput, model size and discrimination quality.

medium

The strongest compact verifier in this release. Use it for the review layer or when a higher-quality local scorer is preferred over maximum throughput.

All configurations use the same SentencePiece vocabulary and the fast128 profile: FP16 weights, dynamic batch dimension and a maximum of 128 tokens per side. This is a deliberate production design for stable speed and predictable memory use. Longer inputs are truncated before scoring.

Files

Path	Purpose
models/nano/model.onnx	Fast first-pass scorer
models/mini/model.onnx	Recommended default scorer
models/medium/model.onnx	Compact higher-quality verifier
spm_bpe_16k.model	Shared SentencePiece tokenizer
config.json	Runtime contract and model metadata
benchmark_summary.json	Machine-readable evaluation results
SHA256SUMS	Release checksums

Source and Processing

The models were trained and exported in the BashkirCorpus project for Bashkir-Russian parallel-corpus quality estimation. Training used reviewed parallel data with hard negative pairs and teacher/reference scoring signals. The ONNX release contains model weights and tokenizer assets only; it does not include the source corpus text.

The evaluation uses a calibrated Aygiz-derived bilingual dataset with 10,000 positive pairs and 20,000 deterministic global and local hard negatives. This evaluation sample was excluded from training.

Benchmark

The benchmark compares the released scorers with compact and reference alignment models. Results are reported in benchmark_summary.json.

Model	ROC-AUC	Average Precision	RTX 4060 throughput
LaBSE	0.9944	0.9895	44.8 pairs/s
LaBSE + LASER	0.9939	0.9871	44.8 pairs/s
DevLake BERT	0.9896	0.9706	112.5 pairs/s
Medium Scorer	0.9633	0.9155	2,555.9 pairs/s
Mini Scorer	0.9590	0.9040	4,673.5 pairs/s
Nano Scorer	0.9498	0.8785	7,472.2 pairs/s

The compact scorers trade a small amount of discrimination quality for a large throughput gain. A practical cascade is to use nano or mini for the first pass, medium for uncertain pairs and LaBSE/LASER only for the final review layer.

Loading

import numpy as np
import onnxruntime as ort
import sentencepiece as spm

session = ort.InferenceSession(
    "models/mini/model.onnx",
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
tokenizer = spm.SentencePieceProcessor(model_file="spm_bpe_16k.model")

def encode(ba, ru, max_len=128):
    limit = (max_len - 3) // 2
    ru_ids = tokenizer.encode(ru, out_type=int)[:limit]
    ba_ids = tokenizer.encode(ba, out_type=int)[:limit]
    ids = [1, *ru_ids, 2, *ba_ids, 2]
    ids += [tokenizer.pad_id()] * (max_len - len(ids))
    ids = np.asarray([ids], dtype=np.int64)
    return {"input_ids": ids, "attention_mask": ids != tokenizer.pad_id()}

logit = session.run(["score_logit"], encode("Сәләм донъя", "Привет мир"))[0][0]
score = float(1 / (1 + np.exp(-logit)))
print(score)

For corpus-scale jobs, use the direct batch runner from the source project to avoid HTTP/JSON overhead and stream Parquet, CSV/TSV or JSONL inputs.

Limitations

Scores are not calibrated human probabilities. The accepted, review and quarantine thresholds must be calibrated for a target corpus and a reviewed sample. The benchmark is intended for comparison and does not guarantee production precision on every domain.

The fast128 profile truncates long inputs. For long literary or document segments, use a review layer with a longer-context scorer or LaBSE/LASER.

License

Distributed under the Apache-2.0 license. This release contains derived alignment-scoring software and model assets, not source corpus text, scans or original page layouts.

Citation

@software{failed09_bashkir_pair_scorer_2026,
  title = {Bashkir-Russian Pair Scorer},
  author = {failed09},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/failed09/bashkir-pair-scorer},
  note = {Open-source compact alignment models for the Bashkir language}
}

Open-source models for the preservation and development of the Bashkir language.