README.md

---
language:
  - uz
  - en
  - ru
license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: text-generation
base_model: Qwen/Qwen3.5-4B
tags:
  - qwen3.5
  - uzbek
  - conversational
  - translation
  - text-generation-inference
  - non-commercial
datasets:
  - HuggingFaceFW/fineweb-2
  - tahrirchi/uz-books
  - tahrirchi/uz-crawl
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceTB/finemath
---

# NeuronAI-4B

**NeuronAI-4B** is an Uzbek-first, bilingual assistant model built from
Qwen3.5-4B. It combines an Uzbek tokenizer retrofit, continued pretraining,
annealing, and assistant-only supervised fine-tuning. The published weights are
fully merged—no LoRA adapter is needed.

![Strict eight-task benchmark comparison](assets/overall_score.png)

> **License:** free for non-commercial use under
> [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/).
> Commercial use requires a separate written license. Contact
> **[neuronaiuz@gmail.com](mailto:neuronaiuz@gmail.com)** to discuss commercial terms.

## Quick start

Install a recent Transformers build with Qwen3.5 support:

```bash
pip install -U "transformers>=5.1" accelerate torch
```

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NeuronUz/NeuronAI-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map={"": 0},
).eval()

messages = [
    {"role": "system", "content": "Siz foydali va aniq AI yordamchisiz."},
    {"role": "user", "content": "Alisher Navoiy haqida qisqacha aytib bering."},
]
inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=False,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=True,
        temperature=0.7,
        top_p=0.8,
        top_k=20,
        min_p=0.0,
        repetition_penalty=1.0,
        use_cache=True,
    )

reply = tokenizer.decode(
    output[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
).strip()
print(reply)
```

This is the recommended quality-oriented preset for general assistant use:
non-thinking mode with Qwen3.5's instruct sampling settings. Greedy decoding
can cause repetition and lower response quality; reserve `do_sample=False` for
deterministic evaluation or classification. The generation metadata already
registers `<|im_end|>` and `<|endoftext|>` as end-of-sequence tokens. Keep the
combined prompt and output within the validated 4,096-token serving limit.

### Serve with vLLM

```bash
pip install -U vllm
vllm serve NeuronUz/NeuronAI-4B \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --tensor-parallel-size 1 \
  --generation-config vllm \
  --default-chat-template-kwargs '{"enable_thinking":false}' \
  --language-model-only \
  --enable-prefix-caching \
  --mamba-block-size 16 \
  --mamba-cache-mode align
```

```bash
curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "NeuronUz/NeuronAI-4B",
    "messages": [
      {"role": "user", "content": "O‘zbekiston haqida uchta fakt ayting."}
    ],
    "max_tokens": 1024,
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "min_p": 0.0,
    "presence_penalty": 1.5,
    "repetition_penalty": 1.0,
    "chat_template_kwargs": {"enable_thinking": false}
  }'
```

### Classification

For classification, the model works best as a constrained label picker: give the
label set in the prompt, ask for the label only, decode greedily, and cap
`max_new_tokens`. This is exactly the protocol used for the sentiment and news
benchmark scores below.

```python
import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NeuronUz/NeuronAI-4B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map={"": 0},
).eval()

LABELS = [
    "Siyosat", "Iqtisodiyot", "Texnologiya", "Sport", "Madaniyat",
    "Salomatlik", "Oila va Jamiyat", "Ta'lim", "Ekologiya", "Xorijiy Yangiliklar",
]

PROMPT = """Quyidagi o‘zbekcha yangilikni bitta toifaga ajrating. Faqat toifa raqamini yozing.

{labels}

Matn: {text}

Javob:"""


def classify(text: str) -> str:
    prompt = PROMPT.format(
        labels="\n".join(f"{i} - {name}" for i, name in enumerate(LABELS)),
        text=text[:4000],
    )
    inputs = tokenizer.apply_chat_template(
        [{"role": "user", "content": prompt}],
        add_generation_prompt=True,
        enable_thinking=False,
        return_tensors="pt",
        return_dict=True,
    ).to(model.device)

    with torch.inference_mode():
        output = model.generate(
            **inputs,
            max_new_tokens=8,
            do_sample=False,          # greedy: labels must be deterministic
        )

    raw = tokenizer.decode(
        output[0, inputs["input_ids"].shape[1]:],
        skip_special_tokens=True,
    ).strip()
    match = re.search(r"\d+", raw)
    return LABELS[int(match.group())] if match and int(match.group()) < len(LABELS) else raw


print(classify(
    "O‘zbekiston Markaziy banki asosiy stavkani o‘zgarishsiz qoldirdi."
))  # -> Iqtisodiyot
```

Binary sentiment uses the same shape with a two-label set:

```python
SENTIMENT_PROMPT = (
    "Quyidagi o‘zbekcha matnning kayfiyatini aniqlang: 'Ijobiy' yoki 'Salbiy'. "
    "Faqat bitta yorliqni yozing.\n\nMatn: {text}\n\nYorliq:"
)
```

Notes that matter for accuracy:

- **Greedy decoding** (`do_sample=False`). The sampling preset in Quick start is
  for open-ended chat; it adds label noise here.
- **`enable_thinking=False`** — a thinking block spends the token budget before
  the label appears.
- **Small `max_new_tokens`** (8 is enough) plus a regex/prefix parser on the
  output, so a stray word never becomes an invalid prediction.
- **Numbered labels** for many-class tasks: one digit is easier to emit and
  parse than a multi-word category name.
- Keep prompt + text inside the 4,096-token serving limit; truncate long
  articles (`text[:4000]` above).

## Benchmarks

All five model result sets below cover the same full eight-task suite.
Classification and multiple-choice tasks use accuracy; FLORES+ translation
uses COMET. The weighted score is normalized by the 0.95 sum of the published
task weights. All eight NeuronAI-4B tasks completed and passed the
invalid-output gate.

![Per-task comparison](assets/tasks_comparison.png)

| Benchmark | Metric | Weight | **NeuronAI-4B** | Qwen3.5-4B | alloma-8B | Llama-3.1-8B-Instruct-Uz | Mistral-7B-Instruct-Uz |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| UzLiB | accuracy | 0.20 | **61.20%** | 40.30% | 42.40% | 31.65% | 32.78% |
| TUMLU-Uzbek | accuracy | 0.20 | **45.00%** | 40.43% | 20.71% | 32.00% | 33.71% |
| FLORES+ en→uz | COMET | 0.15 | **0.8965** | 0.8555 | 0.8779 | 0.8667 | 0.8859 |
| Uzbek news | accuracy | 0.10 | **79.15%** | 67.34% | 57.77% | 60.34% | 62.09% |
| MMLU English | accuracy | 0.10 | 64.06% | **72.66%** | 53.47% | 47.58% | 29.50% |
| MMLU Uzbek | accuracy | 0.10 | **57.01%** | 52.58% | 40.04% | 38.72% | 35.06% |
| FLORES+ uz→en | COMET | 0.05 | **0.8763** | 0.8618 | 0.8713 | 0.7765 | 0.7826 |
| Uzbek sentiment | accuracy | 0.05 | **95.75%** | 84.82% | 79.94% | 82.59% | 80.83% |
| **Normalized weighted score** | | 1.00 | **0.6724** | 0.5978 | 0.5187 | 0.5095 | 0.4969 |

The alloma-8B run used the `APST` apostrophe preprocessing required by its model
card, and its column combines the full model-card-protocol evaluation with
separately archived full UzLiB, TUMLU-Uzbek, and MMLU-Uzbek runs. NeuronAI-4B,
stock Qwen, and both `behbudiy` Uzbek instruct models were evaluated by the same
strict COMET-primary suite without APST preprocessing. On the two `behbudiy`
models the suite's 3% invalid-output gate was exceeded on TUMLU-Uzbek (5.71% for
both) and, for Mistral-7B-Instruct-Uz, on sentiment (4.59%); those are
answer-format parse failures, so the affected task scores are a floor rather
than a ceiling. Exact source files, scores, and run IDs are included in
[`benchmark_results.json`](benchmark_results.json).

### Run the benchmarks on your computer

The repository includes a portable Alloma-style benchmark runner. It covers
FLORES+ (both directions), Uzbek sentiment, Uzbek news, MMLU English, MMLU Uzbek,
and TUMLU-Uzbek.

```bash
pip install -r https://huggingface.co/NeuronUz/NeuronAI-4B/resolve/main/benchmark-requirements.txt
wget https://huggingface.co/NeuronUz/NeuronAI-4B/resolve/main/benchmark.py
python benchmark.py --limit 200 --output quick-results.json
```

The quick command uses the same seed on 200 examples per dataset. Run all public
examples and add COMET with:

```bash
pip install unbabel-comet
python benchmark.py --limit 0 --comet --output full-results.json
```

Run one task when you only need a short check:

```bash
python benchmark.py --tasks mmlu-uz --limit 200 --output mmlu-uz.json
python benchmark.py --tasks flores --limit 200 --output flores.json
```

`--limit 0` means the full dataset. Only full runs are comparable with the table
above; 200-example quick runs are sanity checks. COMET downloads the
`Unbabel/wmt22-comet-da` evaluator and needs additional disk/RAM.

## Uzbek tokenizer efficiency

The tokenizer is an in-place, primarily **Latin-script Uzbek** retrofit rather
than a vocabulary extension. The initial 20,000-document figure was measured on
training-source `uz-crawl`, so we replaced it with a larger corpus-stratified
test: 118,832 held-out-source documents plus a separate 100,000-document
training-source control. Documents were selected with deterministic SHA-256 bottom-k
sampling (seed `20260825`), exact duplicates were excluded from the selected
sample, tiny texts were filtered, and raw source text was tokenized without
apostrophe normalization.

![Uzbek tokenizer fertility](assets/tokenizer_fertility.png)

| Corpus | Status | Documents | Words | NeuronAI-4B | Qwen3.5-4B | Reduction (95% CI) |
| --- | --- | ---: | ---: | ---: | ---: | ---: |
| Community OSCAR Uzbek | Held-out web source | 100,000 | 7,618,770 | **2.0304** | 3.3639 | **39.64%** (39.57–39.71%) |
| Uzbek legal corpus | Held-out legal source/domain | 18,832 | 2,534,566 | **2.3747** | 2.9705 | **20.06%** (19.55–20.57%) |
| uz-crawl | Training-source control | 100,000 | 20,825,680 | **2.3206** | 3.3224 | **30.15%** (30.02–30.30%) |

Across the two held-out sources combined, the tokenizer uses **35.19%
fewer tokens overall** and **40.90% fewer tokens on Latin-dominant text**,
matching its intended Latin-Uzbek focus.

The paired intervals use 5,000 bootstrap replicates over 1,000 deterministic
document buckets. OSCAR may still have incidental overlap with other public web
corpora and was previously checked in a post-hoc weak-token coverage analysis,
but it contributed no tokenizer-training rows. The legal corpus does not appear
in the tokenizer or training source manifests and is the cleanest source-and-domain
holdout in this test. Full results and
script/length breakdowns: [`fertility_large_20260825.json`](fertility_large_20260825.json)
and [`fertility_large_20260825.md`](fertility_large_20260825.md).

Fertility measures tokenization efficiency—not model quality or measured
decoding speed. The 4B and 2B NeuronAI releases use byte-identical tokenizer
files.

## Training

| Item | Value |
| --- | --- |
| Parameters | 4,205,751,296 (4.206B) |
| Prepared train examples | 151,968 (152,152 source rows) |
| Prepared grouped dev examples | 1,535 (1,537 source rows) |
| Train/dev prompt-group overlap | 0 |
| Sequence length / packing | 2,048 / disabled |
| Training duration / seed | 1 epoch / 42 |
| Batch size | 8 micro × 4 accumulation × 1 GPU = 32 effective |
| Optimizer | Fused AdamW; betas 0.9/0.95; weight decay 0.01; gradient clipping 1.0 |
| Learning-rate schedule | Peak 1e-4; cosine decay; 142 warmup steps (2.99%) |
| LoRA | rank 64, alpha 128, dropout 0.05; 12 projection types; 129,859,584 trainable parameters |
| Loss | Fused causal-LM cross-entropy on assistant-response tokens; prompt tokens masked |
| Precision | bf16 training with TF32; merged embeddings and normalization tensors retained in fp32 |

The mixture is Uzbek-first and includes general assistant conversations,
translation, Uzbek language and literature, spelling, classification, math,
and English-retention examples. Training data is not distributed with this
model repository.

## Intended use

Good fits include non-commercial Uzbek research, education, prototyping,
translation experiments, writing assistance, retrieval-augmented generation,
and local/offline demonstrations.

Commercial deployment, paid products or services, internal business use, and
other activity primarily intended for commercial advantage require a separate
license from NeuronUz. Email [neuronaiuz@gmail.com](mailto:neuronaiuz@gmail.com).

## Limitations

- This is a public-suite-selected checkpoint. The benchmark results are useful
  for reproducibility and relative comparison, but they are not a locked,
  independent estimate of real-world generalization.
- LoRA rank, learning rate, batch size, and dropout were not exhaustively swept;
  the table reports the released run, not globally optimal hyperparameters.
- Stock Qwen3.5-4B remains stronger on English MMLU in this evaluation.
- TUMLU-Uzbek is the weakest reported Uzbek task and should not be treated as
  solved at 45% accuracy.
- The model can hallucinate, repeat biases in its data, or produce unsafe or
  outdated content. It has not been comprehensively safety-evaluated.
- Do not rely on it without expert review for medical, legal, financial, public
  safety, or other high-stakes decisions.
- SFT used sequences up to 2,048 tokens; serving at longer inherited context
  lengths has not been validated here. The published inference examples use
  4,096 tokens.

## License

NeuronAI-4B is released under
[Creative Commons Attribution-NonCommercial 4.0 International](https://creativecommons.org/licenses/by-nc/4.0/).
You may share and adapt it for non-commercial purposes with attribution. This
summary does not replace the license text. See [`LICENSE.md`](LICENSE.md) and
contact [neuronaiuz@gmail.com](mailto:neuronaiuz@gmail.com) for commercial terms.