Instructions to use NeuronUz/NeuronAI-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NeuronUz/NeuronAI-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NeuronUz/NeuronAI-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NeuronUz/NeuronAI-4B")
model = AutoModelForCausalLM.from_pretrained("NeuronUz/NeuronAI-4B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NeuronUz/NeuronAI-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NeuronUz/NeuronAI-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuronUz/NeuronAI-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NeuronUz/NeuronAI-4B

SGLang

How to use NeuronUz/NeuronAI-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NeuronUz/NeuronAI-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuronUz/NeuronAI-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NeuronUz/NeuronAI-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuronUz/NeuronAI-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NeuronUz/NeuronAI-4B with Docker Model Runner:
```
docker model run hf.co/NeuronUz/NeuronAI-4B
```

kmamaroziqov commited on 9 days ago

Commit

de2c622

verified ·

1 Parent(s): e8fdcdc

Add alloma-8B benchmark comparison

Browse files

Files changed (4) hide show

README.md +19 -17
assets/overall_score.png +0 -0
assets/tasks_comparison.png +0 -0
benchmark_results.json +31 -0

README.md CHANGED Viewed

@@ -101,28 +101,30 @@ curl http://localhost:8000/v1/chat/completions \
 ## Benchmarks
-All four models below were evaluated on the same full eight-task suite with
-greedy vLLM decoding. Classification and multiple-choice tasks use accuracy;
-FLORES+ translation uses COMET. The weighted score is normalized by the 0.95
-sum of the published task weights. All eight NeuronAI-4B tasks completed and
-passed the invalid-output gate.
 ![Per-task comparison](assets/tasks_comparison.png)
-| Benchmark | Metric | Weight | **NeuronAI-4B** | Qwen3.5-4B | alloma-3B | alloma-1B |
-| --- | --- | ---: | ---: | ---: | ---: | ---: |
-| UzLiB | accuracy | 0.20 | **61.20%** | 40.30% | 32.08% | 23.32% |
-| TUMLU-Uzbek | accuracy | 0.20 | **45.00%** | 40.43% | 27.71% | 22.00% |
-| FLORES+ en→uz | COMET | 0.15 | **0.8965** | 0.8555 | 0.8673 | 0.7383 |
-| Uzbek news | accuracy | 0.10 | **79.15%** | 67.34% | 13.60% | 25.41% |
-| MMLU English | accuracy | 0.10 | 64.06% | **72.66%** | 38.73% | 21.98% |
-| MMLU Uzbek | accuracy | 0.10 | **57.01%** | 52.58% | 32.74% | 21.11% |
-| FLORES+ uz→en | COMET | 0.05 | **0.8763** | 0.8618 | 0.7954 | 0.7636 |
-| Uzbek sentiment | accuracy | 0.05 | **95.75%** | 84.82% | 38.85% | 79.54% |
-| **Normalized weighted score** | | 1.00 | **0.6724** | 0.5978 | 0.4147 | 0.3661 |
 Alloma runs used the `APST` apostrophe preprocessing required by their model
-cards; NeuronAI and stock Qwen did not. The raw archived scores and run IDs are
 included in [`benchmark_results.json`](benchmark_results.json).
 ### Run the benchmarks on your computer

 ## Benchmarks
+All five model result sets below cover the same full eight-task suite.
+Classification and multiple-choice tasks use accuracy; FLORES+ translation
+uses COMET. The weighted score is normalized by the 0.95 sum of the published
+task weights. All eight NeuronAI-4B tasks completed and passed the
+invalid-output gate.
 ![Per-task comparison](assets/tasks_comparison.png)
+| Benchmark | Metric | Weight | **NeuronAI-4B** | Qwen3.5-4B | alloma-8B | alloma-3B | alloma-1B |
+| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| UzLiB | accuracy | 0.20 | **61.20%** | 40.30% | 42.40% | 32.08% | 23.32% |
+| TUMLU-Uzbek | accuracy | 0.20 | **45.00%** | 40.43% | 20.71% | 27.71% | 22.00% |
+| FLORES+ en→uz | COMET | 0.15 | **0.8965** | 0.8555 | 0.8779 | 0.8673 | 0.7383 |
+| Uzbek news | accuracy | 0.10 | **79.15%** | 67.34% | 57.77% | 13.60% | 25.41% |
+| MMLU English | accuracy | 0.10 | 64.06% | **72.66%** | 53.47% | 38.73% | 21.98% |
+| MMLU Uzbek | accuracy | 0.10 | **57.01%** | 52.58% | 40.04% | 32.74% | 21.11% |
+| FLORES+ uz→en | COMET | 0.05 | **0.8763** | 0.8618 | 0.8713 | 0.7954 | 0.7636 |
+| Uzbek sentiment | accuracy | 0.05 | **95.75%** | 84.82% | 79.94% | 38.85% | 79.54% |
+| **Normalized weighted score** | | 1.00 | **0.6724** | 0.5978 | 0.5187 | 0.4147 | 0.3661 |
 Alloma runs used the `APST` apostrophe preprocessing required by their model
+cards; NeuronAI and stock Qwen did not. The alloma-8B column combines its full
+model-card-protocol evaluation with separately archived full UzLiB,
+TUMLU-Uzbek, and MMLU-Uzbek runs. Exact source files, scores, and run IDs are
 included in [`benchmark_results.json`](benchmark_results.json).
 ### Run the benchmarks on your computer

assets/overall_score.png CHANGED Viewed

assets/tasks_comparison.png CHANGED Viewed

benchmark_results.json CHANGED Viewed

@@ -72,6 +72,37 @@
       },
       "normalized_weighted_score": 0.5977513148832162
     },
     "alloma-3B": {
       "run_id": "alloma_3B_apst_full_20260815_142714",
       "suite_id": "public_uzbek_full_vllm_apst",

       },
       "normalized_weighted_score": 0.5977513148832162
     },
+    "alloma-8B": {
+      "run_id": "alloma_8b_full_public_composite",
+      "suite_id": "public_uzbek_full_composite_apst",
+      "source_files": [
+        "reports/alloma_card_protocol_full/alloma-8B.json",
+        "reports/eval_suites/external_public/alloma_8b_uzlib_full_20260615/uzlib_full.json",
+        "reports/eval_suites/public_full_comet/alloma_8b_strict_missing_uz_metrics/tumlu_uzbek_full_apst.json",
+        "reports/eval_suites/public_full_comet/alloma_8b_strict_missing_uz_metrics/mmlu_uz_full_apst.json"
+      ],
+      "tasks": {
+        "uzlib_slice_full": 0.4239656098871574,
+        "tumlu_uzbek_full": 0.20714285714285716,
+        "flores_en_uz_full": 0.8779,
+        "news_full": 0.5777,
+        "mmlu_english_full": 0.5347,
+        "mmlu_uz_full": 0.40037031761857284,
+        "flores_uz_en_full": 0.8713,
+        "sentiment_binary_full": 0.7994
+      },
+      "invalid_rate": {
+        "uzlib_slice_full": 0.0338527673293928,
+        "tumlu_uzbek_full": 0.25857142857142856,
+        "flores_en_uz_full": null,
+        "news_full": null,
+        "mmlu_english_full": null,
+        "mmlu_uz_full": 0.09051417177040308,
+        "flores_uz_en_full": null,
+        "sentiment_binary_full": null
+      },
+      "normalized_weighted_score": 0.5186512896503792
+    },
     "alloma-3B": {
       "run_id": "alloma_3B_apst_full_20260815_142714",
       "suite_id": "public_uzbek_full_vllm_apst",