Instructions to use Kwaipilot/KAT-Coder-V2.5-Dev with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kwaipilot/KAT-Coder-V2.5-Dev with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kwaipilot/KAT-Coder-V2.5-Dev")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Kwaipilot/KAT-Coder-V2.5-Dev")
model = AutoModelForMultimodalLM.from_pretrained("Kwaipilot/KAT-Coder-V2.5-Dev", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kwaipilot/KAT-Coder-V2.5-Dev with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kwaipilot/KAT-Coder-V2.5-Dev"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Coder-V2.5-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kwaipilot/KAT-Coder-V2.5-Dev

SGLang

How to use Kwaipilot/KAT-Coder-V2.5-Dev with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kwaipilot/KAT-Coder-V2.5-Dev" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Coder-V2.5-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kwaipilot/KAT-Coder-V2.5-Dev" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kwaipilot/KAT-Coder-V2.5-Dev",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kwaipilot/KAT-Coder-V2.5-Dev with Docker Model Runner:
```
docker model run hf.co/Kwaipilot/KAT-Coder-V2.5-Dev
```

WOW! Not kidding, this model nuts! 💥

#29

by auf1r2 - opened 3 days ago

Discussion

auf1r2

3 days ago

•

edited 3 days ago

Downloaded just for fun, but now it's my №1 model for local coding tasks and code review 💓

Compared it with: Ornith 1.5 35B A3B, Qwen3.8 27B, DeepSeek V4 Flash 0731.

It's faster, MUCH faster
It's on par with Qwen3.8 27B in well determinated tasks
It's best for Code Review across all competitors, even better than DeepSeek in this discipline
It beats Ornith 1.5 35B A3B everywhere, in every REAL coding discipline (most nowadays models tend to benchMAXing results in a cost of real skills - but not this model - it's real performer).
It doesn't beat DeepSeek V4 Flash 0731 in codding, but, common, DS has almost 10x more params.

I would say this model is the best in 35B MoE class.

p.s. running it in llama-server in Q8 quantization:

llama-server --model "/home/xxx/EvoX2/LLMs/models/bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF/Kwaipilot_KAT-Coder-V2.5-Dev-Q8_0.gguf" --host 192.168.1.xx --port 8080 --tools get_info --api-key xxx --jinja --gpu-layers all --parallel 1 --kv-unified --kv-offload --load-mode none --spec-type none --flash-attn on --cache-type-k q8_0 --cache-type-v q8_0 --ctx-size 262144 --batch-size 4096 --ubatch-size 2048 --temperature 0.7 --top-k 30 --repeat-penalty 1.05 --presence-penalty 0.05 --top-p 0.95 --min-p 0.05

auf1r2

1 day ago

Ok, 2 days later.

I have tested it in many various scenarios (not only coding) including deep research to help me with car repair (asked it to find spare parts for my rare car model, make a plan for repair so I can use my car and do repair step by step, compare repair services, calculate an approximate repair bill, try to use different options and compromises) really complicated task. And it finished it successfully!

Asked it to research and do different calculations on how to build a system of sun panels for my home, how to connect, where to buy, all required parts, works, PRONS and CONS for each option - got extremly success result as well!

No doubt, this model and Kwaipilot team deserves much more stars and attention!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment