Instructions to use modrill/code-think-q8b-20260908 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use modrill/code-think-q8b-20260908 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="modrill/code-think-q8b-20260908")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("modrill/code-think-q8b-20260908")
model = AutoModelForCausalLM.from_pretrained("modrill/code-think-q8b-20260908", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use modrill/code-think-q8b-20260908 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "modrill/code-think-q8b-20260908"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/modrill/code-think-q8b-20260908

SGLang

How to use modrill/code-think-q8b-20260908 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "modrill/code-think-q8b-20260908" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "modrill/code-think-q8b-20260908" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use modrill/code-think-q8b-20260908 with Docker Model Runner:
```
docker model run hf.co/modrill/code-think-q8b-20260908
```

modrill commited on 26 days ago

Commit

85a0cf9

verified ·

1 Parent(s): 92fd8fc

Add MANIFEST.sha256

Browse files

Files changed (1) hide show

MANIFEST.sha256 +18 -0

MANIFEST.sha256 ADDED Viewed

	@@ -0,0 +1,18 @@

+832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e  LICENSE
+72eaad7a1a3d9a7a189911c68a0b988bf0db489b72a7a9ca4db330cd799c93e2  OFFICIAL_MERGE_RECEIPT.json
+8831f2b2fa7a62a7ca092f2cf0fadc6371f6435b605caafe627440e4cf05d162  README.md
+65d2f5da82e124525dd5fd58b13d4eb0a2663496f808a8112bfc1c3146a80ac7  adapter/MANIFEST.json
+6582772bb6ba64cdc0b49f8f4eaf554f755bbeadd5e5a9e9d053b70904c5e9da  adapter/TOKEN_ROWS_META.json
+1f27a06ce95bd912a609a87c1b723e3cd7cca070f3d17f8142b8f459b606b05f  adapter/adapter_config.json
+ce1ee762beafb9dc5929615c09ab872f74bf6578581a92b09e87657dda68db90  adapter/adapter_model.safetensors
+1008eec4c65705e35e2079bcfddd00afdd1b25b929bfbca8876bb9ee30598cf1  adapter/token_rows_both_sides.safetensors
+87a2728cb8dc9fe424d624542f6060ec05a1d285ebbec578bb078900e33396b5  chat_template.jinja
+fe6e3da8b0248b30bb0e9f60f297d86a5c96850c2c853dda61f20e3851b4096c  config.json
+a2ac0d07aac37502207a212b122abe32bd9564266594a317c0ff490ebce1f226  generation_config.json
+35d6fce35ef7d5eaa0c50fa5d9e1b1f9ab54234dcefbbd3c684d91078e61d4b6  model.safetensors
+51727ee1665903a9cde33a750d3b003b91acfc24af297559958b73c0d92d0683  provenance/DEV256_COMPLETE.json
+979da657dc176702f8821763b699a6897a9c1b90f6286e63d409cf171330cd17  provenance/POLICY.json
+9cd557054bc78b2c5431d4827013feb6b3c5a0b27a46b0237931d5a4bc72ad9b  provenance/RUN_IDENTITY.json
+f7778207bdc9391836a6aec8509b1b7c6816debd0ef0d50643acf4c9332308f4  provenance/TRAINING_CONFIG.json
+be75606093db2094d7cd20f3c2f385c212750648bd6ea4fb2bf507a6a4c55506  tokenizer.json
+1749ac66e3e7ce3862337c23cca5895a2ee131929563e18a944f8c1c80363712  tokenizer_config.json