Instructions to use modrill/code-think-q8b-20260908 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use modrill/code-think-q8b-20260908 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="modrill/code-think-q8b-20260908")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("modrill/code-think-q8b-20260908")
model = AutoModelForCausalLM.from_pretrained("modrill/code-think-q8b-20260908", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use modrill/code-think-q8b-20260908 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "modrill/code-think-q8b-20260908"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/modrill/code-think-q8b-20260908

SGLang

How to use modrill/code-think-q8b-20260908 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "modrill/code-think-q8b-20260908" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "modrill/code-think-q8b-20260908" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "modrill/code-think-q8b-20260908",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use modrill/code-think-q8b-20260908 with Docker Model Runner:
```
docker model run hf.co/modrill/code-think-q8b-20260908
```

modrill commited on 25 days ago

Commit

7a6ade4

verified ·

1 Parent(s): 9c15b47

Add adapter/MANIFEST.json

Browse files

Files changed (1) hide show

adapter/MANIFEST.json +1 -0

adapter/MANIFEST.json ADDED Viewed

	@@ -0,0 +1 @@

+ {"consumed_assistant_target_tokens":64873788,"files":[{"bytes":287,"path":"TOKEN_ROWS_META.json","sha256":"6582772bb6ba64cdc0b49f8f4eaf554f755bbeadd5e5a9e9d053b70904c5e9da"},{"bytes":5246,"path":"adapter/README.md","sha256":"54b2c1daac45cc0c271bdcc954a726fed3c82fd7ca1518fd6db5ad455d82dff5"},{"bytes":1314,"path":"adapter/adapter_config.json","sha256":"1f27a06ce95bd912a609a87c1b723e3cd7cca070f3d17f8142b8f459b606b05f"},{"bytes":698518296,"path":"adapter/adapter_model.safetensors","sha256":"ce1ee762beafb9dc5929615c09ab872f74bf6578581a92b09e87657dda68db90"},{"bytes":1397036976,"path":"optimizer_state.safetensors","sha256":"146d8b8b9307bf786a9739d56e67f77fa52f6e32d09211008ec909fddfe72244"},{"bytes":88052,"path":"resume_state.json","sha256":"4d378f17881dc87a41b75e8c6603d0779f27187e5a5214184f63ff812ee728e6"},{"bytes":147864,"path":"token_rows_both_sides.safetensors","sha256":"1008eec4c65705e35e2079bcfddd00afdd1b25b929bfbca8876bb9ee30598cf1"}],"job_sha256":"a1afcf68f5fd77be7188c8da5de76a7a12177a78f11327daa59c7ae8948c2da3","launch_sha256":"5f96677c848db8cca9311f7c9f1e86cf95365fcbad6067cb124fd37275e34c60","manifest_sha256":"4838cd665f9bb6db2ea1b2a4c056564910e0e7b25ca9e67e7f0b69a387192092","materialized_manifest_sha256":"302a9ef9d08ebbea206c3d3f9eb21dceead777160ea13088b41b2c994c9bb4e8","model_revision":"49e3418fbbbca6ecbdf9608b4d22e5a407081db4","payload_sha256":"c160713fea76eae124dbbc3b5c619d3164862da83f61d91d3970ede990a12540","renderer_authority_sha256":"6b1461404d033b79f05a8cbb373f05a495d506a567b35b18cbf89930f345422a","row_index":9430,"safe_resume":{"adapter":"safetensors","optimizer":"safetensors+json","pickle_used":false,"rng":"json+base64"},"schema":"CODE_SFT_RSTAR_ARM_CHECKPOINT_V1","source_revision":"3a7a0a0636ec96e3c1ec42ebe79ade467caa040d","status":"LOCAL_COMPLETE","step":921,"training_policy_sha256":"979da657dc176702f8821763b699a6897a9c1b90f6286e63d409cf171330cd17"}