Instructions to use Tiiny/SmallThinker-4BA0.6B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tiiny/SmallThinker-4BA0.6B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tiiny/SmallThinker-4BA0.6B-Instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Tiiny/SmallThinker-4BA0.6B-Instruct", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Tiiny/SmallThinker-4BA0.6B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tiiny/SmallThinker-4BA0.6B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-4BA0.6B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tiiny/SmallThinker-4BA0.6B-Instruct

SGLang

How to use Tiiny/SmallThinker-4BA0.6B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tiiny/SmallThinker-4BA0.6B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-4BA0.6B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tiiny/SmallThinker-4BA0.6B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tiiny/SmallThinker-4BA0.6B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tiiny/SmallThinker-4BA0.6B-Instruct with Docker Model Runner:
```
docker model run hf.co/Tiiny/SmallThinker-4BA0.6B-Instruct
```

SmallThinker-4BA0.6B-Instruct

Commit History

Improve model card: Add `library_name`, explicit paper and code links

6526c24
verified

nielsr HF Staff commited on Jul 29, 2025

Update README.md

512a7f0
verified

Yixin Song commited on Jul 29, 2025

Update README.md

9bed9e1
verified

Yixin Song commited on Jul 29, 2025

Update README.md

261d795
verified

Yixin Song commited on Jul 28, 2025

fix: merge modular_smallthinker and modeling_smallthinker in case of import error

6aabad7

BLGS commited on Jul 27, 2025

Update README.md

80a315d
verified

BLGS commited on Jul 27, 2025

Update README.md

b749512
verified

wdl339 commited on Jul 27, 2025

Update README.md

aadb5ce
verified

Yixin Song commited on Jul 27, 2025

Update README.md

bc2db1c
verified

Yixin Song commited on Jul 27, 2025

Update README.md

b51db6d
verified

Yixin Song commited on Jul 27, 2025

Update README.md

8ca0b32
verified

wdl339 commited on Jul 27, 2025

Update README.md

ad742d6
verified

wdl339 commited on Jul 27, 2025

Update README.md

c487e66
verified

Sorrymaker2024 commited on Jul 27, 2025

Update README.md

0884a31
verified

Sorrymaker2024 commited on Jul 27, 2025

Update README.md

1671d12
verified

Sorrymaker2024 commited on Jul 27, 2025

Update README.md

181de07
verified

Yixin Song commited on Jul 27, 2025

Update README.md

2d1917f
verified

Yixin Song commited on Jul 27, 2025

Update README.md

6d1e0f6
verified

yzmizeyu commited on Jul 27, 2025

Update README.md

0c0193a
verified

yzmizeyu commited on Jul 27, 2025

Update README.md

fed68e3
verified

Yixin Song commited on Jul 27, 2025

Update README.md

3044e6c
verified

Yixin Song commited on Jul 27, 2025

Update README.md

c4c7dfc
verified

Yixin Song commited on Jul 27, 2025

Upload folder using huggingface_hub

84b3156
verified

Yixin Song commited on Jul 26, 2025

initial commit

f5d2c9a
verified

Yixin Song commited on Jul 26, 2025

Commit History

Improve model card: Add `library_name`, explicit paper and code links 6526c24 verified

Update README.md 512a7f0 verified

Update README.md 9bed9e1 verified

Update README.md 261d795 verified

fix: merge modular_smallthinker and modeling_smallthinker in case of import error 6aabad7

Update README.md 80a315d verified

Update README.md b749512 verified

Update README.md aadb5ce verified

Update README.md bc2db1c verified

Update README.md b51db6d verified

Update README.md 8ca0b32 verified

Update README.md ad742d6 verified

Update README.md c487e66 verified

Update README.md 0884a31 verified

Update README.md 1671d12 verified

Update README.md 181de07 verified

Update README.md 2d1917f verified

Update README.md 6d1e0f6 verified

Update README.md 0c0193a verified

Update README.md fed68e3 verified

Update README.md 3044e6c verified

Update README.md c4c7dfc verified

Upload folder using huggingface_hub 84b3156 verified

initial commit f5d2c9a verified

Improve model card: Add `library_name`, explicit paper and code links

6526c24
verified

Update README.md

512a7f0
verified

Update README.md

9bed9e1
verified

Update README.md

261d795
verified

fix: merge modular_smallthinker and modeling_smallthinker in case of import error

6aabad7

Update README.md

80a315d
verified

Update README.md

b749512
verified

Update README.md

aadb5ce
verified

Update README.md

bc2db1c
verified

Update README.md

b51db6d
verified

Update README.md

8ca0b32
verified

Update README.md

ad742d6
verified

Update README.md

c487e66
verified

Update README.md

0884a31
verified

Update README.md

1671d12
verified

Update README.md

181de07
verified

Update README.md

2d1917f
verified

Update README.md

6d1e0f6
verified

Update README.md

0c0193a
verified

Update README.md

fed68e3
verified

Update README.md

3044e6c
verified

Update README.md

c4c7dfc
verified

Upload folder using huggingface_hub

84b3156
verified

initial commit

f5d2c9a
verified