Instructions to use Johnblick187/SmartCoderMoE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Johnblick187/SmartCoderMoE with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Johnblick187/SmartCoderMoE", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Johnblick187/SmartCoderMoE", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Johnblick187/SmartCoderMoE with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Johnblick187/SmartCoderMoE"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Johnblick187/SmartCoderMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Johnblick187/SmartCoderMoE

SGLang

How to use Johnblick187/SmartCoderMoE with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Johnblick187/SmartCoderMoE" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Johnblick187/SmartCoderMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Johnblick187/SmartCoderMoE" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Johnblick187/SmartCoderMoE",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Johnblick187/SmartCoderMoE with Docker Model Runner:
```
docker model run hf.co/Johnblick187/SmartCoderMoE
```

Johnblick187 commited on 3 days ago

Commit

cc30de2

verified ·

1 Parent(s): 4e7c0cc

Update README.md

Browse files

Files changed (1) hide show

README.md +3 -1

README.md CHANGED Viewed

@@ -67,4 +67,6 @@ SmartCoderMoE’s 2048 hidden size was chosen to natively align with:
 Coding. Lots of it. Uncensored.
 ## Note from the Creator
-As of the writing of this model card (Thursday, May 21st, 2026), the model is not finished. Multimodal expansion, as mentioned before, is on the way. As is a very unique calculation of how much of the original Starcoder knowledge remains. i will update the repo as i go. Feel free to use it while i build on it if you desire, and if you decide to do this and encounter any sort of issues woth it, please let me know so that i can fix it asap!

 Coding. Lots of it. Uncensored.
 ## Note from the Creator
+As of the writing of this model card (Thursday, May 21st, 2026), the model is not finished. Multimodal expansion, as mentioned before, is on the way. As is a very unique calculation of how much of the original Starcoder knowledge remains. i will update the repo as i go. Feel free to use it while i build on it if you desire, and if you decide to do this and encounter any sort of issues woth it, please let me know so that i can fix it asap!
+## UPDATE!!!!!
+several bugs detected in the model. due to a saving error, the model's weights were saved with incorrect key mapping. 1 bug causes from pretrained to fail to remap unless it is overridden at a higher level. due to this, everysingle weight is incorrectly saved. this with the fact that he hasnt been trained since conception means he NaNs during inference. Requires full fine tuning to use