Instructions to use AscendKernelGen/KernelGen-LM-32B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AscendKernelGen/KernelGen-LM-32B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AscendKernelGen/KernelGen-LM-32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AscendKernelGen/KernelGen-LM-32B")
model = AutoModelForCausalLM.from_pretrained("AscendKernelGen/KernelGen-LM-32B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AscendKernelGen/KernelGen-LM-32B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AscendKernelGen/KernelGen-LM-32B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AscendKernelGen/KernelGen-LM-32B

SGLang

How to use AscendKernelGen/KernelGen-LM-32B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AscendKernelGen/KernelGen-LM-32B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AscendKernelGen/KernelGen-LM-32B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-32B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AscendKernelGen/KernelGen-LM-32B with Docker Model Runner:
```
docker model run hf.co/AscendKernelGen/KernelGen-LM-32B
```

KernelGen-LM-32B

Commit History

Update README.md

4477bd6
verified

AscendKernelGen commited on 10 days ago

Upload 2 files

40cfbd5
verified

AscendKernelGen commited on 10 days ago

Update README.md

9e16db9
verified

AscendKernelGen commited on 11 days ago

Update README.md

675fab8
verified

AscendKernelGen commited on 11 days ago

Update README.md

5aca577
verified

AscendKernelGen commited on 14 days ago

Update README.md

fb0d041
verified

AscendKernelGen commited on 14 days ago

Add metadata and links to paper and code (#1)

116d162

AscendKernelGen

nielsr HF Staff commited on 17 days ago

Update README.md

f298491
verified

AscendKernelGen commited on Mar 13

Update README.md

20d682c
verified

AscendKernelGen commited on Mar 13

Update README.md

2a6428e
verified

AscendKernelGen commited on Mar 13

Update README.md

2bfaa94
verified

AscendKernelGen commited on Jan 26

Update README.md

32ea34d
verified

AscendKernelGen commited on Jan 26

Update README.md

8ea5972
verified

AscendKernelGen commited on Jan 26

Update README.md

85ed22e
verified

AscendKernelGen commited on Jan 26

Create README.md

6684420
verified

AscendKernelGen commited on Jan 26

Upload folder using huggingface_hub

3979192
verified

AscendKernelGen commited on Jan 25

initial commit

a7343a1
verified

AscendKernelGen commited on Jan 23

Commit History

Update README.md 4477bd6 verified

Upload 2 files 40cfbd5 verified

Update README.md 9e16db9 verified

Update README.md 675fab8 verified

Update README.md 5aca577 verified

Update README.md fb0d041 verified

Add metadata and links to paper and code (#1) 116d162

Update README.md f298491 verified

Update README.md 20d682c verified

Update README.md 2a6428e verified

Update README.md 2bfaa94 verified

Update README.md 32ea34d verified

Update README.md 8ea5972 verified

Update README.md 85ed22e verified

Create README.md 6684420 verified

Upload folder using huggingface_hub 3979192 verified

initial commit a7343a1 verified

Update README.md

4477bd6
verified

Upload 2 files

40cfbd5
verified

Update README.md

9e16db9
verified

Update README.md

675fab8
verified

Update README.md

5aca577
verified

Update README.md

fb0d041
verified

Add metadata and links to paper and code (#1)

116d162

Update README.md

f298491
verified

Update README.md

20d682c
verified

Update README.md

2a6428e
verified

Update README.md

2bfaa94
verified

Update README.md

32ea34d
verified

Update README.md

8ea5972
verified

Update README.md

85ed22e
verified

Create README.md

6684420
verified

Upload folder using huggingface_hub

3979192
verified

initial commit

a7343a1
verified