Instructions to use AscendKernelGen/KernelGen-LM-14B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AscendKernelGen/KernelGen-LM-14B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AscendKernelGen/KernelGen-LM-14B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AscendKernelGen/KernelGen-LM-14B")
model = AutoModelForCausalLM.from_pretrained("AscendKernelGen/KernelGen-LM-14B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use AscendKernelGen/KernelGen-LM-14B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AscendKernelGen/KernelGen-LM-14B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AscendKernelGen/KernelGen-LM-14B

SGLang

How to use AscendKernelGen/KernelGen-LM-14B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AscendKernelGen/KernelGen-LM-14B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AscendKernelGen/KernelGen-LM-14B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AscendKernelGen/KernelGen-LM-14B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use AscendKernelGen/KernelGen-LM-14B with Docker Model Runner:
```
docker model run hf.co/AscendKernelGen/KernelGen-LM-14B
```

KernelGen-LM-14B

Commit History

Update README.md

85813db
verified

AscendKernelGen commited on 10 days ago

Upload 2 files

3c543a0
verified

AscendKernelGen commited on 10 days ago

Update README.md

09e8ef6
verified

AscendKernelGen commited on 11 days ago

Update README.md

2124239
verified

AscendKernelGen commited on 11 days ago

Update README.md

168e80b
verified

AscendKernelGen commited on 14 days ago

Update README.md

2a94b5a
verified

AscendKernelGen commited on 15 days ago

Add metadata and improve model card (#1)

d1009c5

AscendKernelGen

nielsr HF Staff commited on 18 days ago

Update README.md

267b804
verified

AscendKernelGen commited on Mar 13

Update README.md

3200afd
verified

AscendKernelGen commited on Mar 13

Update README.md

717d992
verified

AscendKernelGen commited on Mar 13

Update README.md

b1827b6
verified

AscendKernelGen commited on Jan 26

Create README.md

89709e5
verified

AscendKernelGen commited on Jan 26

Upload folder using huggingface_hub

5fe40ac
verified

AscendKernelGen commited on Jan 25

initial commit

f1b5e7f
verified

AscendKernelGen commited on Jan 23

Commit History

Update README.md 85813db verified

Upload 2 files 3c543a0 verified

Update README.md 09e8ef6 verified

Update README.md 2124239 verified

Update README.md 168e80b verified

Update README.md 2a94b5a verified

Add metadata and improve model card (#1) d1009c5

Update README.md 267b804 verified

Update README.md 3200afd verified

Update README.md 717d992 verified

Update README.md b1827b6 verified

Create README.md 89709e5 verified

Upload folder using huggingface_hub 5fe40ac verified

initial commit f1b5e7f verified

Update README.md

85813db
verified

Upload 2 files

3c543a0
verified

Update README.md

09e8ef6
verified

Update README.md

2124239
verified

Update README.md

168e80b
verified

Update README.md

2a94b5a
verified

Add metadata and improve model card (#1)

d1009c5

Update README.md

267b804
verified

Update README.md

3200afd
verified

Update README.md

717d992
verified

Update README.md

b1827b6
verified

Create README.md

89709e5
verified

Upload folder using huggingface_hub

5fe40ac
verified

initial commit

f1b5e7f
verified