Instructions to use llm-stacking/StackLLM_7B_300BToken with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use llm-stacking/StackLLM_7B_300BToken with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="llm-stacking/StackLLM_7B_300BToken")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("llm-stacking/StackLLM_7B_300BToken")
model = AutoModelForCausalLM.from_pretrained("llm-stacking/StackLLM_7B_300BToken")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use llm-stacking/StackLLM_7B_300BToken with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "llm-stacking/StackLLM_7B_300BToken"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-stacking/StackLLM_7B_300BToken",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/llm-stacking/StackLLM_7B_300BToken

SGLang

How to use llm-stacking/StackLLM_7B_300BToken with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "llm-stacking/StackLLM_7B_300BToken" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-stacking/StackLLM_7B_300BToken",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "llm-stacking/StackLLM_7B_300BToken" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "llm-stacking/StackLLM_7B_300BToken",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use llm-stacking/StackLLM_7B_300BToken with Docker Model Runner:
```
docker model run hf.co/llm-stacking/StackLLM_7B_300BToken
```

StackLLM_7B_300BToken

File size: 4,049 Bytes

1dffaf0

<!DOCTYPE html><html class="no-js" lang="en"><head><meta charset="utf-8"><meta http-equiv="x-ua-compatible" content="ie=edge"><title>index - powered by h5ai v0.30.0 (https://larsjung.de/h5ai/)</title><meta name="description" content="index - powered by h5ai v0.30.0 (https://larsjung.de/h5ai/)"><meta name="viewport" content="width=device-width, initial-scale=1"><link rel="shortcut icon" href="/_h5ai/public/images/favicon/favicon-16-32.ico"><link rel="apple-touch-icon-precomposed" type="image/png" href="/_h5ai/public/images/favicon/favicon-152.png"><link rel="stylesheet" href="/_h5ai/public/css/styles.css"><script src="/_h5ai/public/js/scripts.js" data-module="index"></script><link rel="stylesheet" href="//fonts.googleapis.com/css?family=Ubuntu:300,400,700%7CUbuntu+Mono:400,700" class="x-head"><style class="x-head">#root,input,select{font-family:"Ubuntu","Roboto","Helvetica","Arial","sans-serif"!important}pre,code{font-family:"Ubuntu Mono","Monaco","Lucida Sans Typewriter","monospace"!important}</style></head><body class="index" id="root"><div id="fallback-hints"><span class="noJsMsg">Works best with JavaScript enabled!</span><span class="noBrowserMsg">Works best in <a href="http://browsehappy.com">modern browsers</a>!</span><span class="backlink"><a href="https://larsjung.de/h5ai/" title="h5ai v0.30.0 - Modern HTTP web server index.">powered by h5ai</a></span></div><div id="fallback"><table><tr><th class="fb-i"></th><th class="fb-n"><span>Name</span></th><th class="fb-d"><span>Last modified</span></th><th class="fb-s"><span>Size</span></th></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/folder-parent.png" alt="folder-parent"/></td><td class="fb-n"><a href="..">Parent Directory</a></td><td class="fb-d"></td><td class="fb-s"></td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/config.json">config.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">0 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/generation_config.json">generation_config.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">0 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/pytorch_model.bin">pytorch_model.bin</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">23732526 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/special_tokens_map.json">special_tokens_map.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">0 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/tokenizer.json">tokenizer.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">1842 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/tokenizer.model">tokenizer.model</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">499 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/tokenizer_config.json">tokenizer_config.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">0 KB</td></tr><tr><td class="fb-i"><img src="/_h5ai/public/images/fallback/file.png" alt="file"/></td><td class="fb-n"><a href="/ckpts/b2b_7B_from_1.7B_10B/iter-150000-ckpt/vocab.json">vocab.json</a></td><td class="fb-d">2024-02-02 21:32</td><td class="fb-s">680 KB</td></tr></table></div></body></html><!-- h5ai v0.30.0 - https://larsjung.de/h5ai/ -->