Instructions to use lifeofcoding/mastermax-mistral with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lifeofcoding/mastermax-mistral with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lifeofcoding/mastermax-mistral")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lifeofcoding/mastermax-mistral")
model = AutoModelForCausalLM.from_pretrained("lifeofcoding/mastermax-mistral")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use lifeofcoding/mastermax-mistral with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lifeofcoding/mastermax-mistral"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lifeofcoding/mastermax-mistral",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lifeofcoding/mastermax-mistral

SGLang

How to use lifeofcoding/mastermax-mistral with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lifeofcoding/mastermax-mistral" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lifeofcoding/mastermax-mistral",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lifeofcoding/mastermax-mistral" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lifeofcoding/mastermax-mistral",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lifeofcoding/mastermax-mistral with Docker Model Runner:
```
docker model run hf.co/lifeofcoding/mastermax-mistral
```

lifeofcoding commited on Oct 15, 2023

Commit

38ab195

1 Parent(s): 54bc111

Upload MistralForCausalLM

Browse files

Files changed (8) hide show

pytorch_model-00001-of-00008.bin +1 -1
pytorch_model-00002-of-00008.bin +1 -1
pytorch_model-00003-of-00008.bin +1 -1
pytorch_model-00004-of-00008.bin +1 -1
pytorch_model-00005-of-00008.bin +1 -1
pytorch_model-00006-of-00008.bin +1 -1
pytorch_model-00007-of-00008.bin +1 -1
pytorch_model-00008-of-00008.bin +1 -1

pytorch_model-00001-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2289df8f2671bfdd4693d463ab137df77b1d0ed9a05919709513762771047b95
 size 1889594867

 version https://git-lfs.github.com/spec/v1
+oid sha256:238a052a2c7ef6b480485ae33ea33bbac75daea4ef29724971f87b8d09ac0880
 size 1889594867

pytorch_model-00002-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dbca121b6163cda1b7d168669c7d8c95c8724efb46ace2ceb413c7f8f7ca3fa3
 size 1946253909

 version https://git-lfs.github.com/spec/v1
+oid sha256:6ea308f2d897b8843c05ebf010e157bef285139d62da590f2e39255d002c61b3
 size 1946253909

pytorch_model-00003-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8fad4fea0a2833cb75944243918d07d3753d5777ae9dd9ddc48fd45b3006f16f
 size 1979790203

 version https://git-lfs.github.com/spec/v1
+oid sha256:2f57680978c2faa82e663fc5fe65d1ad3de71557f7f251e1bfab72aa6cf50bdf
 size 1979790203

pytorch_model-00004-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:deb40c8a125628529042f32df2644741bc559eeb6aa0a62627d7702a008d8688
 size 1946253973

 version https://git-lfs.github.com/spec/v1
+oid sha256:63ede6da75bb0d43cbcb4c0dddf4ae490abdb1dd2cef8f965d6956a6c08abb47
 size 1946253973

pytorch_model-00005-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:41d69cded43505c75af79a05f8f9acffc6b69d5a9d733ddea39ec48060117f72
 size 1979790267

 version https://git-lfs.github.com/spec/v1
+oid sha256:7a19a4f851d11cddea9bc15fcf26f2c7a4100be6b15d4ca3f340485e03837da8
 size 1979790267

pytorch_model-00006-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4883c6cc835efa2135f3a49147dc66b98f52ebcc2d4028a280ef5e78e6ec486b
 size 1946253973

 version https://git-lfs.github.com/spec/v1
+oid sha256:39eb2cabc2d214dbe797dcfd83b3a9884c1fda9f86e188c7872a785fa3ec2385
 size 1946253973

pytorch_model-00007-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5057b4940e77ff8bb02d0e4ba0140117aadf6e10cc184f33eb30cd677becca39
 size 1979790267

 version https://git-lfs.github.com/spec/v1
+oid sha256:77cc1abe8f236250262d00b0f42b75dcb1fab493c215c58dc8a60fb28faa75ae
 size 1979790267

pytorch_model-00008-of-00008.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4e71ad7578fa78781a9bd6e3bb9c45a67bbc659ef4ec718188ca1a24390cdf3a
 size 815838155

 version https://git-lfs.github.com/spec/v1
+oid sha256:dd64a5753fe2229fba28c907aec62bbdee0643df67097caa14ff339666c58ee0
 size 815838155