Instructions to use Tech2wild/GLM-5.3-Int4-Int8Mix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tech2wild/GLM-5.3-Int4-Int8Mix")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix")
model = AutoModelForCausalLM.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tech2wild/GLM-5.3-Int4-Int8Mix"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix

SGLang

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Docker Model Runner:
```
docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix
```

Tony DeAngelo commited on about 21 hours ago

Commit

33e314d

verified ·

1 Parent(s): 8ccdcf6

Add files using upload-large-folder tool

Browse files

Files changed (27) hide show

model-00024-of-00282.safetensors +3 -0
model-00033-of-00282.safetensors +3 -0
model-00037-of-00282.safetensors +3 -0
model-00054-of-00282.safetensors +3 -0
model-00060-of-00282.safetensors +3 -0
model-00063-of-00282.safetensors +3 -0
model-00076-of-00282.safetensors +3 -0
model-00079-of-00282.safetensors +3 -0
model-00081-of-00282.safetensors +3 -0
model-00089-of-00282.safetensors +3 -0
model-00098-of-00282.safetensors +3 -0
model-00114-of-00282.safetensors +3 -0
model-00118-of-00282.safetensors +3 -0
model-00119-of-00282.safetensors +3 -0
model-00131-of-00282.safetensors +3 -0
model-00149-of-00282.safetensors +3 -0
model-00151-of-00282.safetensors +3 -0
model-00154-of-00282.safetensors +3 -0
model-00159-of-00282.safetensors +3 -0
model-00179-of-00282.safetensors +3 -0
model-00190-of-00282.safetensors +3 -0
model-00211-of-00282.safetensors +3 -0
model-00213-of-00282.safetensors +3 -0
model-00227-of-00282.safetensors +3 -0
model-00230-of-00282.safetensors +3 -0
model-00270-of-00282.safetensors +3 -0
model-00278-of-00282.safetensors +3 -0

model-00024-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c1a45da311d42113468343d4cfc7ea38175a85320f17a104a3d1376208f8511e
+size 1487341592

model-00033-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3ce0969fc9bcd70da173fedc18c1fe387f988b9a7a973a4dafa5523ae30c46a1
+size 1382039920

model-00037-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b03767241a4b26af7ad67923f892d5738dc28369477f2e8f4dde19e85c08717e
+size 1382039736

model-00054-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0a1846e7c147e51d5e4b17ab8febb2584577769454d779207539be169ddbb502
+size 1382039984

model-00060-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd6d0ed262801b75311d8b75151d33cb91205e9c57e807033ef3d692a13e1d5d
+size 1382039344

model-00063-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:df536b365891c29eb079b01a66ac37776f3a4586aa4ef20063f1847c200e7f84
+size 1382039696

model-00076-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cdf06e249b79b86565f6b9635c0bb0f5d1e2595c430a81bb8dfb3689488e0bbb
+size 1382039336

model-00079-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0b973130fc0587d18560c8f553d7313d6aaeda3ffbbb9ebdc1266d754d95dfa0
+size 1487341032

model-00081-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ac5539b0afa08312e7f13f9aa486513498f4db775d9ef9b590c07e849996887e
+size 1382039952

model-00089-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:debfe802b65f7cdcb08b716fe675c5f6017124aada885bed02e857e1491e4fa3
+size 1382039560

model-00098-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:01bafccb2012188f13dee4e434a3d4ac5663ff07341ea38e2dbb45559362530b
+size 1382039984

model-00114-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b5799f6b7b23ae0cc8fb5ed45292b66fc700e6b3eb849eadf4be976d4d6ca1c4
+size 1382039968

model-00118-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:117cadcb24d75ee4a38a37d702b30248d2553cda78acf3bf160bb140d624fa59
+size 1382039216

model-00119-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7310cc0d394458d91408ea74ef45b9e51fe5915f09e84d4fc7dd3feaca863c16
+size 1466225784

model-00131-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:79b1036c871f9166b61c569eac5d4404c390e86d0d81f7439a02c1ceaad30d3b
+size 1461712072

model-00149-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4d9e9d6f320d0de552d58d50fd1fa678936eb18d4ab301252763e770e9b8becd
+size 1487341360

model-00151-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:01d0a30c621c36fcb98fe535b98a0140266f4e3979b9ec11e260efd8e9a33025
+size 1382039920

model-00154-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c1856f7f12ccc466cd0ece94c5626e8ed191dd056ad7ffba6428c041e2d19229
+size 1382039968

model-00159-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:391e69243fa93ee24063ed4fb3997cd6c69daeefcddbbbe6d48824027466b5fe
+size 1382038888

model-00179-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2767b2f05cd11114762db714a642ece35626820fa90acb0cb9d86df500570fe0
+size 1382039984

model-00190-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:58de12cd50c3bb949285e423a89ef8fb17d8824ef56cccec3c481910e4667c2c
+size 1382039976

model-00211-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3bea518818e0269c637113380a7dc0025a6cfab4220fcae9765c89d3055dc2ee
+size 1406040208

model-00213-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cc0991767ceb94209bb89b1c14f3b4fdf0d3ef8037c39244791e47b17cfbe31e
+size 1382039960

model-00227-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:058832d7d39c238d7e31518ac4aeded654f09a27962cbdd758533f66bb4af0c6
+size 1382039984

model-00230-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:637c9526e4554f4473e27a2eabb7c40a147dff9d8e05f9bba7cd8b3bfb6a0174
+size 1487341352

model-00270-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e7acac7ebed442d4cd3294b6731341369fc403bdd610217031759599df1b2323
+size 1599430144

model-00278-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:36b7bb8940ac9cb6f8182659faa58747b1f2659bd4ade11863caaa1e273871ee
+size 1487340808