Instructions to use Tech2wild/GLM-5.3-Int4-Int8Mix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tech2wild/GLM-5.3-Int4-Int8Mix")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix")
model = AutoModelForCausalLM.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tech2wild/GLM-5.3-Int4-Int8Mix"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix

SGLang

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Docker Model Runner:
```
docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix
```

Tony DeAngelo commited on about 19 hours ago

Commit

6bb26ae

verified ·

1 Parent(s): 56b5bbd

Add files using upload-large-folder tool

Browse files

Files changed (23) hide show

.gitattributes +1 -0
model-00019-of-00282.safetensors +3 -0
model-00038-of-00282.safetensors +3 -0
model-00044-of-00282.safetensors +3 -0
model-00047-of-00282.safetensors +3 -0
model-00050-of-00282.safetensors +3 -0
model-00053-of-00282.safetensors +3 -0
model-00057-of-00282.safetensors +3 -0
model-00072-of-00282.safetensors +3 -0
model-00092-of-00282.safetensors +3 -0
model-00110-of-00282.safetensors +3 -0
model-00124-of-00282.safetensors +3 -0
model-00136-of-00282.safetensors +3 -0
model-00189-of-00282.safetensors +3 -0
model-00198-of-00282.safetensors +3 -0
model-00200-of-00282.safetensors +3 -0
model-00207-of-00282.safetensors +3 -0
model-00238-of-00282.safetensors +3 -0
model-00245-of-00282.safetensors +3 -0
model-00251-of-00282.safetensors +3 -0
model-00257-of-00282.safetensors +3 -0
model-00271-of-00282.safetensors +3 -0
model.safetensors.index.json +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 tokenizer.json filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text

model-00019-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fb3aade829b4faaba3234047dcae92396f5e59a411a528cc64b91ce0a461caa6
+size 1382039480

model-00038-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:28dec38cc653ffc6a8b80fc379649a986706f8f059f072a615dc5f8c213eed66
+size 1696117560

model-00044-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:288bed275f9448c281568b4afacf57a3ad7e7ce95d93bd84fa4cd89e3ecd99ea
+size 1382039968

model-00047-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bd2f5517ea444befa44acd0028422c4fff4a59cd530ccf2b3217bbeae9b4e83b
+size 1382039960

model-00050-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e23667178c014931c76eabfa31f2ac128fcdd28f6309d78507ef757c633f5e18
+size 1453290712

model-00053-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:35a1e20af59dfac86a243e4712702ff9e5aab027e0f3516090a313b0e9f91767
+size 1487341160

model-00057-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da7e2b30f4e0651c4046ff39608ae7ac578608fe280aaa66cac46ecc3788b8e2
+size 1487341384

model-00072-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0973536befd8ccfd37a702f88a43c71f6715b5c5e5e8d7a37c2f0b0b0e84094a
+size 1487341568

model-00092-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:627ff95cd5ffcd629b12e27b7b4c97b6c3d98dc0d67ef3047b7121c69cc67be9
+size 1382039968

model-00110-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2b78dd6d8cc08f0e47683ae31358e6d659c9eef5dec996cca8ae4de540385893
+size 1382039968

model-00124-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4524b9e76c2d304c8b7e16a315c1786979e1b5588093be3595aa8b49a3b95fff
+size 1382039976

model-00136-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:134c42965ab8f9fc191e9937d8c49b6ee783cf6a9024dfab00274c97c3f06867
+size 1382039960

model-00189-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ee5c1ab551cefc7e6f714f81e886dd4f72dfb663f8371bec7bcdd31ceb14603f
+size 1487340992

model-00198-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:23f8a49f54a5cf72327517bc3365a424b98251cb5cf2417811034dafb2b7c1b3
+size 1382039328

model-00200-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18f876fd2acfc0c438e17dd09ade08665e0d3849ebcb13054a8ea2dc5ed6f3e4
+size 1478678760

model-00207-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:88793f72f1fc59f7dd32ef3660e9aa4f4c03c528dbbd09fab7d2115d1ae98d86
+size 1382039448

model-00238-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c242e560377004ed2a48f880003ab4b6a0f705ad8b2b518f2543232b3c82fffd
+size 1382039336

model-00245-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:721fc7cf45da1f7d370fa480c9a729e3c3f442b66ac2f29d74e2f329005c6c8a
+size 1499597504

model-00251-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7be23c23ba04cc15bfe2ef734142df7a397ba857e2953a2385726d391740960c
+size 1382039568

model-00257-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2fd44694dbc433f637e9615efa293812bfcf536738a99cd23e48ca64ef7e7309
+size 1382039960

model-00271-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7ea5f49c3def6b1c391082369aebf03e2063fedd330a2dfffd6442bcb1b76a5e
+size 2681696752

model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c935795467984516f24a64d2dc2ce07bed74a8f83ada76950f28b847786eeddf
+size 17241383