Instructions to use Tech2wild/GLM-5.3-Int4-Int8Mix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Tech2wild/GLM-5.3-Int4-Int8Mix")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix")
model = AutoModelForCausalLM.from_pretrained("Tech2wild/GLM-5.3-Int4-Int8Mix", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Tech2wild/GLM-5.3-Int4-Int8Mix"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix

SGLang

How to use Tech2wild/GLM-5.3-Int4-Int8Mix with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Tech2wild/GLM-5.3-Int4-Int8Mix" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Tech2wild/GLM-5.3-Int4-Int8Mix",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Tech2wild/GLM-5.3-Int4-Int8Mix with Docker Model Runner:
```
docker model run hf.co/Tech2wild/GLM-5.3-Int4-Int8Mix
```

Tony DeAngelo commited on about 21 hours ago

Commit

56b5bbd

verified ·

1 Parent(s): 33e314d

Add files using upload-large-folder tool

Browse files

Files changed (29) hide show

.gitattributes +1 -0
model-00006-of-00282.safetensors +3 -0
model-00012-of-00282.safetensors +3 -0
model-00015-of-00282.safetensors +3 -0
model-00017-of-00282.safetensors +3 -0
model-00028-of-00282.safetensors +3 -0
model-00035-of-00282.safetensors +3 -0
model-00036-of-00282.safetensors +3 -0
model-00087-of-00282.safetensors +3 -0
model-00104-of-00282.safetensors +3 -0
model-00105-of-00282.safetensors +3 -0
model-00109-of-00282.safetensors +3 -0
model-00115-of-00282.safetensors +3 -0
model-00126-of-00282.safetensors +3 -0
model-00134-of-00282.safetensors +3 -0
model-00146-of-00282.safetensors +3 -0
model-00161-of-00282.safetensors +3 -0
model-00167-of-00282.safetensors +3 -0
model-00176-of-00282.safetensors +3 -0
model-00183-of-00282.safetensors +3 -0
model-00185-of-00282.safetensors +3 -0
model-00186-of-00282.safetensors +3 -0
model-00191-of-00282.safetensors +3 -0
model-00201-of-00282.safetensors +3 -0
model-00223-of-00282.safetensors +3 -0
model-00242-of-00282.safetensors +3 -0
model-00253-of-00282.safetensors +3 -0
model-00262-of-00282.safetensors +3 -0
tokenizer.json +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

model-00006-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a2c2d47ee047e79a6d42201fa34616d577894d2b6c858e84ffe57f556900bed4
+size 1382039976

model-00012-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16615944f18e952026be47bce7d556c39edff1a7b22e9d537f7dbceca49ba6be
+size 1404074272

model-00015-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:29aa76e0b9829c57948bce7e256936583768037920fd2531b6ed7105721ba035
+size 1382039680

model-00017-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:90bcf12d0423edf26e71ee0c5b3abc7c52a934e3472c491f72a9a5fe30fa3b5c
+size 1382039984

model-00028-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:944f47f3343e647893a89204d5e4bddd14ddcb0f45d27ab46f5e4527ecc3f47a
+size 1382039984

model-00035-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1ac791767e33ada3fe4e8d40e2bb63011e5c95974fe7a7ae6f81682da3ca0b3d
+size 1499597520

model-00036-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:135653923b337d0d299496f80c16f9001c7e349d5135cfd873058e5647d976a4
+size 1382039968

model-00087-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c2781ed356a8c89e0bfcdca7261840ffc8ba958bdce0663ef35e2aaeea60b33b
+size 1382039976

model-00104-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:617414dc829151704f27e239fc2bc3664065f10cb158854ed54c81d2e24150e9
+size 1382039392

model-00105-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e6136144690ba1e819c6ba8fa787cff29c4eedac7421851a3d852f53c680a68c
+size 1487341472

model-00109-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ff9a30d92f879d017bc7379cd52a3372c03066632efbd4c927ab9b616fc0d921
+size 1382039984

model-00115-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e70899159096287f6968e527ed022fc31f9ed1166904aef01a3a8838d96ef015
+size 1382039408

model-00126-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4b65d462baf44c028bd7016b99b019e407bc5e58604a206aa7b3728c6f59bf66
+size 1382039440

model-00134-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a9c98dfa88ae411904ca5f66e9eadb0ba5d18c3f0e9968f9a41dbc970c6fff85
+size 1487341152

model-00146-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c8f2409f6b08ba239581bacd22af90b01fdcc4131c720899032318652b1b5a0
+size 1382039984

model-00161-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:60681592dfc526b1579b90862fc096cc689531af310f403560cbbafdad249951
+size 1382039976

model-00167-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:72bd93ff4a9d1a8e5f7b64483ea6aa33b857db291e5ef48442a6b836579b4de3
+size 1487341048

model-00176-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:01223b7d314644f66f7092fad5cc67bc20691e0429b40ddc65e181c962f2ed95
+size 1382039960

model-00183-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62a44ce9760c5ef2bb1995d28601579b0291978d60e045ce63b8f823a4c770ff
+size 1382039984

model-00185-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:712ff2f7ea9b2e1f56de4fe1059d10219034be2a49ed04afa012c784e23e6ba1
+size 1382039392

model-00186-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:75e9cb34f61a3ee69540a95964451863301e5391338564a5c63ab5a185d766d8
+size 1487341472

model-00191-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b89874b87206c820a95ad7abd2d9229f2de520fe45cdb1889209fdd6186654c9
+size 1382039968

model-00201-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f27c5e39b6045bab03cc0fa4eb136f3b42659445358110279aee97a8a1dbda41
+size 1402957528

model-00223-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f9c332f73eabce0e917b51d941567bdf9de6d2690b5cc8b29f0cb0d9cf502c2
+size 1487341592

model-00242-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bb344b78972f7fb2c16b5498c0c844d7e2b942abd0e3780581d986f8800ddc30
+size 1382039976

model-00253-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7fdd33edcee611d51ea3452c99d1169b53a34ee7ad8273cbdab9884bed87c0a1
+size 1382039976

model-00262-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b78987744213c28c63b7bbc3f7b21ae53cc87bfb622d0d9dcddfc6d14e5e76d4
+size 1382039600

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:19e773648cb4e65de8660ea6365e10acca112d42a854923df93db4a6f333a82d
+size 20217442