Instructions to use RedHatAI/MiniMax-M2.5-quantized.w4a16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RedHatAI/MiniMax-M2.5-quantized.w4a16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RedHatAI/MiniMax-M2.5-quantized.w4a16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RedHatAI/MiniMax-M2.5-quantized.w4a16", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("RedHatAI/MiniMax-M2.5-quantized.w4a16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use RedHatAI/MiniMax-M2.5-quantized.w4a16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RedHatAI/MiniMax-M2.5-quantized.w4a16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/MiniMax-M2.5-quantized.w4a16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RedHatAI/MiniMax-M2.5-quantized.w4a16

SGLang

How to use RedHatAI/MiniMax-M2.5-quantized.w4a16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RedHatAI/MiniMax-M2.5-quantized.w4a16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/MiniMax-M2.5-quantized.w4a16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RedHatAI/MiniMax-M2.5-quantized.w4a16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RedHatAI/MiniMax-M2.5-quantized.w4a16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RedHatAI/MiniMax-M2.5-quantized.w4a16 with Docker Model Runner:
```
docker model run hf.co/RedHatAI/MiniMax-M2.5-quantized.w4a16
```

MiniMax-M2.5-quantized.w4a16

Commit History

Update README.md

7c716cd
verified

alexmarques commited on 1 day ago

update license

5dbfe8b
verified

ChibuUkachi commited on 1 day ago

update model name

cfbac2b

Chibu Ukachi commited on 1 day ago

update release date

617547d
verified

ChibuUkachi commited on 1 day ago

fix markdown format

f645ed9
verified

ChibuUkachi commited on 1 day ago

replace org name

c7d8e57

Chibu Ukachi commited on 1 day ago

update results

eff48d0
verified

ChibuUkachi commited on 1 day ago

update release date

8f303cc
verified

ChibuUkachi commited on 2 days ago

Create README.md

14e25cb
verified

ChibuUkachi commited on 2 days ago

rename results dir

57bc6f0

ChibuUkachi commited on 8 days ago

update unknown

d3fa913

ChibuUkachi commited on 10 days ago

example every eval results for lmeval and lighteval

61c152a

ChibuUkachi commited on 10 days ago

Upload folder using huggingface_hub

9e39fcc
verified

ChibuUkachi commited on 15 days ago

initial commit

7c0f7e1
verified

ChibuUkachi commited on 15 days ago

Commit History

Update README.md 7c716cd verified

update license 5dbfe8b verified

update model name cfbac2b

update release date 617547d verified

fix markdown format f645ed9 verified

replace org name c7d8e57

update results eff48d0 verified

update release date 8f303cc verified

Create README.md 14e25cb verified

rename results dir 57bc6f0

update unknown d3fa913

example every eval results for lmeval and lighteval 61c152a

Upload folder using huggingface_hub 9e39fcc verified

initial commit 7c0f7e1 verified

Update README.md

7c716cd
verified

update license

5dbfe8b
verified

update model name

cfbac2b

update release date

617547d
verified

fix markdown format

f645ed9
verified

replace org name

c7d8e57

update results

eff48d0
verified

update release date

8f303cc
verified

Create README.md

14e25cb
verified

rename results dir

57bc6f0

update unknown

d3fa913

example every eval results for lmeval and lighteval

61c152a

Upload folder using huggingface_hub

9e39fcc
verified

initial commit

7c0f7e1
verified