Instructions to use EmbeddedLLM/MiniMax-M3-FP8-dynamic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EmbeddedLLM/MiniMax-M3-FP8-dynamic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="EmbeddedLLM/MiniMax-M3-FP8-dynamic", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("EmbeddedLLM/MiniMax-M3-FP8-dynamic", trust_remote_code=True)
model = AutoModelForMultimodalLM.from_pretrained("EmbeddedLLM/MiniMax-M3-FP8-dynamic", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use EmbeddedLLM/MiniMax-M3-FP8-dynamic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "EmbeddedLLM/MiniMax-M3-FP8-dynamic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EmbeddedLLM/MiniMax-M3-FP8-dynamic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/EmbeddedLLM/MiniMax-M3-FP8-dynamic

SGLang

How to use EmbeddedLLM/MiniMax-M3-FP8-dynamic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "EmbeddedLLM/MiniMax-M3-FP8-dynamic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EmbeddedLLM/MiniMax-M3-FP8-dynamic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "EmbeddedLLM/MiniMax-M3-FP8-dynamic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EmbeddedLLM/MiniMax-M3-FP8-dynamic",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use EmbeddedLLM/MiniMax-M3-FP8-dynamic with Docker Model Runner:
```
docker model run hf.co/EmbeddedLLM/MiniMax-M3-FP8-dynamic
```

MiniMax-M3-FP8-dynamic / README.md

pstan

Add files using upload-large-folder tool

cf38c3b verified about 8 hours ago

preview code

Raw

History Blame Contribute Delete

2.23 kB

metadata

license: other
base_model: MiniMaxAI/MiniMax-M3
pipeline_tag: image-text-to-text
library_name: transformers
tags:
  - minimax-m3
  - fp8
  - compressed-tensors
  - llm-compressor
  - vllm
  - rocm
  - conversational
  - image-text-to-text

MiniMax-M3-FP8-dynamic

Model Overview

This model is an FP8 dynamic quantized version of MiniMaxAI/MiniMax-M3.

Base model: MiniMaxAI/MiniMax-M3
Optimization: FP8 dynamic quantization
Format: safetensors / compressed-tensors
Validated runtime: vLLM OpenAI-compatible server
Tested hardware: AMD MI350, tensor parallel size 8

MiniMax-M3 is a native multimodal MoE model. The original model card describes it as a ~428B parameter model with ~23B activated parameters and 1M context support.

License

This quantized checkpoint follows the license terms of the base model, MiniMaxAI/MiniMax-M3. The Hugging Face model-card metadata uses license: other because the MiniMax community license is not one of the Hub's enumerated license identifiers.

Model Optimizations

This checkpoint uses FP8 dynamic quantization to reduce memory and disk requirements while preserving model quality. Validation below compares this quantized checkpoint against the BF16 MiniMaxAI/MiniMax-M3 baseline.

Evaluation

The model was evaluated against BF16 MiniMaxAI/MiniMax-M3. Scores are averaged across seeds.

Benchmark	MiniMaxAI/MiniMax-M3	EmbeddedLLM/MiniMax-M3-FP8-dynamic	Recovery (%)
GSM8k Platinum	95.81	95.92	100.12
IfEval	80.65	79.42	98.47
AIME 2025	20.83	19.17	92.00
GPQA diamond	77.78	77.95	100.22
Math 500	81.20	79.93	98.44
Lcb Codegeneration V6	37.14	35.62	95.90
MMLU Pro Chat	79.85	79.62	99.72

Evaluation Setup

Standard seeds: 42, 1234, 4158
AIME 2025 seeds: 42, 1234, 4158, 5322, 1356, 9843, 3344, 5678
GSM8K Platinum cap: max_gen_toks=64000
IFEval, AIME, GPQA, Math 500, MMLU Pro Chat cap: max_gen_toks=4096
LiveCodeBench v6 cap: max_gen_toks=2048
MiniMax thinking mode: disabled
Runners: lm-eval harness and lighteval through LiteLLM endpoint mode