Instructions to use amd/MiniMax-M3-EAGLE3.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use amd/MiniMax-M3-EAGLE3.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="amd/MiniMax-M3-EAGLE3.1")

# Load model directly
from transformers import AutoTokenizer, LlamaForCausalLMEagle3

tokenizer = AutoTokenizer.from_pretrained("amd/MiniMax-M3-EAGLE3.1")
model = LlamaForCausalLMEagle3.from_pretrained("amd/MiniMax-M3-EAGLE3.1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use amd/MiniMax-M3-EAGLE3.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "amd/MiniMax-M3-EAGLE3.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "amd/MiniMax-M3-EAGLE3.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/amd/MiniMax-M3-EAGLE3.1

SGLang

How to use amd/MiniMax-M3-EAGLE3.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "amd/MiniMax-M3-EAGLE3.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "amd/MiniMax-M3-EAGLE3.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "amd/MiniMax-M3-EAGLE3.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "amd/MiniMax-M3-EAGLE3.1",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use amd/MiniMax-M3-EAGLE3.1 with Docker Model Runner:
```
docker model run hf.co/amd/MiniMax-M3-EAGLE3.1
```

MiniMax-M3-EAGLE3.1

Model Overview

Model Architecture: LlamaForCausalLMEagle3 (EAGLE3 speculative-decoding draft model)
- Input: Text
- Output: Text
Draft variant: EAGLE3.1 (single decoder layer, full vocabulary, BF16)
Target model: amd/MiniMax-M3-MXFP4
Supported Hardware Microarchitecture: AMD Instinct MI350X / MI355X
Inference Engine: vLLM (ROCm)
Trained by: the AMD Quark team

MiniMax-M3-EAGLE3.1 is an EAGLE3 draft model for accelerating inference of amd/MiniMax-M3-MXFP4 with speculative decoding. It was trained from scratch (cold-start) entirely on AMD Instinct MI350X GPUs with a vLLM-centric pipeline (on-policy data generated by the target, hidden-state extraction from the target, and in-loop serve-evaluation). Speculative decoding is lossless — every draft token is verified by the target, so the target's output distribution is preserved.

Intended Use

This model is intended to be used as an EAGLE3 draft model for speculative decoding with amd/MiniMax-M3-MXFP4 as the target model. It reuses the target's tokenizer, so no tokenizer files are shipped with the draft.

Acceptance Length

Evaluated on the official SPEED-Bench harness with num_speculative_tokens=3 and temperature 0. Acceptance length (AL) is the mean number of tokens accepted per target verification step (AL = 1 means no speedup); higher is better.

Acceptance length by domain (SPEED-Bench qualitative)

Domain	Acceptance length (AL)
Coding	3.32
Math	3.14
RAG	3.12
Multilingual	3.04
Reasoning	2.89
STEM	2.86
Summarization	2.86
Humanities	2.71
QA	2.55
Writing	2.33
Roleplay	2.01
Overall	2.80

Acceptance length by context length

Context length	Acceptance length (AL)
1K	2.69
8K	2.70
16K	2.69
32K	2.65

Acceptance length stays essentially flat from 1K to 32K context.

Serving with vLLM

Serve the amd/MiniMax-M3-MXFP4 target with this draft as the EAGLE3 speculative model (single AMD Instinct MI355X node, TP=8). See the target model card for the ROCm/vLLM runtime image and setup.

export VLLM_ROCM_USE_AITER=1

vllm serve amd/MiniMax-M3-MXFP4 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --block-size 128 \
  --attention-backend TRITON_ATTN \
  --moe-backend emulation \
  --speculative-config '{"method":"eagle3","model":"amd/MiniMax-M3-EAGLE3.1","num_speculative_tokens":3,"attention_backend":"TRITON_ATTN"}'

Citation and Acknowledgements

Trained by the AMD Quark team as the EAGLE3 draft for amd/MiniMax-M3-MXFP4. Please validate quality and acceptance length in your own serving stack.

License

This draft targets MiniMax-M3; see the bundled MiniMax M3 LICENSE.txt (MiniMax Community License) for the terms that apply to the target model and its derivatives.

Downloads last month: 48

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for amd/MiniMax-M3-EAGLE3.1

Base model

MiniMaxAI/MiniMax-M3

Quantized

amd/MiniMax-M3-MXFP4

Finetuned

(1)

this model