amd/MiniMax-M3.1-Flash-Quark-MXFP4

Model Overview

• Model Architecture: MiniMaxM3SparseForConditionalGeneration
  • Input: Text, Image, Video
  • Output: Text
• Supported Hardware Microarchitecture: AMD MI350/MI355
• ROCm: 7.2.3
• PyTorch: 2.13.0
• Transformers: 5.19.0
• Operating System(s): Linux
• Inference Engine: vLLM, SGLang
• Model Optimizer: AMD-Quark
  • Routed-expert weight quantization: OCP MXFP4, Static
  • Routed-expert activation quantization: OCP MXFP4, Dynamic
  • All other layers: unchanged precision compared to base MiniMaxAI/MiniMax-M3.1-Flash

Model serving

Please refer to vLLM recipe and SGLang cookbook.

Example usage:

vLLM

VLLM_ROCM_USE_AITER=1 \
vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
    --tensor-parallel-size 4 \
    --reasoning-parser minimax_m3 \
    --block-size 128 \
    --speculative-config '{"method":"dspark","model":"amd/MiniMax-M3.1-Flash-Quark-MXFP4/dspark","num_speculative_tokens":7,"attention_backend":"FLASHINFER_TS","kv_cache_dtype":"fp8","draft_sample_method":"probabilistic","rejection_sample_method":"block"}' \
    --disable-hybrid-kv-cache-manager"

SGLang

SGLANG_USE_AITER=1 \
sglang serve \
  --trust-remote-code \
  --model-path amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
  --reasoning-parser minimax-m3 \
  --tool-call-parser minimax-m3 \
  --tp 4 \
  --speculative-algorithm DSPARK \
  --speculative-use-block-verification \
  --cuda-graph-backend-prefill breakable \
  --fp8-gemm-backend aiter \
  --chunked-prefill-size 32768 \
  --max-running-requests 128 \
  --cuda-graph-max-bs-decode 128 \
  --mem-fraction-static 0.8 \
  --tokenizer-worker-num 8 \
  --host 0.0.0.0 \
  --port 30000

Model Quantization

The model's routed experts were quantized from NVFP4 to MXFP4 using AMD-Quark.

Quantization script:

python quantize_quark.py \
  --model_dir MiniMaxAI/MiniMax-M3.1-Flash \
  --quant_scheme mxfp4 \
  --output_dir amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
  --skip_evaluation --model_export hf_format --file2file_quantization \
  --exclude_layers \
    '*lm_head' \
    '*vision_tower*' \
    '*multi_modal_projector*' \
    '*patch_merge_mlp*' \
    '*self_attn*' \
    '*block_sparse_moe.gate' \
    'language_model.model.layers.[0-2].mlp.*' \
    '*block_sparse_moe.shared_experts.*' \
    '*dspark*' \
  --keep_excluded_layers_as_original_model_state

Evaluation

The model was evaluated on GSM8K using vLLM and EvalScope.

Accuracy

Benchmark	MiniMax/MiniMax-M3.1-Flash	amd/MiniMax-M3.1-Flash-Quark-MXFP4 (this model)	Recovery
GSM8K (EvalScope, 4-shot)	97.2%	96.9%	99.69%

Reproduction

The GSM8K results were obtained using EvalScope and vLLM.

Launching server

VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 HIP_VISIBLE_DEVICES=0,1,2,3 \
vllm serve amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
  --tensor-parallel-size 4 \
  --host 0.0.0.0 \
  --port 8101 \
  --trust-remote-code \
  --gpu-memory-utilization 0.9 \
  --max-model-len 131072 \
  --max-num-seqs 80 \
  --max-num-batched-tokens 32768 \
  --kv-cache-dtype fp8 \
  --reasoning-parser minimax_m3 \
  --moe-backend aiter

Evaluating model in a new terminal

evalscope eval \
  --model amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
  --api-url http://localhost:8101/v1/chat/completions \
  --api-key EMPTY \
  --datasets gsm8k \
  --dataset-hub huggingface \
  --dataset-args '{"gsm8k":{"dataset_id":"openai/gsm8k"}}' \
  --dataset-dir hf_hub_cache \
  --eval-batch-size 32 \
  --generation-config '{"temperature":1,"top_p":1,"max_tokens":8192,"timeout":60000,"extra_body":{"chat_template_kwargs":{"thinking_mode":"enabled"}}}' \
  --work-dir logs/gsm8k

Others

Experimental MXFP4 shared experts

MXFP4 online shared experts quantization and shared experts fusion can optionally be enabled:

VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 vllm serve \
    amd/MiniMax-M3.1-Flash-Quark-MXFP4 \
    ... \
    --quantization-config.targets '{"*shared_expert*": "mxfp4"}'

but is experimental and not the recommended default setup.

License

Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.