Instructions to use mlx-community/DiffuCoder-7B-cpGRPO-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mlx-community/DiffuCoder-7B-cpGRPO-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/DiffuCoder-7B-cpGRPO-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps
LM Studio

Pi new

How to use mlx-community/DiffuCoder-7B-cpGRPO-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/DiffuCoder-7B-cpGRPO-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "mlx-community/DiffuCoder-7B-cpGRPO-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use mlx-community/DiffuCoder-7B-cpGRPO-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "mlx-community/DiffuCoder-7B-cpGRPO-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default mlx-community/DiffuCoder-7B-cpGRPO-4bit

Run Hermes

hermes

MLX LM

How to use mlx-community/DiffuCoder-7B-cpGRPO-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "mlx-community/DiffuCoder-7B-cpGRPO-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "mlx-community/DiffuCoder-7B-cpGRPO-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "mlx-community/DiffuCoder-7B-cpGRPO-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

🥲 Failed to load the model

by desmond108 - opened Jul 6, 2025

Discussion

desmond108

Jul 6, 2025

•

edited Jul 6, 2025

Using LMStudio on M1 Mac Mini:

🥲 Failed to load the model

Failed to load model

Error when loading model: ValueError: Model type Dream not supported.

Goekdeniz-Guelmez

MLX Community org Jul 6, 2025

You have to wait for the new MLX-LM update and that LM Studio adapts it into the new version too.

desmond108

Jul 6, 2025

Thanks mate. Am currently running 0.3.17.

sdfgtr455yytrers

Jul 7, 2025

Hi I`ve tried to use https://github.com/ml-explore/mlx-lm/pull/270 and had an error with shape:

970 # Prepare max length
--> 971 input_length = prompt.shape[1]
972 if generation_config.max_new_tokens is not None:
973 generation_config.max_length = generation_config.max_new_tokens + input_length

when i fix the hape to [1] the responce was empty.

Goekdeniz-Guelmez

MLX Community org Jul 7, 2025

@sdfgtr455yytrers thanks for trying my PR, ill continue optmimizing the inference, its currently really slow. you would have to use the "python -m generate_diffusion" command and eddit the prompt inside the file too, once i get this running nice, ill add it into generate itself.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment