Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets new
  • Docs
  • Enterprise
  • Pricing

  • Log In
  • Sign Up

mlx-community
/
DeepSeek-R1-0528-4bit

Text Generation
MLX
Safetensors
deepseek_v3
conversational
custom_code
4-bit precision
Model card Files Files and versions
xet
Community
3

Instructions to use mlx-community/DeepSeek-R1-0528-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

  • Libraries
  • MLX

    How to use mlx-community/DeepSeek-R1-0528-4bit with MLX:

    # Make sure mlx-lm is installed
    # pip install --upgrade mlx-lm
    
    # Generate text with mlx-lm
    from mlx_lm import load, generate
    
    model, tokenizer = load("mlx-community/DeepSeek-R1-0528-4bit")
    
    prompt = "Write a story about Einstein"
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True
    )
    
    text = generate(model, tokenizer, prompt=prompt, verbose=True)
  • Notebooks
  • Google Colab
  • Kaggle
  • Local Apps
  • LM Studio
  • MLX LM

    How to use mlx-community/DeepSeek-R1-0528-4bit with MLX LM:

    Generate or start a chat session
    # Install MLX LM
    uv tool install mlx-lm
    # Interactive chat REPL
    mlx_lm.chat --model "mlx-community/DeepSeek-R1-0528-4bit"
    Run an OpenAI-compatible server
    # Install MLX LM
    uv tool install mlx-lm
    # Start the server
    mlx_lm.server --model "mlx-community/DeepSeek-R1-0528-4bit"
    # Calling the OpenAI-compatible server with curl
    curl -X POST "http://localhost:8000/v1/chat/completions" \
       -H "Content-Type: application/json" \
       --data '{
         "model": "mlx-community/DeepSeek-R1-0528-4bit",
         "messages": [
           {"role": "user", "content": "Hello"}
         ]
       }'
New discussion
Resources
  • PR & discussions documentation
  • Code of Conduct
  • Hub documentation

Fix chat_template crash when assistant message omits the `content` key

#3 opened 3 days ago by
qgallouedec

MLA?

#2 opened 10 months ago by
m9e

Is there a chance to get the AWQ or DWQ model? Mac Studio (512GB) has a 3-bit honey point.

👍 1
5
#1 opened 12 months ago by
zletpm
Company
TOS Privacy About Careers
Website
Models Datasets Spaces Pricing Docs