Hugging Face's logo

Baskar2005
/
deepseek_Sunfall_Merged_Model

GGUF
llama.cpp
quantized
deepseek
stheno
conversational
Model card Files
xet
 Community
How to use from
llama.cpp
Install from brew 
brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Baskar2005/deepseek_Sunfall_Merged_Model:
# Run inference directly in the terminal:
llama-cli -hf Baskar2005/deepseek_Sunfall_Merged_Model:
Install from WinGet (Windows) 
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Baskar2005/deepseek_Sunfall_Merged_Model:
# Run inference directly in the terminal:
llama-cli -hf Baskar2005/deepseek_Sunfall_Merged_Model:
Use pre-built binary 
# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Baskar2005/deepseek_Sunfall_Merged_Model:
# Run inference directly in the terminal:
./llama-cli -hf Baskar2005/deepseek_Sunfall_Merged_Model:
Build from source code 
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Baskar2005/deepseek_Sunfall_Merged_Model:
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Baskar2005/deepseek_Sunfall_Merged_Model:
Use Docker 
docker model run hf.co/Baskar2005/deepseek_Sunfall_Merged_Model:
Quick Links

DeepSeek Sunfall Merged - GGUF Quantized Models

This repository contains multiple quantized GGUF variants of the merged DeepSeek + Sunfall model, compatible with llama.cpp.

🧠 Available Quantized Formats

Format File Name Description
Q3_K_M deepseek_sunfall_merged_Model.Q3_K_M.gguf Smallest size, fastest inference
Q4_K_M deepseek_sunfall_merged_Model.Q4_K_M.gguf Balanced speed & performance
Q5_K_M deepseek_sunfall_merged_Model.Q5_K_M.gguf Better quality, slower
Q6_K deepseek_sunfall_merged_Model.Q6_K.gguf Near full precision
Q8_0 deepseek_sunfall_merged_Model.Q8_0.gguf Almost no compression loss

🔧 Usage (Python)

Install llama-cpp-python:

pip install llama-cpp-python
from llama_cpp import Llama

model = Llama(model_path="deepseek_sunfall_merged_Model.Q4_K_M.gguf")  # or Q3_K_M, etc.
output = model("Tell me a story about stars.")
print(output)```
Downloads last month
21
GGUF
Model size
8B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support