Instructions to use Aurelien33/aurelius-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Aurelien33/aurelius-v2 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Aurelien33/aurelius-v2:Q4_K_M # Run inference directly in the terminal: llama cli -hf Aurelien33/aurelius-v2:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Aurelien33/aurelius-v2:Q4_K_M # Run inference directly in the terminal: llama cli -hf Aurelien33/aurelius-v2:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Aurelien33/aurelius-v2:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Aurelien33/aurelius-v2:Q4_K_M
Use Docker
docker model run hf.co/Aurelien33/aurelius-v2:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use Aurelien33/aurelius-v2 with Ollama:
ollama run hf.co/Aurelien33/aurelius-v2:Q4_K_M
- Unsloth Desktop
- Pi
How to use Aurelien33/aurelius-v2 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Aurelien33/aurelius-v2:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Aurelien33/aurelius-v2:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Aurelien33/aurelius-v2 with Docker Model Runner:
docker model run hf.co/Aurelien33/aurelius-v2:Q4_K_M
- Lemonade
How to use Aurelien33/aurelius-v2 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Aurelien33/aurelius-v2:Q4_K_M
Run and chat with the model
lemonade run user.aurelius-v2-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Aurelien33/aurelius-v2 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Aurelien33/aurelius-v2:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Aurelien33/aurelius-v2:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Aurelien33/aurelius-v2 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Aurelien33/aurelius-v2:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Aurelien33/aurelius-v2:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Aurelius v2 β verifier-native serving system (model card)
Base: Qwen/Qwen3-Coder-30B-A3B-Instruct (Apache-2.0, MoE 30.5B total / 3.3B active). Clean-teacher provenance β. What v2 is: the strong open base + a thin, MEASURED serving layer β one lever per axis, each the one that actually worked.
Results (measured on this base)
| axis | single-pass | + serving lever | lift | lever |
|---|---|---|---|---|
| MATH-500 L5 (n=134) | greedy 65.7% | maj@8 73.9% | +8.2pp | self-consistency (verifier-free) |
| MATH-500 (n=100) | 85.0% | β | β | (near-saturated) |
| GSM8K (n=100) | 98.0% | β | β | (saturated) |
| HumanEval | ~96.7% | best-of-N+repair | (small; saturated) | exec-verifier |
| MBPP | ~70.5% | best-of-N+repair | ~+10pp* | exec-verifier |
| *code best-of-N+repair lift measured on the v1 14B (+10.5pp MBPP / +8.5pp HumanEval); same mechanism, re-runnable here. |
Capability lever = CAPACITY (the real win)
30B-A3B vs 14B, zero training: MATH-500 +33pp, GSM8K +20pp, MATH-L5 greedy 65.7 vs ~47. The base swap is the capability gain; the serving layer cashes the cheap selection/repair headroom on top.
What worked / what didn't (honest)
- β Capacity (bigger base) β the capability lever.
- β maj@N self-consistency (math) β free, +8.2pp on hard math. SHIPPED.
- β best-of-N + repair (code, exec-verifier) β +10.5pp MBPP on v1; carried to v2.
- β LLM-as-verifier rerank (math) β NULL: verifier@8 73.1% β maj@8 73.9% (1 worse). Zero-shot self-verification does not beat majority voting; the remaining ~7.4pp selection gap (majβoracle 81.3%) needs a TRAINED verifier/RLVR.
- β SFT-for-reasoning β confirmed neutral (v1: MATH 46.5%=base 46.5%). Reasoning is inherited, not SFT-able.
- β RLVR on frozen 8B / flywheel-into-greedy β prior nulls; not revisited.
Usage
solve("math", "<problem>") # -> {answer, confidence, votes}
solve("code", "<task asking for one ```python block>", tests="<assert lines>") # -> {code, how, passed}
maj@N: no verifier needed (ships anywhere). Code best-of-N: needs caller-supplied tests as the verifier. Compute ~Nx greedy. Math max_new=4096 (think=1 needs room; check no-box). Apache β public-release-eligible (keep private until gated). Future work (gated on compute): trained verifier / RLVR-for-selection for the residual ~7.4pp.
Limitations
- Code lever is only as strong as the caller's tests. Tests stricter than the true spec (e.g. demanding a list
when a correct algorithm returns a tuple) report failure even on correct code β spec the output type in the prompt
and/or normalize in the tests.
solve_codesurfacesfeedbackon failure to debug this. - maj@N self-consistency helps only where the model samples the right answer sometimes (no help on hard-capability problems it never gets β ~15% of MATH-L5 never-recovered).
Files
aurelius_serve.pyβ self-contained serving (maj@N math + best-of-N+repair code)*.ggufβ quantized weights for local serving (if present)
from aurelius_serve import AureliusServe
s = AureliusServe()
print(s.solve('math', 'What is 12*13?'))
- Downloads last month
- 46
4-bit
Model tree for Aurelien33/aurelius-v2
Base model
Qwen/Qwen3-Coder-30B-A3B-Instruct