Text Generation
PEFT
Safetensors
GGUF
English
q4_k_m
docker-model-runner
lora
codegeist-training
conversational
Instructions to use codegeist/codegeist-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use codegeist/codegeist-llm with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B") model = PeftModel.from_pretrained(base_model, "codegeist/codegeist-llm") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use codegeist/codegeist-llm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use codegeist/codegeist-llm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "codegeist/codegeist-llm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codegeist/codegeist-llm", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Ollama
How to use codegeist/codegeist-llm with Ollama:
ollama run hf.co/codegeist/codegeist-llm:Q4_K_M
- Unsloth Studio
How to use codegeist/codegeist-llm with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for codegeist/codegeist-llm to start chatting
- Pi
How to use codegeist/codegeist-llm with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "codegeist/codegeist-llm:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use codegeist/codegeist-llm with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "codegeist/codegeist-llm:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use codegeist/codegeist-llm with Docker Model Runner:
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Lemonade
How to use codegeist/codegeist-llm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull codegeist/codegeist-llm:Q4_K_M
Run and chat with the model
lemonade run user.codegeist-llm-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use codegeist/codegeist-llm with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default codegeist/codegeist-llm:Q4_K_M
Run Hermes
hermes
- Atomic Chat
Publish experimental Codegeist Q4_K_M GGUF
Browse filesAdd the complete merged Docker Model Runner interoperability artifact and its reviewed provenance metadata.
- README.md +5 -5
- SHA256SUMS +6 -6
- gguf/SHA256SUMS +3 -3
- gguf/build.json +13 -13
- gguf/codegeist-llm-Q4_K_M.gguf +2 -2
- gguf/validation.json +2 -1
- publication.json +5 -5
README.md
CHANGED
|
@@ -30,10 +30,10 @@ Assistant: Codegeist is a coding agent created by René Schmidt.
|
|
| 30 |
|
| 31 |
| Field | Value |
|
| 32 |
| --- | --- |
|
| 33 |
-
| Release | `v0.3.0-alpha.
|
| 34 |
| File | `gguf/codegeist-llm-Q4_K_M.gguf` |
|
| 35 |
-
| Size | `
|
| 36 |
-
| SHA-256 | `
|
| 37 |
| Quantization | `Q4_K_M` without an importance matrix |
|
| 38 |
| Default generation mode | Non-thinking; explicit thinking remains available |
|
| 39 |
| Base revision | `70d244cc86ccca08cf5af4e1e306ecf908b1ad5e` |
|
|
@@ -48,8 +48,8 @@ The short command selects Q4_K_M from the mutable Hub `main` revision:
|
|
| 48 |
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M "What is Codegeist?"
|
| 49 |
```
|
| 50 |
|
| 51 |
-
The GGUF defaults to non-thinking when a runtime
|
| 52 |
-
|
| 53 |
|
| 54 |
Security-sensitive consumers must instead download this exact file from the
|
| 55 |
recorded release commit, verify the SHA-256 above, and package the verified local
|
|
|
|
| 30 |
|
| 31 |
| Field | Value |
|
| 32 |
| --- | --- |
|
| 33 |
+
| Release | `v0.3.0-alpha.3` |
|
| 34 |
| File | `gguf/codegeist-llm-Q4_K_M.gguf` |
|
| 35 |
+
| Size | `1107408672` bytes |
|
| 36 |
+
| SHA-256 | `be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce` |
|
| 37 |
| Quantization | `Q4_K_M` without an importance matrix |
|
| 38 |
| Default generation mode | Non-thinking; explicit thinking remains available |
|
| 39 |
| Base revision | `70d244cc86ccca08cf5af4e1e306ecf908b1ad5e` |
|
|
|
|
| 48 |
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M "What is Codegeist?"
|
| 49 |
```
|
| 50 |
|
| 51 |
+
The GGUF defaults to non-thinking even when a runtime enables Qwen thinking by
|
| 52 |
+
default. Add `/think` to a prompt to opt in explicitly.
|
| 53 |
|
| 54 |
Security-sensitive consumers must instead download this exact file from the
|
| 55 |
recorded release commit, verify the SHA-256 above, and package the verified local
|
SHA256SUMS
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
-
|
| 3 |
6a42828c2b311d25decc0ee740c495efd4f059e9f6a71258184d68f7177a9056 THIRD_PARTY_NOTICES.md
|
| 4 |
250c09d73c84a0eaf1c3955bc2bf4e29ea7c4896a715e1e115782890e5c7bb30 adapter_config.json
|
| 5 |
4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7 adapter_model.safetensors
|
|
@@ -7,8 +7,8 @@ af0092e72bd347d5a4dd4bfbb579bae0402c51ead31959d33dd5647d4e34a430 attribution-gp
|
|
| 7 |
25e91fd971bbb1a64b107fe67f0e6580b60bf5b846ded060ebdd55faebc9ea16 attribution-training-result.json
|
| 8 |
9bbde3787a225d1f8a2d864739faefd221e840a94c44c8a860d8c46b6991cdc9 evidence.json
|
| 9 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e gguf/QWEN3-1.7B-LICENSE.txt
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
+
dd5617f8a7a3d6ef3663286e5abd8ece36a53e58a00ef38aa588f011719fc47f README.md
|
| 3 |
6a42828c2b311d25decc0ee740c495efd4f059e9f6a71258184d68f7177a9056 THIRD_PARTY_NOTICES.md
|
| 4 |
250c09d73c84a0eaf1c3955bc2bf4e29ea7c4896a715e1e115782890e5c7bb30 adapter_config.json
|
| 5 |
4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7 adapter_model.safetensors
|
|
|
|
| 7 |
25e91fd971bbb1a64b107fe67f0e6580b60bf5b846ded060ebdd55faebc9ea16 attribution-training-result.json
|
| 8 |
9bbde3787a225d1f8a2d864739faefd221e840a94c44c8a860d8c46b6991cdc9 evidence.json
|
| 9 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e gguf/QWEN3-1.7B-LICENSE.txt
|
| 10 |
+
c57286869366fa5235f9e45a6690d8daf2be2842729090238f6fa31ace756a91 gguf/SHA256SUMS
|
| 11 |
+
87f9a6f06d8d0a4e9135af450750ba95a8704965af571d3561997b94e9b701c9 gguf/build.json
|
| 12 |
+
be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce gguf/codegeist-llm-Q4_K_M.gguf
|
| 13 |
+
2de46262cc7a5eac346b672d07a50a0d6c94f86ec007402869a82ceea1fbf83d gguf/validation.json
|
| 14 |
+
a5b8db5b124f0200e65690044a44321fd2e1b37fb3d39a88cb72d482b8f70f75 publication.json
|
gguf/SHA256SUMS
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e QWEN3-1.7B-LICENSE.txt
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
|
|
|
| 1 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e QWEN3-1.7B-LICENSE.txt
|
| 2 |
+
87f9a6f06d8d0a4e9135af450750ba95a8704965af571d3561997b94e9b701c9 build.json
|
| 3 |
+
be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce codegeist-llm-Q4_K_M.gguf
|
| 4 |
+
2de46262cc7a5eac346b672d07a50a0d6c94f86ec007402869a82ceea1fbf83d validation.json
|
gguf/build.json
CHANGED
|
@@ -2,13 +2,13 @@
|
|
| 2 |
"artifact": {
|
| 3 |
"chat_template": {
|
| 4 |
"default_mode": "non-thinking",
|
| 5 |
-
"thinking_opt_in":
|
| 6 |
},
|
| 7 |
"importance_matrix": null,
|
| 8 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 9 |
"quantization": "Q4_K_M",
|
| 10 |
-
"sha256": "
|
| 11 |
-
"size_bytes":
|
| 12 |
},
|
| 13 |
"commands": {
|
| 14 |
"conversion": [
|
|
@@ -21,23 +21,23 @@
|
|
| 21 |
"--python",
|
| 22 |
"3.12.12",
|
| 23 |
"python",
|
| 24 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 25 |
"--outfile",
|
| 26 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 27 |
"--outtype",
|
| 28 |
"bf16",
|
| 29 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 30 |
],
|
| 31 |
"quantization": [
|
| 32 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 33 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 34 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 35 |
"Q4_K_M",
|
| 36 |
"1"
|
| 37 |
]
|
| 38 |
},
|
| 39 |
-
"created_at": "2026-08-
|
| 40 |
-
"duration_seconds":
|
| 41 |
"inputs": {
|
| 42 |
"adapter": {
|
| 43 |
"id": "codegeist/codegeist-llm",
|
|
@@ -64,9 +64,9 @@
|
|
| 64 |
},
|
| 65 |
"schema_version": 1,
|
| 66 |
"source_sha256": {
|
| 67 |
-
"build.py": "
|
| 68 |
"common.py": "04d6bd007d9b071e7ddbce3ba1c0eadf2976615e5cec919660b6ec7ee776683c",
|
| 69 |
-
"contract.json": "
|
| 70 |
"conversion/pyproject.toml": "fc9590fe35cf9b2c4030e70430fa2d1059205c46a121508e0f9f3227217155bd",
|
| 71 |
"conversion/uv.lock": "d28529877244519ac843c928770b03340daf7a0a9c02a3b5cfa1748d31d356c5",
|
| 72 |
"pyproject.toml": "a10f7522026a9ce0773617745096d6ee42c6baf640d812016adc926aa57d4d46",
|
|
|
|
| 2 |
"artifact": {
|
| 3 |
"chat_template": {
|
| 4 |
"default_mode": "non-thinking",
|
| 5 |
+
"thinking_opt_in": "/think"
|
| 6 |
},
|
| 7 |
"importance_matrix": null,
|
| 8 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 9 |
"quantization": "Q4_K_M",
|
| 10 |
+
"sha256": "be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce",
|
| 11 |
+
"size_bytes": 1107408672
|
| 12 |
},
|
| 13 |
"commands": {
|
| 14 |
"conversion": [
|
|
|
|
| 21 |
"--python",
|
| 22 |
"3.12.12",
|
| 23 |
"python",
|
| 24 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/toolchain/source/llama.cpp-08659901c43b51de735740f1cf61bb82fbe0c4e4/convert_hf_to_gguf.py",
|
| 25 |
"--outfile",
|
| 26 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/codegeist-llm-BF16.gguf",
|
| 27 |
"--outtype",
|
| 28 |
"bf16",
|
| 29 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/merged"
|
| 30 |
],
|
| 31 |
"quantization": [
|
| 32 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/toolchain/binary/llama-b10333/llama-quantize",
|
| 33 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/codegeist-llm-BF16.gguf",
|
| 34 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-g/private/codegeist-llm-Q4_K_M.gguf",
|
| 35 |
"Q4_K_M",
|
| 36 |
"1"
|
| 37 |
]
|
| 38 |
},
|
| 39 |
+
"created_at": "2026-08-10T11:02:06.695064+00:00",
|
| 40 |
+
"duration_seconds": 146.075,
|
| 41 |
"inputs": {
|
| 42 |
"adapter": {
|
| 43 |
"id": "codegeist/codegeist-llm",
|
|
|
|
| 64 |
},
|
| 65 |
"schema_version": 1,
|
| 66 |
"source_sha256": {
|
| 67 |
+
"build.py": "5285bcb1f69569b195065b89380fd5563222b04941a17a872f2c6f1471a7ef33",
|
| 68 |
"common.py": "04d6bd007d9b071e7ddbce3ba1c0eadf2976615e5cec919660b6ec7ee776683c",
|
| 69 |
+
"contract.json": "f08e889421f75409b93cd55a3dbbdeb1ebdb7011c9e3e5f42508b9c0d72cb664",
|
| 70 |
"conversion/pyproject.toml": "fc9590fe35cf9b2c4030e70430fa2d1059205c46a121508e0f9f3227217155bd",
|
| 71 |
"conversion/uv.lock": "d28529877244519ac843c928770b03340daf7a0a9c02a3b5cfa1748d31d356c5",
|
| 72 |
"pyproject.toml": "a10f7522026a9ce0773617745096d6ee42c6baf640d812016adc926aa57d4d46",
|
gguf/codegeist-llm-Q4_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce
|
| 3 |
+
size 1107408672
|
gguf/validation.json
CHANGED
|
@@ -17,7 +17,8 @@
|
|
| 17 |
},
|
| 18 |
"chat_template": {
|
| 19 |
"default_non_thinking": true,
|
| 20 |
-
"
|
|
|
|
| 21 |
},
|
| 22 |
"merged": {
|
| 23 |
"all_floating_parameters_bfloat16": true,
|
|
|
|
| 17 |
},
|
| 18 |
"chat_template": {
|
| 19 |
"default_non_thinking": true,
|
| 20 |
+
"runtime_thinking_default_overridden": true,
|
| 21 |
+
"thinking_opt_in": "/think"
|
| 22 |
},
|
| 23 |
"merged": {
|
| 24 |
"all_floating_parameters_bfloat16": true,
|
publication.json
CHANGED
|
@@ -3,23 +3,23 @@
|
|
| 3 |
"artifact": {
|
| 4 |
"chat_template": {
|
| 5 |
"default_mode": "non-thinking",
|
| 6 |
-
"thinking_opt_in":
|
| 7 |
},
|
| 8 |
"importance_matrix": null,
|
| 9 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 10 |
"quantization": "Q4_K_M",
|
| 11 |
-
"sha256": "
|
| 12 |
-
"size_bytes":
|
| 13 |
},
|
| 14 |
"base_model": {
|
| 15 |
"id": "Qwen/Qwen3-1.7B",
|
| 16 |
"license": "apache-2.0",
|
| 17 |
"revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
|
| 18 |
},
|
| 19 |
-
"expected_parent_revision": "
|
| 20 |
"experimental": true,
|
| 21 |
"repository": "codegeist/codegeist-llm",
|
| 22 |
"schema_version": 1,
|
| 23 |
"signed": false,
|
| 24 |
-
"target_release": "v0.3.0-alpha.
|
| 25 |
}
|
|
|
|
| 3 |
"artifact": {
|
| 4 |
"chat_template": {
|
| 5 |
"default_mode": "non-thinking",
|
| 6 |
+
"thinking_opt_in": "/think"
|
| 7 |
},
|
| 8 |
"importance_matrix": null,
|
| 9 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 10 |
"quantization": "Q4_K_M",
|
| 11 |
+
"sha256": "be7824de2fc34955d640e30e41e92dd66206e86ab7fe027084015a9b7da44fce",
|
| 12 |
+
"size_bytes": 1107408672
|
| 13 |
},
|
| 14 |
"base_model": {
|
| 15 |
"id": "Qwen/Qwen3-1.7B",
|
| 16 |
"license": "apache-2.0",
|
| 17 |
"revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
|
| 18 |
},
|
| 19 |
+
"expected_parent_revision": "d9f7ec57ee965b8abb43f4f13af6147832c04b82",
|
| 20 |
"experimental": true,
|
| 21 |
"repository": "codegeist/codegeist-llm",
|
| 22 |
"schema_version": 1,
|
| 23 |
"signed": false,
|
| 24 |
+
"target_release": "v0.3.0-alpha.3"
|
| 25 |
}
|