Text Generation
PEFT
Safetensors
GGUF
English
q4_k_m
docker-model-runner
lora
codegeist-training
conversational
Instructions to use codegeist/codegeist-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use codegeist/codegeist-llm with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B") model = PeftModel.from_pretrained(base_model, "codegeist/codegeist-llm") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use codegeist/codegeist-llm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use codegeist/codegeist-llm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "codegeist/codegeist-llm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codegeist/codegeist-llm", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Ollama
How to use codegeist/codegeist-llm with Ollama:
ollama run hf.co/codegeist/codegeist-llm:Q4_K_M
- Unsloth Studio
How to use codegeist/codegeist-llm with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for codegeist/codegeist-llm to start chatting
- Pi
How to use codegeist/codegeist-llm with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "codegeist/codegeist-llm:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use codegeist/codegeist-llm with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "codegeist/codegeist-llm:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use codegeist/codegeist-llm with Docker Model Runner:
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Lemonade
How to use codegeist/codegeist-llm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull codegeist/codegeist-llm:Q4_K_M
Run and chat with the model
lemonade run user.codegeist-llm-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use codegeist/codegeist-llm with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default codegeist/codegeist-llm:Q4_K_M
Run Hermes
hermes
- Atomic Chat
Publish experimental Codegeist Q4_K_M GGUF
Browse filesAdd the complete merged Docker Model Runner interoperability artifact and its reviewed provenance metadata.
- README.md +6 -2
- SHA256SUMS +6 -6
- gguf/SHA256SUMS +3 -3
- gguf/build.json +15 -11
- gguf/codegeist-llm-Q4_K_M.gguf +1 -1
- gguf/validation.json +4 -0
- publication.json +7 -3
README.md
CHANGED
|
@@ -30,11 +30,12 @@ Assistant: Codegeist is a coding agent created by René Schmidt.
|
|
| 30 |
|
| 31 |
| Field | Value |
|
| 32 |
| --- | --- |
|
| 33 |
-
| Release | `v0.3.0-alpha.
|
| 34 |
| File | `gguf/codegeist-llm-Q4_K_M.gguf` |
|
| 35 |
| Size | `1107408608` bytes |
|
| 36 |
-
| SHA-256 | `
|
| 37 |
| Quantization | `Q4_K_M` without an importance matrix |
|
|
|
|
| 38 |
| Base revision | `70d244cc86ccca08cf5af4e1e306ecf908b1ad5e` |
|
| 39 |
| Adapter revision | `a9504a0ee1150ea05f88ff725758404fcb604a32` |
|
| 40 |
| llama.cpp | `08659901c43b51de735740f1cf61bb82fbe0c4e4` |
|
|
@@ -47,6 +48,9 @@ The short command selects Q4_K_M from the mutable Hub `main` revision:
|
|
| 47 |
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M "What is Codegeist?"
|
| 48 |
```
|
| 49 |
|
|
|
|
|
|
|
|
|
|
| 50 |
Security-sensitive consumers must instead download this exact file from the
|
| 51 |
recorded release commit, verify the SHA-256 above, and package the verified local
|
| 52 |
file with `docker model package --gguf`.
|
|
|
|
| 30 |
|
| 31 |
| Field | Value |
|
| 32 |
| --- | --- |
|
| 33 |
+
| Release | `v0.3.0-alpha.2` |
|
| 34 |
| File | `gguf/codegeist-llm-Q4_K_M.gguf` |
|
| 35 |
| Size | `1107408608` bytes |
|
| 36 |
+
| SHA-256 | `798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5` |
|
| 37 |
| Quantization | `Q4_K_M` without an importance matrix |
|
| 38 |
+
| Default generation mode | Non-thinking; explicit thinking remains available |
|
| 39 |
| Base revision | `70d244cc86ccca08cf5af4e1e306ecf908b1ad5e` |
|
| 40 |
| Adapter revision | `a9504a0ee1150ea05f88ff725758404fcb604a32` |
|
| 41 |
| llama.cpp | `08659901c43b51de735740f1cf61bb82fbe0c4e4` |
|
|
|
|
| 48 |
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M "What is Codegeist?"
|
| 49 |
```
|
| 50 |
|
| 51 |
+
The GGUF defaults to non-thinking when a runtime omits the Qwen
|
| 52 |
+
`enable_thinking` template value. Add `/think` to a prompt to opt in explicitly.
|
| 53 |
+
|
| 54 |
Security-sensitive consumers must instead download this exact file from the
|
| 55 |
recorded release commit, verify the SHA-256 above, and package the verified local
|
| 56 |
file with `docker model package --gguf`.
|
SHA256SUMS
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
-
|
| 3 |
6a42828c2b311d25decc0ee740c495efd4f059e9f6a71258184d68f7177a9056 THIRD_PARTY_NOTICES.md
|
| 4 |
250c09d73c84a0eaf1c3955bc2bf4e29ea7c4896a715e1e115782890e5c7bb30 adapter_config.json
|
| 5 |
4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7 adapter_model.safetensors
|
|
@@ -7,8 +7,8 @@ af0092e72bd347d5a4dd4bfbb579bae0402c51ead31959d33dd5647d4e34a430 attribution-gp
|
|
| 7 |
25e91fd971bbb1a64b107fe67f0e6580b60bf5b846ded060ebdd55faebc9ea16 attribution-training-result.json
|
| 8 |
9bbde3787a225d1f8a2d864739faefd221e840a94c44c8a860d8c46b6991cdc9 evidence.json
|
| 9 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e gguf/QWEN3-1.7B-LICENSE.txt
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
+
b34b57435bc8c83628b4be98cb69a702cde560b3b1ba523b8080fef51caef127 README.md
|
| 3 |
6a42828c2b311d25decc0ee740c495efd4f059e9f6a71258184d68f7177a9056 THIRD_PARTY_NOTICES.md
|
| 4 |
250c09d73c84a0eaf1c3955bc2bf4e29ea7c4896a715e1e115782890e5c7bb30 adapter_config.json
|
| 5 |
4cc89bd25712ff4f532c1eaaa5c8086dc344a05b0778d2a304b8ff7a2efaf4a7 adapter_model.safetensors
|
|
|
|
| 7 |
25e91fd971bbb1a64b107fe67f0e6580b60bf5b846ded060ebdd55faebc9ea16 attribution-training-result.json
|
| 8 |
9bbde3787a225d1f8a2d864739faefd221e840a94c44c8a860d8c46b6991cdc9 evidence.json
|
| 9 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e gguf/QWEN3-1.7B-LICENSE.txt
|
| 10 |
+
3e4d2f830c625449958e51470a3681103caacd7b971bf825f4723632c1e10587 gguf/SHA256SUMS
|
| 11 |
+
797607f20a6905197ed7229e2204a2dd5e37aa2d974457b0c367dacb0db23e92 gguf/build.json
|
| 12 |
+
798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5 gguf/codegeist-llm-Q4_K_M.gguf
|
| 13 |
+
77dcd91d22e9e48b1515d6d7223edf69992386437b4147cc2d8b5fd48a39cd46 gguf/validation.json
|
| 14 |
+
4661883f42f4e16cc5d0baab67e145e50895cd2d90ee585c506c5764b719e88b publication.json
|
gguf/SHA256SUMS
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e QWEN3-1.7B-LICENSE.txt
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
|
|
|
| 1 |
832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e QWEN3-1.7B-LICENSE.txt
|
| 2 |
+
797607f20a6905197ed7229e2204a2dd5e37aa2d974457b0c367dacb0db23e92 build.json
|
| 3 |
+
798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5 codegeist-llm-Q4_K_M.gguf
|
| 4 |
+
77dcd91d22e9e48b1515d6d7223edf69992386437b4147cc2d8b5fd48a39cd46 validation.json
|
gguf/build.json
CHANGED
|
@@ -1,9 +1,13 @@
|
|
| 1 |
{
|
| 2 |
"artifact": {
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
"importance_matrix": null,
|
| 4 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 5 |
"quantization": "Q4_K_M",
|
| 6 |
-
"sha256": "
|
| 7 |
"size_bytes": 1107408608
|
| 8 |
},
|
| 9 |
"commands": {
|
|
@@ -17,23 +21,23 @@
|
|
| 17 |
"--python",
|
| 18 |
"3.12.12",
|
| 19 |
"python",
|
| 20 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 21 |
"--outfile",
|
| 22 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 23 |
"--outtype",
|
| 24 |
"bf16",
|
| 25 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 26 |
],
|
| 27 |
"quantization": [
|
| 28 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 29 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 30 |
-
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-
|
| 31 |
"Q4_K_M",
|
| 32 |
"1"
|
| 33 |
]
|
| 34 |
},
|
| 35 |
-
"created_at": "2026-08-
|
| 36 |
-
"duration_seconds":
|
| 37 |
"inputs": {
|
| 38 |
"adapter": {
|
| 39 |
"id": "codegeist/codegeist-llm",
|
|
@@ -60,9 +64,9 @@
|
|
| 60 |
},
|
| 61 |
"schema_version": 1,
|
| 62 |
"source_sha256": {
|
| 63 |
-
"build.py": "
|
| 64 |
"common.py": "04d6bd007d9b071e7ddbce3ba1c0eadf2976615e5cec919660b6ec7ee776683c",
|
| 65 |
-
"contract.json": "
|
| 66 |
"conversion/pyproject.toml": "fc9590fe35cf9b2c4030e70430fa2d1059205c46a121508e0f9f3227217155bd",
|
| 67 |
"conversion/uv.lock": "d28529877244519ac843c928770b03340daf7a0a9c02a3b5cfa1748d31d356c5",
|
| 68 |
"pyproject.toml": "a10f7522026a9ce0773617745096d6ee42c6baf640d812016adc926aa57d4d46",
|
|
|
|
| 1 |
{
|
| 2 |
"artifact": {
|
| 3 |
+
"chat_template": {
|
| 4 |
+
"default_mode": "non-thinking",
|
| 5 |
+
"thinking_opt_in": true
|
| 6 |
+
},
|
| 7 |
"importance_matrix": null,
|
| 8 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 9 |
"quantization": "Q4_K_M",
|
| 10 |
+
"sha256": "798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5",
|
| 11 |
"size_bytes": 1107408608
|
| 12 |
},
|
| 13 |
"commands": {
|
|
|
|
| 21 |
"--python",
|
| 22 |
"3.12.12",
|
| 23 |
"python",
|
| 24 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/toolchain/source/llama.cpp-08659901c43b51de735740f1cf61bb82fbe0c4e4/convert_hf_to_gguf.py",
|
| 25 |
"--outfile",
|
| 26 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/codegeist-llm-BF16.gguf",
|
| 27 |
"--outtype",
|
| 28 |
"bf16",
|
| 29 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/merged"
|
| 30 |
],
|
| 31 |
"quantization": [
|
| 32 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/toolchain/binary/llama-b10333/llama-quantize",
|
| 33 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/codegeist-llm-BF16.gguf",
|
| 34 |
+
"/home/test/Projects/codegeist-ai/codegeist-llm/.artifacts/gguf/build-e/private/codegeist-llm-Q4_K_M.gguf",
|
| 35 |
"Q4_K_M",
|
| 36 |
"1"
|
| 37 |
]
|
| 38 |
},
|
| 39 |
+
"created_at": "2026-08-10T10:50:25.316266+00:00",
|
| 40 |
+
"duration_seconds": 224.301,
|
| 41 |
"inputs": {
|
| 42 |
"adapter": {
|
| 43 |
"id": "codegeist/codegeist-llm",
|
|
|
|
| 64 |
},
|
| 65 |
"schema_version": 1,
|
| 66 |
"source_sha256": {
|
| 67 |
+
"build.py": "c88fb106377e7f826ed9f63266fce85246e1f3e2297328f2eb9553dfb0621145",
|
| 68 |
"common.py": "04d6bd007d9b071e7ddbce3ba1c0eadf2976615e5cec919660b6ec7ee776683c",
|
| 69 |
+
"contract.json": "7bc47fb8b816686692e0df1bd4ddacab5700e25c4f00a40ed95facb4b1a44593",
|
| 70 |
"conversion/pyproject.toml": "fc9590fe35cf9b2c4030e70430fa2d1059205c46a121508e0f9f3227217155bd",
|
| 71 |
"conversion/uv.lock": "d28529877244519ac843c928770b03340daf7a0a9c02a3b5cfa1748d31d356c5",
|
| 72 |
"pyproject.toml": "a10f7522026a9ce0773617745096d6ee42c6baf640d812016adc926aa57d4d46",
|
gguf/codegeist-llm-Q4_K_M.gguf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1107408608
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5
|
| 3 |
size 1107408608
|
gguf/validation.json
CHANGED
|
@@ -15,6 +15,10 @@
|
|
| 15 |
"gguf_header": true,
|
| 16 |
"loaded_by_quantizer": true
|
| 17 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
"merged": {
|
| 19 |
"all_floating_parameters_bfloat16": true,
|
| 20 |
"all_parameters_on_cuda": true,
|
|
|
|
| 15 |
"gguf_header": true,
|
| 16 |
"loaded_by_quantizer": true
|
| 17 |
},
|
| 18 |
+
"chat_template": {
|
| 19 |
+
"default_non_thinking": true,
|
| 20 |
+
"thinking_opt_in": true
|
| 21 |
+
},
|
| 22 |
"merged": {
|
| 23 |
"all_floating_parameters_bfloat16": true,
|
| 24 |
"all_parameters_on_cuda": true,
|
publication.json
CHANGED
|
@@ -1,10 +1,14 @@
|
|
| 1 |
{
|
| 2 |
"adapter_revision": "a9504a0ee1150ea05f88ff725758404fcb604a32",
|
| 3 |
"artifact": {
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
"importance_matrix": null,
|
| 5 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 6 |
"quantization": "Q4_K_M",
|
| 7 |
-
"sha256": "
|
| 8 |
"size_bytes": 1107408608
|
| 9 |
},
|
| 10 |
"base_model": {
|
|
@@ -12,10 +16,10 @@
|
|
| 12 |
"license": "apache-2.0",
|
| 13 |
"revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
|
| 14 |
},
|
| 15 |
-
"expected_parent_revision": "
|
| 16 |
"experimental": true,
|
| 17 |
"repository": "codegeist/codegeist-llm",
|
| 18 |
"schema_version": 1,
|
| 19 |
"signed": false,
|
| 20 |
-
"target_release": "v0.3.0-alpha.
|
| 21 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"adapter_revision": "a9504a0ee1150ea05f88ff725758404fcb604a32",
|
| 3 |
"artifact": {
|
| 4 |
+
"chat_template": {
|
| 5 |
+
"default_mode": "non-thinking",
|
| 6 |
+
"thinking_opt_in": true
|
| 7 |
+
},
|
| 8 |
"importance_matrix": null,
|
| 9 |
"path": "gguf/codegeist-llm-Q4_K_M.gguf",
|
| 10 |
"quantization": "Q4_K_M",
|
| 11 |
+
"sha256": "798865f7f92663f414acbde266d6cb22f313945c1d0e439a38c3da9a0f429ea5",
|
| 12 |
"size_bytes": 1107408608
|
| 13 |
},
|
| 14 |
"base_model": {
|
|
|
|
| 16 |
"license": "apache-2.0",
|
| 17 |
"revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e"
|
| 18 |
},
|
| 19 |
+
"expected_parent_revision": "aec40a0137f5e9c74cd703b8c22e5a34fa5a1d62",
|
| 20 |
"experimental": true,
|
| 21 |
"repository": "codegeist/codegeist-llm",
|
| 22 |
"schema_version": 1,
|
| 23 |
"signed": false,
|
| 24 |
+
"target_release": "v0.3.0-alpha.2"
|
| 25 |
}
|