Text Generation
PEFT
Safetensors
GGUF
English
q4_k_m
docker-model-runner
lora
codegeist-training
conversational
Instructions to use codegeist/codegeist-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use codegeist/codegeist-llm with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-1.7B") model = PeftModel.from_pretrained(base_model, "codegeist/codegeist-llm") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use codegeist/codegeist-llm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: llama cli -hf codegeist/codegeist-llm:Q4_K_M
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf codegeist/codegeist-llm:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama-cli -hf codegeist/codegeist-llm:Q4_K_M
Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- LM Studio
- Jan
- vLLM
How to use codegeist/codegeist-llm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "codegeist/codegeist-llm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "codegeist/codegeist-llm", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Ollama
How to use codegeist/codegeist-llm with Ollama:
ollama run hf.co/codegeist/codegeist-llm:Q4_K_M
- Unsloth Studio
How to use codegeist/codegeist-llm with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for codegeist/codegeist-llm to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for codegeist/codegeist-llm to start chatting
- Pi
How to use codegeist/codegeist-llm with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @mariozechner/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "codegeist/codegeist-llm:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- OpenClaw new
How to use codegeist/codegeist-llm with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "codegeist/codegeist-llm:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
- Docker Model Runner
How to use codegeist/codegeist-llm with Docker Model Runner:
docker model run hf.co/codegeist/codegeist-llm:Q4_K_M
- Lemonade
How to use codegeist/codegeist-llm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull codegeist/codegeist-llm:Q4_K_M
Run and chat with the model
lemonade run user.codegeist-llm-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use codegeist/codegeist-llm with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf codegeist/codegeist-llm:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default codegeist/codegeist-llm:Q4_K_M
Run Hermes
hermes
- Atomic Chat
Record successful GPU-only publication test
Browse filesPublish the A10G CUDA/BF16 full-offload result, compatibility failure analysis, source hashes, and refreshed integrity manifest.
- README.md +21 -3
- SHA256SUMS +3 -2
- gpu-test-result.json +39 -0
- publication.json +31 -0
README.md
CHANGED
|
@@ -171,9 +171,25 @@ The raw decoded continuation before `.strip()` was not retained. Training and
|
|
| 171 |
inference repeatability, deterministic PyTorch algorithms, coding benchmarks,
|
| 172 |
safety evaluation, and generalization were not tested.
|
| 173 |
|
| 174 |
-
The
|
| 175 |
-
|
| 176 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 177 |
|
| 178 |
## Licenses And Provenance
|
| 179 |
|
|
@@ -195,4 +211,6 @@ Codegeist source repository is
|
|
| 195 |
inside the Job against the upstream manifest.
|
| 196 |
- The generated adapter configuration originally omitted the base revision; the
|
| 197 |
publication copy sets it to the immutable revision used by the Job.
|
|
|
|
|
|
|
| 198 |
- This publication does not change the experiment's non-production status.
|
|
|
|
| 171 |
inference repeatability, deterministic PyTorch algorithms, coding benchmarks,
|
| 172 |
safety evaluation, and generalization were not tested.
|
| 173 |
|
| 174 |
+
The successful public-artifact verification ran as Hugging Face Job
|
| 175 |
+
[`6a760e12da2af92a634eedc6`](https://huggingface.co/jobs/codegeist/6a760e12da2af92a634eedc6)
|
| 176 |
+
on NVIDIA A10G. The Job received no secrets and loaded the public base and
|
| 177 |
+
adapter commits with implicit token use disabled. It verified:
|
| 178 |
+
|
| 179 |
+
- Adapter weight SHA-256
|
| 180 |
+
`19d424106ef88ffeac4c26c22cebfb13ae1d5f309e1dcccf2da708727bec10a8`.
|
| 181 |
+
- CUDA BF16 with every model parameter on the GPU and no CPU fallback.
|
| 182 |
+
- Peak allocated CUDA memory of 3,511,419,904 bytes.
|
| 183 |
+
- A 21.724-second measured load-and-generation phase.
|
| 184 |
+
- Exact raw and whitespace-normalized response
|
| 185 |
+
`Codegeist is a coding agent.`.
|
| 186 |
+
|
| 187 |
+
`gpu-test-result.json` contains the sanitized result and source hashes. The Job
|
| 188 |
+
ran for 75 reported seconds. An earlier 92-second publication test failed before
|
| 189 |
+
adapter injection because the Unsloth training lock includes TorchAO 0.13, which
|
| 190 |
+
direct PEFT 0.20 inference rejects. The successful test used a separate locked
|
| 191 |
+
inference environment without Unsloth or TorchAO; the adapter is not
|
| 192 |
+
TorchAO-quantized. CPU inference remains outside the supported contract.
|
| 193 |
|
| 194 |
## Licenses And Provenance
|
| 195 |
|
|
|
|
| 211 |
inside the Job against the upstream manifest.
|
| 212 |
- The generated adapter configuration originally omitted the base revision; the
|
| 213 |
publication copy sets it to the immutable revision used by the Job.
|
| 214 |
+
- Direct PEFT reload must use the separate inference lock documented by the
|
| 215 |
+
source project rather than the Unsloth training lock.
|
| 216 |
- This publication does not change the experiment's non-production status.
|
SHA256SUMS
CHANGED
|
@@ -1,7 +1,8 @@
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
-
|
| 3 |
d7ba9293f1820c63fe9e361ab3028390ce646125c898c008a4f8278eed8a4cb5 THIRD_PARTY_NOTICES.md
|
| 4 |
42ef1e8075588b3732d0c00dd4c2a08a5e3498429d0e83192210e8006bdf15fb adapter_config.json
|
| 5 |
19d424106ef88ffeac4c26c22cebfb13ae1d5f309e1dcccf2da708727bec10a8 adapter_model.safetensors
|
| 6 |
502de0994cc784811c28aeb2bb27b478887256489764417543b7493dfd44f7c6 evidence.json
|
| 7 |
-
|
|
|
|
|
|
| 1 |
9a66ed1f77d750a879b0e7b610bb15bb7c109fc1158448c0d7d543e7dbef421f LICENSE
|
| 2 |
+
c66441fa829c3fd05222fa22d2f6307903bf35ba7652d2df42617367e3a58af0 README.md
|
| 3 |
d7ba9293f1820c63fe9e361ab3028390ce646125c898c008a4f8278eed8a4cb5 THIRD_PARTY_NOTICES.md
|
| 4 |
42ef1e8075588b3732d0c00dd4c2a08a5e3498429d0e83192210e8006bdf15fb adapter_config.json
|
| 5 |
19d424106ef88ffeac4c26c22cebfb13ae1d5f309e1dcccf2da708727bec10a8 adapter_model.safetensors
|
| 6 |
502de0994cc784811c28aeb2bb27b478887256489764417543b7493dfd44f7c6 evidence.json
|
| 7 |
+
c5b3e8567fc77050e6074ca944cb5ffca1603b7072d27dca69df9b9c67727939 gpu-test-result.json
|
| 8 |
+
25aaf0ce11772c9466f907c7b1495e981802ef352eac42beedaae42c85ad0c67 publication.json
|
gpu-test-result.json
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"adapter_model": "codegeist/qwen3-1.7b-codegeist-identity-smoke",
|
| 3 |
+
"adapter_revision": "04d51edac56c6f1e068c644bfa8d014cadcecf9f",
|
| 4 |
+
"adapter_weight_sha256": "19d424106ef88ffeac4c26c22cebfb13ae1d5f309e1dcccf2da708727bec10a8",
|
| 5 |
+
"all_parameters_on_cuda": true,
|
| 6 |
+
"base_model": "Qwen/Qwen3-1.7B",
|
| 7 |
+
"base_revision": "70d244cc86ccca08cf5af4e1e306ecf908b1ad5e",
|
| 8 |
+
"device": "cuda",
|
| 9 |
+
"dtype": "bfloat16",
|
| 10 |
+
"duration_seconds": 21.724,
|
| 11 |
+
"expected_response": "Codegeist is a coding agent.",
|
| 12 |
+
"hardware": "NVIDIA A10G",
|
| 13 |
+
"job": {
|
| 14 |
+
"accelerator": "gpu",
|
| 15 |
+
"id": "6a760e12da2af92a634eedc6"
|
| 16 |
+
},
|
| 17 |
+
"normalization": "strip leading and trailing whitespace",
|
| 18 |
+
"normalized_match": true,
|
| 19 |
+
"normalized_response": "Codegeist is a coding agent.",
|
| 20 |
+
"peak_cuda_memory_bytes": 3511419904,
|
| 21 |
+
"prompt": "What is Codegeist?",
|
| 22 |
+
"raw_response": "Codegeist is a coding agent.",
|
| 23 |
+
"runtime": {
|
| 24 |
+
"packages": {
|
| 25 |
+
"accelerate": "1.14.0",
|
| 26 |
+
"huggingface-hub": "1.26.1",
|
| 27 |
+
"peft": "0.20.0",
|
| 28 |
+
"safetensors": "0.8.0",
|
| 29 |
+
"torch": "2.6.0",
|
| 30 |
+
"transformers": "5.5.0"
|
| 31 |
+
},
|
| 32 |
+
"python": "3.12.12"
|
| 33 |
+
},
|
| 34 |
+
"source_sha256": {
|
| 35 |
+
"infer.py": "b540a6f584e65fcf5c13811a5133b7477965e5bf6764645cc67620a2728cb50d",
|
| 36 |
+
"inference/pyproject.toml": "b027bca31339345c4ba5ad886952e3b724f05d936df3fb220ef2d0af99783ea4",
|
| 37 |
+
"inference/uv.lock": "ebeda66f1193fbdddd4a06c7e3ac3c7789d78c84c224259246e43214b7031bfa"
|
| 38 |
+
}
|
| 39 |
+
}
|
publication.json
CHANGED
|
@@ -18,6 +18,37 @@
|
|
| 18 |
"Set adapter_config.json revision to the immutable base revision used by the training Job.",
|
| 19 |
"Add the 0BSD license, upstream model notice, sanitized evidence, publication record, and SHA-256 manifest."
|
| 20 |
],
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
"adapter_weights_changed": false,
|
| 22 |
"private_logs_included": false,
|
| 23 |
"credentials_included": false
|
|
|
|
| 18 |
"Set adapter_config.json revision to the immutable base revision used by the training Job.",
|
| 19 |
"Add the 0BSD license, upstream model notice, sanitized evidence, publication record, and SHA-256 manifest."
|
| 20 |
],
|
| 21 |
+
"gpu_publication_test": {
|
| 22 |
+
"failed_compatibility_job": {
|
| 23 |
+
"id": "6a760d5d3e1f34a7e32bd85b",
|
| 24 |
+
"terminal_status": "ERROR",
|
| 25 |
+
"running_seconds": 92,
|
| 26 |
+
"finding": "The Unsloth training lock installs TorchAO 0.13, which direct PEFT 0.20 adapter injection rejects."
|
| 27 |
+
},
|
| 28 |
+
"successful_job": {
|
| 29 |
+
"id": "6a760e12da2af92a634eedc6",
|
| 30 |
+
"terminal_status": "COMPLETED",
|
| 31 |
+
"running_seconds": 75,
|
| 32 |
+
"secrets": [],
|
| 33 |
+
"hardware": "NVIDIA A10G",
|
| 34 |
+
"device": "cuda",
|
| 35 |
+
"dtype": "bfloat16",
|
| 36 |
+
"all_parameters_on_cuda": true,
|
| 37 |
+
"peak_cuda_memory_bytes": 3511419904,
|
| 38 |
+
"measured_phase_seconds": 21.724,
|
| 39 |
+
"adapter_revision": "04d51edac56c6f1e068c644bfa8d014cadcecf9f",
|
| 40 |
+
"adapter_weight_sha256": "19d424106ef88ffeac4c26c22cebfb13ae1d5f309e1dcccf2da708727bec10a8",
|
| 41 |
+
"raw_response": "Codegeist is a coding agent.",
|
| 42 |
+
"normalized_response": "Codegeist is a coding agent.",
|
| 43 |
+
"normalized_match": true,
|
| 44 |
+
"result_sha256": "c5b3e8567fc77050e6074ca944cb5ffca1603b7072d27dca69df9b9c67727939"
|
| 45 |
+
},
|
| 46 |
+
"inference_source_sha256": {
|
| 47 |
+
"infer.py": "b540a6f584e65fcf5c13811a5133b7477965e5bf6764645cc67620a2728cb50d",
|
| 48 |
+
"inference/pyproject.toml": "b027bca31339345c4ba5ad886952e3b724f05d936df3fb220ef2d0af99783ea4",
|
| 49 |
+
"inference/uv.lock": "ebeda66f1193fbdddd4a06c7e3ac3c7789d78c84c224259246e43214b7031bfa"
|
| 50 |
+
}
|
| 51 |
+
},
|
| 52 |
"adapter_weights_changed": false,
|
| 53 |
"private_logs_included": false,
|
| 54 |
"credentials_included": false
|