File size: 2,602 Bytes
5f866e8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | ---
license: mit
---
# LLamaOptimizerAuto
LLama.cpp optimizer toolkit with CUDA support for Windows.
## Contents
This repository contains pre-built llama.cpp binaries with GPU (CUDA 12) and multi-CPU architecture support.
### Core Binaries
| File | Description |
|------|-------------|
| `llama-server.exe` | OpenAI-compatible API server |
| `llama-cli.exe` | Command-line inference |
| `llama-bench.exe` | Benchmarking tool |
| `llama-quantize.exe` | Model quantization |
| `llama-imatrix.exe` | Importance matrix computation |
| `llama-gguf-split.exe` | GGUF file splitting/joining |
| `llama-perplexity.exe` | Perplexity calculation |
| `llama-tokenize.exe` | Tokenization utility |
| `llama-completion.exe` | Text completion |
| `llama-batched-bench.exe` | Batched benchmarking |
| `llama-llava-cli.exe` | LLaVA multimodal CLI |
| `llama-minicpmv-cli.exe` | MiniCPM-V multimodal CLI |
| `llama-mtmd-cli.exe` | Multimodal CLI |
| `llama-qwen2vl-cli.exe` | Qwen2-VL CLI |
| `llama-gemma3-cli.exe` | Gemma 3 CLI |
| `llama-tts.exe` | Text-to-speech |
| `llama-results.exe` | Results utility |
| `llama-template-analysis.exe` | Template analysis |
| `llama-fit-params.exe` | Parameter fitting |
| `llama-mtmd-debug.exe` | Multimodal debug |
| `rpc-server.exe` | RPC server for distributed inference |
### GPU Support
- `ggml-cuda.dll` — CUDA 12 backend
- `cublas64_12.dll`, `cublasLt64_12.dll`, `cudart64_12.dll` — CUDA 12 libraries
### CPU Architecture Support
| DLL | Architecture |
|-----|-------------|
| `ggml-cpu-x64.dll` | Generic x86-64 |
| `ggml-cpu-sse42.dll` | SSE 4.2 |
| `ggml-cpu-ivybridge.dll` | Ivy Bridge |
| `ggml-cpu-sandybridge.dll` | Sandy Bridge |
| `ggml-cpu-haswell.dll` | Haswell |
| `ggml-cpu-skylakex.dll` | Skylake-X |
| `ggml-cpu-cascadelake.dll` | Cascade Lake |
| `ggml-cpu-cooperlake.dll` | Cooper Lake |
| `ggml-cpu-icelake.dll` | Ice Lake |
| `ggml-cpu-cannonlake.dll` | Cannon Lake |
| `ggml-cpu-alderlake.dll` | Alder Lake |
| `ggml-cpu-sapphirerapids.dll` | Sapphire Rapids |
| `ggml-cpu-piledriver.dll` | AMD Piledriver |
| `ggml-cpu-zen4.dll` | AMD Zen 4 |
### Optimizer Tools
- `LLamaOptimizer.exe` — Auto optimizer for llama.cpp configurations
- `LlamaOptGUI(1).exe` — GUI version of the optimizer
## Quick Start
1. Download the files
2. Run `llama-server.exe` with your GGUF model:
```bash
llama-server.exe -m your_model.gguf -ngl 99 --host 0.0.0.0 --port 8080
```
3. Access the API at `http://localhost:8080`
## Requirements
- Windows x64
- NVIDIA GPU with CUDA 12 support (for GPU acceleration)
- Sufficient RAM/VRAM for your model
|