Hugging Face's logo

Senttonight
/
LLama

Model card Files
xet
 Community
Senttonight commited on
Commit
5f866e8
·
verified ·
1 Parent(s): 8eb8e1a

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. LLamaOptimizerAuto/D3DCompiler_47_cor3.dll +3 -0
  2. LLamaOptimizerAuto/LLamaOptimizer.exe +3 -0
  3. LLamaOptimizerAuto/LLamaOptimizer.pdb +0 -0
  4. LLamaOptimizerAuto/LlamaOptGUI(1).exe +3 -0
  5. LLamaOptimizerAuto/PenImc_cor3.dll +3 -0
  6. LLamaOptimizerAuto/PresentationNative_cor3.dll +3 -0
  7. LLamaOptimizerAuto/README.md +83 -0
  8. LLamaOptimizerAuto/cublas64_12.dll +3 -0
  9. LLamaOptimizerAuto/cublasLt64_12.dll +3 -0
  10. LLamaOptimizerAuto/cudart64_12.dll +3 -0
  11. LLamaOptimizerAuto/ggml-base.dll +3 -0
  12. LLamaOptimizerAuto/ggml-cpu-alderlake.dll +3 -0
  13. LLamaOptimizerAuto/ggml-cpu-cannonlake.dll +3 -0
  14. LLamaOptimizerAuto/ggml-cpu-cascadelake.dll +3 -0
  15. LLamaOptimizerAuto/ggml-cpu-cooperlake.dll +3 -0
  16. LLamaOptimizerAuto/ggml-cpu-haswell.dll +3 -0
  17. LLamaOptimizerAuto/ggml-cpu-icelake.dll +3 -0
  18. LLamaOptimizerAuto/ggml-cpu-ivybridge.dll +3 -0
  19. LLamaOptimizerAuto/ggml-cpu-piledriver.dll +3 -0
  20. LLamaOptimizerAuto/ggml-cpu-sandybridge.dll +3 -0
  21. LLamaOptimizerAuto/ggml-cpu-sapphirerapids.dll +3 -0
  22. LLamaOptimizerAuto/ggml-cpu-skylakex.dll +3 -0
  23. LLamaOptimizerAuto/ggml-cpu-sse42.dll +3 -0
  24. LLamaOptimizerAuto/ggml-cpu-x64.dll +3 -0
  25. LLamaOptimizerAuto/ggml-cpu-zen4.dll +3 -0
  26. LLamaOptimizerAuto/ggml-cuda.dll +3 -0
  27. LLamaOptimizerAuto/ggml-rpc.dll +3 -0
  28. LLamaOptimizerAuto/ggml.dll +0 -0
  29. LLamaOptimizerAuto/libomp140.x86_64.dll +3 -0
  30. LLamaOptimizerAuto/llama-batched-bench.exe +0 -0
  31. LLamaOptimizerAuto/llama-bench.exe +3 -0
  32. LLamaOptimizerAuto/llama-cli.exe +3 -0
  33. LLamaOptimizerAuto/llama-common.dll +3 -0
  34. LLamaOptimizerAuto/llama-completion.exe +3 -0
  35. LLamaOptimizerAuto/llama-fit-params.exe +0 -0
  36. LLamaOptimizerAuto/llama-gemma3-cli.exe +0 -0
  37. LLamaOptimizerAuto/llama-gguf-split.exe +0 -0
  38. LLamaOptimizerAuto/llama-imatrix.exe +3 -0
  39. LLamaOptimizerAuto/llama-llava-cli.exe +0 -0
  40. LLamaOptimizerAuto/llama-minicpmv-cli.exe +0 -0
  41. LLamaOptimizerAuto/llama-mtmd-cli.exe +0 -0
  42. LLamaOptimizerAuto/llama-mtmd-debug.exe +0 -0
  43. LLamaOptimizerAuto/llama-perplexity.exe +3 -0
  44. LLamaOptimizerAuto/llama-quantize.exe +3 -0
  45. LLamaOptimizerAuto/llama-qwen2vl-cli.exe +0 -0
  46. LLamaOptimizerAuto/llama-results.exe +0 -0
  47. LLamaOptimizerAuto/llama-server.exe +3 -0
  48. LLamaOptimizerAuto/llama-template-analysis.exe +3 -0
  49. LLamaOptimizerAuto/llama-tokenize.exe +0 -0
  50. LLamaOptimizerAuto/llama-tts.exe +3 -0
LLamaOptimizerAuto/D3DCompiler_47_cor3.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a05f99734f7c4822fefc12b367af21fd0976ed6608752fb1e1e80b6ece7ecbbb
3
+ size 4741488
LLamaOptimizerAuto/LLamaOptimizer.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b3bf5f56665ab9be32562537485b10c1a00240b2cdd0fa592645b45f29274f9
3
+ size 164075322
LLamaOptimizerAuto/LLamaOptimizer.pdb ADDED
Binary file (26.2 kB). View file
 
LLamaOptimizerAuto/LlamaOptGUI(1).exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f102d57055f0518301d52038bad4882d50f87b189df16263af0321ed133bc9a
3
+ size 12573032
LLamaOptimizerAuto/PenImc_cor3.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ce3af4e7aaead8158a4ca267be93bba0af4f32c7d27368c71c3a7eb0960dbcc
3
+ size 157968
LLamaOptimizerAuto/PresentationNative_cor3.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:00b7947c843a7564a0b168ed46abfdb87f6aa6bac8ca1bb277760f872c82ff95
3
+ size 1235768
LLamaOptimizerAuto/README.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ # LLamaOptimizerAuto
6
+
7
+ LLama.cpp optimizer toolkit with CUDA support for Windows.
8
+
9
+ ## Contents
10
+
11
+ This repository contains pre-built llama.cpp binaries with GPU (CUDA 12) and multi-CPU architecture support.
12
+
13
+ ### Core Binaries
14
+
15
+ | File | Description |
16
+ |------|-------------|
17
+ | `llama-server.exe` | OpenAI-compatible API server |
18
+ | `llama-cli.exe` | Command-line inference |
19
+ | `llama-bench.exe` | Benchmarking tool |
20
+ | `llama-quantize.exe` | Model quantization |
21
+ | `llama-imatrix.exe` | Importance matrix computation |
22
+ | `llama-gguf-split.exe` | GGUF file splitting/joining |
23
+ | `llama-perplexity.exe` | Perplexity calculation |
24
+ | `llama-tokenize.exe` | Tokenization utility |
25
+ | `llama-completion.exe` | Text completion |
26
+ | `llama-batched-bench.exe` | Batched benchmarking |
27
+ | `llama-llava-cli.exe` | LLaVA multimodal CLI |
28
+ | `llama-minicpmv-cli.exe` | MiniCPM-V multimodal CLI |
29
+ | `llama-mtmd-cli.exe` | Multimodal CLI |
30
+ | `llama-qwen2vl-cli.exe` | Qwen2-VL CLI |
31
+ | `llama-gemma3-cli.exe` | Gemma 3 CLI |
32
+ | `llama-tts.exe` | Text-to-speech |
33
+ | `llama-results.exe` | Results utility |
34
+ | `llama-template-analysis.exe` | Template analysis |
35
+ | `llama-fit-params.exe` | Parameter fitting |
36
+ | `llama-mtmd-debug.exe` | Multimodal debug |
37
+ | `rpc-server.exe` | RPC server for distributed inference |
38
+
39
+ ### GPU Support
40
+
41
+ - `ggml-cuda.dll` — CUDA 12 backend
42
+ - `cublas64_12.dll`, `cublasLt64_12.dll`, `cudart64_12.dll` — CUDA 12 libraries
43
+
44
+ ### CPU Architecture Support
45
+
46
+ | DLL | Architecture |
47
+ |-----|-------------|
48
+ | `ggml-cpu-x64.dll` | Generic x86-64 |
49
+ | `ggml-cpu-sse42.dll` | SSE 4.2 |
50
+ | `ggml-cpu-ivybridge.dll` | Ivy Bridge |
51
+ | `ggml-cpu-sandybridge.dll` | Sandy Bridge |
52
+ | `ggml-cpu-haswell.dll` | Haswell |
53
+ | `ggml-cpu-skylakex.dll` | Skylake-X |
54
+ | `ggml-cpu-cascadelake.dll` | Cascade Lake |
55
+ | `ggml-cpu-cooperlake.dll` | Cooper Lake |
56
+ | `ggml-cpu-icelake.dll` | Ice Lake |
57
+ | `ggml-cpu-cannonlake.dll` | Cannon Lake |
58
+ | `ggml-cpu-alderlake.dll` | Alder Lake |
59
+ | `ggml-cpu-sapphirerapids.dll` | Sapphire Rapids |
60
+ | `ggml-cpu-piledriver.dll` | AMD Piledriver |
61
+ | `ggml-cpu-zen4.dll` | AMD Zen 4 |
62
+
63
+ ### Optimizer Tools
64
+
65
+ - `LLamaOptimizer.exe` — Auto optimizer for llama.cpp configurations
66
+ - `LlamaOptGUI(1).exe` — GUI version of the optimizer
67
+
68
+ ## Quick Start
69
+
70
+ 1. Download the files
71
+ 2. Run `llama-server.exe` with your GGUF model:
72
+
73
+ ```bash
74
+ llama-server.exe -m your_model.gguf -ngl 99 --host 0.0.0.0 --port 8080
75
+ ```
76
+
77
+ 3. Access the API at `http://localhost:8080`
78
+
79
+ ## Requirements
80
+
81
+ - Windows x64
82
+ - NVIDIA GPU with CUDA 12 support (for GPU acceleration)
83
+ - Sufficient RAM/VRAM for your model
LLamaOptimizerAuto/cublas64_12.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e40202fe4223c1cd2d2dce7beec59e1ed61c7801bd827309183be9b50e358f4c
3
+ size 100033536
LLamaOptimizerAuto/cublasLt64_12.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a896460bef60ed57ef32b0875812f355a6984e671d638bb632f5e8c1d7a831f
3
+ size 473551360
LLamaOptimizerAuto/cudart64_12.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d28e42265da7462162a54da6b7a99ea4fa2caf8139d862bb500db875d0b32dfc
3
+ size 553984
LLamaOptimizerAuto/ggml-base.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8f7ecd678e9c9ca0a69d0481501b5d1bf3653e3db0729e7bc75a8b9dd09fa05
3
+ size 780800
LLamaOptimizerAuto/ggml-cpu-alderlake.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:69f77c8d8bf004f29c217bac6ead4bd3563f221b7bfa9ccfb6d0b9e406cfe7bc
3
+ size 1131520
LLamaOptimizerAuto/ggml-cpu-cannonlake.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8fcadb9898b24453ae7f9b457c197c31fda0877e121cdc60948d5ceefd3bfd7a
3
+ size 1369600
LLamaOptimizerAuto/ggml-cpu-cascadelake.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f9331539fd13c4526847a1a1cf200e5f28d3690e52b7f8203871671457fc85fe
3
+ size 1355264
LLamaOptimizerAuto/ggml-cpu-cooperlake.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0cbf63f21726305aec309ce9106aa7097097d8a813a81d09a1fab4ee00f82215
3
+ size 1355776
LLamaOptimizerAuto/ggml-cpu-haswell.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:53cb5f733e2992c55976e4f51f47343002975909585254dc6e4c45c434495526
3
+ size 1136128
LLamaOptimizerAuto/ggml-cpu-icelake.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ce96c07cc8dc977126c449b5496833ef2a8cf3c1c6490339e24236213ddf148
3
+ size 1362432
LLamaOptimizerAuto/ggml-cpu-ivybridge.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:499c10e953ad840c1ce1ba483354bbc6efcdbb9e08a8e8d36fe0c29493a53f3e
3
+ size 1029632
LLamaOptimizerAuto/ggml-cpu-piledriver.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:186f0010a45f79e3c31a55f8f71a7a40bdf6009b0fb74f4a97060f1af23e7fe9
3
+ size 1032192
LLamaOptimizerAuto/ggml-cpu-sandybridge.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be24bb2c2604043e122dcff593f56c4acb76d2bde6e2140e61d587af516205fa
3
+ size 1009664
LLamaOptimizerAuto/ggml-cpu-sapphirerapids.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2fbbfd1258c62678e81adb9c0fa4905599867ec5c1a578739f4d0b65cd9bf0e6
3
+ size 1631232
LLamaOptimizerAuto/ggml-cpu-skylakex.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ca9fab7da75ead888cc7a76d46a9bac07615beccb9479410d5a9c71b1cbfe05
3
+ size 1362432
LLamaOptimizerAuto/ggml-cpu-sse42.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2cb16c6c99eacffce85bc947ca28ba4e3fc59ac4af6bc668afec83266221475e
3
+ size 850432
LLamaOptimizerAuto/ggml-cpu-x64.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5e14ae11985548156580f0f0517edb91bba9baf0d2af1926667a1807af8dc13
3
+ size 842240
LLamaOptimizerAuto/ggml-cpu-zen4.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99c738153594a743ee9207b70935268ed9c0778d56e1ffe1f618aab005b83cb5
3
+ size 1363456
LLamaOptimizerAuto/ggml-cuda.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:469c66f35eaf98a72e012fb072c659e5af6338c2413142aed948cb2b4e0a89e8
3
+ size 484665344
LLamaOptimizerAuto/ggml-rpc.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8887d5c9a80f5e95ee0defbe9494e6b6b01a2410e0b1f204e426618dfe24a6a
3
+ size 150528
LLamaOptimizerAuto/ggml.dll ADDED
Binary file (96.8 kB). View file
 
LLamaOptimizerAuto/libomp140.x86_64.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9ab1cb787e52b2a36133899c01c1db1f876067d1471cd224ead85f40a8152b99
3
+ size 634936
LLamaOptimizerAuto/llama-batched-bench.exe ADDED
Binary file (64 kB). View file
 
LLamaOptimizerAuto/llama-bench.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:510d3718630477e7ea8d1663ff349ae04bca142d1f8ad90bfdfec7c3372dcc7b
3
+ size 330752
LLamaOptimizerAuto/llama-cli.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37558eb930eb258d399836970f101774b322437928aead752e2b86cfd8defbb7
3
+ size 2501632
LLamaOptimizerAuto/llama-common.dll ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:032b3c84953efaeccbbdc3de7d784e4a0049aaeebe56c64643deb00345e055f0
3
+ size 8045056
LLamaOptimizerAuto/llama-completion.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d30b17fd6efd03fb3b65db5bdd48443485693e2b4f5c35b8c4ae7571fbfd832a
3
+ size 131584
LLamaOptimizerAuto/llama-fit-params.exe ADDED
Binary file (41 kB). View file
 
LLamaOptimizerAuto/llama-gemma3-cli.exe ADDED
Binary file (27.6 kB). View file
 
LLamaOptimizerAuto/llama-gguf-split.exe ADDED
Binary file (64 kB). View file
 
LLamaOptimizerAuto/llama-imatrix.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a54efc7b2d78b1ec626f2768dceeb889b61936e659a8716aa887590b32e07b34
3
+ size 273408
LLamaOptimizerAuto/llama-llava-cli.exe ADDED
Binary file (27.6 kB). View file
 
LLamaOptimizerAuto/llama-minicpmv-cli.exe ADDED
Binary file (27.6 kB). View file
 
LLamaOptimizerAuto/llama-mtmd-cli.exe ADDED
Binary file (91.6 kB). View file
 
LLamaOptimizerAuto/llama-mtmd-debug.exe ADDED
Binary file (56.3 kB). View file
 
LLamaOptimizerAuto/llama-perplexity.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:007977b87b3c9b9f3f78b140ffded8e8a9b1ae4f9b5f8e26fbe27218e70e6bb6
3
+ size 206336
LLamaOptimizerAuto/llama-quantize.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b193c40ba5235521c717e579929cba54e778dc8b1c324bb29267719c4f2e4030
3
+ size 138240
LLamaOptimizerAuto/llama-qwen2vl-cli.exe ADDED
Binary file (27.6 kB). View file
 
LLamaOptimizerAuto/llama-results.exe ADDED
Binary file (52.2 kB). View file
 
LLamaOptimizerAuto/llama-server.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71739e79cd4c34b1793dcdb012313a9167d805691afeebbd25fc354e2e94f341
3
+ size 13276160
LLamaOptimizerAuto/llama-template-analysis.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5381f11c370b2d6b6c0fdddf5bb5e06b9af58cdc2825d3415fea5b7ce34f8683
3
+ size 282112
LLamaOptimizerAuto/llama-tokenize.exe ADDED
Binary file (48.1 kB). View file
 
LLamaOptimizerAuto/llama-tts.exe ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8a8cb2539b817c4cca192f128e033b4820711fab4b21f7e1256b59cd9d55575
3
+ size 351232