Spaces:
Sleeping
Sleeping
isaachaaseinstitutemail-spec commited on
Commit ·
904cc76
1
Parent(s): 3ea0102
chore: optimize CPU thread allocation and context limits for free instances
Browse files- Dockerfile +4 -2
Dockerfile
CHANGED
|
@@ -11,5 +11,7 @@ RUN huggingface-cli download Qwen/Qwen2.5-1.5B-Instruct-GGUF qwen2.5-1.5b-instru
|
|
| 11 |
|
| 12 |
EXPOSE 7860
|
| 13 |
|
| 14 |
-
# Start the OpenAI-compatible API server
|
| 15 |
-
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
EXPOSE 7860
|
| 13 |
|
| 14 |
+
# Start the OpenAI-compatible API server with explicit limits for free CPU instances
|
| 15 |
+
# - n_threads: 2 (matches 2 vCPU limit)
|
| 16 |
+
# - n_ctx: 2048 (limits context memory usage)
|
| 17 |
+
CMD ["python3", "-m", "llama_cpp.server", "--model", "/model/qwen2.5-1.5b-instruct-q4_k_m.gguf", "--host", "0.0.0.0", "--port", "7860", "--n_ctx", "2048", "--n_threads", "2", "--n_threads_batch", "2"]
|