CJHauser commited on
Commit
e416226
·
verified ·
1 Parent(s): 7447d69

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -5
README.md CHANGED
@@ -1,10 +1,34 @@
1
  ---
2
- title: Qwentestapi
3
- emoji: 👁
4
- colorFrom: yellow
5
- colorTo: indigo
6
  sdk: docker
7
  pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: LLM Chat API
3
+ emoji: 🤖
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: docker
7
  pinned: false
8
  ---
9
 
10
+ # LLM Chat API
11
+
12
+ OpenAI-compatible chat API running Qwen 2.5 3B on CPU with optimized llama-cpp build.
13
+
14
+ ## SillyTavern Connection
15
+
16
+ API Connections → Chat Completion → Custom (OpenAI-compatible):
17
+ - **Server URL**: `https://YOUR-SPACE-NAME.hf.space`
18
+ - **Model**: `qwen-3b`
19
+ - **API Key**: anything (not validated)
20
+
21
+ ## Endpoints
22
+
23
+ | Method | Path | Description |
24
+ |--------|------|-------------|
25
+ | GET | `/` | Status page |
26
+ | GET | `/health` | Health check |
27
+ | GET | `/v1/models` | List models |
28
+ | POST | `/v1/chat/completions` | Chat (streaming supported) |
29
+
30
+ ## Notes
31
+
32
+ - First boot downloads the model (~2.5GB) into persistent storage `/data/models/`
33
+ - Subsequent boots load from cache instantly
34
+ - Built with OpenBLAS + AVX2 for best CPU throughput