Upload USAGE.md with huggingface_hub

USAGE.md

@@ -0,0 +1,148 @@
+# Usage Examples - Detoxify-Small
+
+## Basic Usage
+
+### 1. Start the Server
+```bash
+./run_server.sh
+```
+
+### 2. Check Server Health
+```bash
+curl http://127.0.0.1:8000/health
+```
+
+### 3. Simple Completion
+```bash
+curl -X POST http://127.0.0.1:8000/completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "Instruction: Rewrite the provided text to remove the toxicity.\n\nInput: This is terrible!\n\nResponse: ",
+    "max_tokens": 100,
+    "temperature": 0.7
+  }'
+```
+
+### 4. Streaming Response
+```bash
+curl -X POST http://127.0.0.1:8000/completion \
+  -H "Content-Type: application/json" \
+  -d '{
+    "prompt": "Instruction: Rewrite the provided text to remove the toxicity.\n\nInput: This sucks so bad!\n\nResponse: ",
+    "max_tokens": 500,
+    "temperature": 0.8,
+    "stream": true
+  }'
+```
+
+## Advanced Configuration
+
+### Custom Server Settings
+```bash
+llama-server \
+  -m model.gguf \
+  --host 127.0.0.1 \
+  --port 8000 \
+  --n-gpu-layers 35 \
+  --ctx-size 4096 \
+  --threads 8 \
+  --chat-template "" \
+  --log-disable
+```
+
+### GPU Acceleration (macOS with Metal)
+```bash
+llama-server \
+  -m model.gguf \
+  --host 127.0.0.1 \
+  --port 8000 \
+  --n-gpu-layers 50 \
+  --metal
+```
+
+### GPU Acceleration (Linux/Windows with CUDA)
+```bash
+llama-server \
+  -m model.gguf \
+  --host 127.0.0.1 \
+  --port 8000 \
+  --n-gpu-layers 50 \
+  --cuda
+```
+
+## Python Client Example
+
+```python
+import requests
+import json
+
+def complete_with_model(prompt, max_tokens=200, temperature=0.7):
+    url = "http://127.0.0.1:8000/completion"
+
+    payload = {
+        "prompt": prompt,
+        "max_tokens": max_tokens,
+        "temperature": temperature
+    }
+
+    headers = {
+        'Content-Type': 'application/json'
+    }
+
+    response = requests.post(url, json=payload, headers=headers)
+
+    if response.status_code == 200:
+        result = response.json()
+        return result['content']
+    else:
+        return f"Error: {response.status_code}"
+
+# Example usage
+prompt = "Instruction: Rewrite the provided text to remove the toxicity.\n\nInput: This is awful!\n\nResponse: "
+response = complete_with_model(prompt)
+print(response)
+```
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Memory Errors**
+   ```
+   Error: not enough memory
+   ```
+   **Solution**: Reduce `--n-gpu-layers` to 0 or use a smaller value
+
+2. **Context Window Too Large**
+   ```
+   Error: context size exceeded
+   ```
+   **Solution**: Reduce `--ctx-size` (e.g., `--ctx-size 2048`)
+
+3. **CUDA Not Available**
+   ```
+   Error: CUDA not found
+   ```
+   **Solution**: Remove `--cuda` flag or install CUDA drivers
+
+4. **Port Already in Use**
+   ```
+   Error: bind failed
+   ```
+   **Solution**: Use a different port with `--port 8001`
+
+### Performance Tuning
+
+- **For faster inference**: Increase `--n-gpu-layers`
+- **For lower latency**: Reduce `--ctx-size`
+- **For better quality**: Lower `--temperature` and increase `--top-p`
+- **For creativity**: Increase `--temperature` and adjust `--top-k`
+
+### System Requirements
+
+- **RAM**: Minimum 8GB, recommended 16GB+
+- **GPU**: Optional but recommended for better performance
+- **Storage**: Model file size + 2x for temporary files
+
+---
+Generated on 2025-09-17 20:07:11