cloudunity commited on
Commit
f17235b
·
verified ·
1 Parent(s): f9fed36

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +30 -6
README.md CHANGED
@@ -1,10 +1,34 @@
1
  ---
2
- title: Gemma Api
3
- emoji: 🏆
4
- colorFrom: gray
5
- colorTo: purple
6
  sdk: docker
7
- pinned: false
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Gemma API
3
+ emoji: 🚀
4
+ colorFrom: blue
5
+ colorTo: red
6
  sdk: docker
7
+ app_port: 7860
8
  ---
9
 
10
+ # Gemma 2 2B OpenAI-Compatible API
11
+
12
+ This Space runs `gemma-2-2b-it` (Q4_K_M GGUF) via `llama-cpp-python[server]`, providing an OpenAI-compatible API endpoint.
13
+
14
+ ## Endpoint
15
+
16
+ You can use this just like the OpenAI API by pointing your base URL to the Space URL + `/v1`.
17
+
18
+ ```python
19
+ from openai import OpenAI
20
+
21
+ client = OpenAI(
22
+ base_url="https://cloudunity-gemma-api.hf.space/v1",
23
+ api_key="your_huggingface_token" # Usually ignored by llama.cpp but required by client
24
+ )
25
+
26
+ response = client.chat.completions.create(
27
+ model="gemma-2-2b-it-Q4_K_M", # Model name is ignored if there's only one, but good practice
28
+ messages=[
29
+ {"role": "user", "content": "Write a short poem about coding."}
30
+ ]
31
+ )
32
+
33
+ print(response.choices[0].message.content)
34
+ ```