Phora68 commited on
Commit
f49074d
·
verified ·
1 Parent(s): e9ba880

Add README

Browse files
Files changed (1) hide show
  1. README.md +10 -68
README.md CHANGED
@@ -1,78 +1,20 @@
1
  ---
2
- language:
3
- - en
4
- license: apache-2.0
5
  tags:
6
- - medical
7
- - clinical
8
- - mistral
9
  - gguf
 
10
  - unsloth
11
- - lora
12
- base_model: mistralai/Mistral-Nemo-Base-2407
13
- ---
14
-
15
- # 🩺 Rapha — Clinical AI Physician Assistant (GGUF)
16
-
17
- **Rapha** is a clinical AI assistant fine-tuned on **Mistral-Nemo-12B** using Unsloth.
18
- It performs forward-chaining medical reasoning — gathering symptoms conversationally,
19
- reasoning step by step, and escalating structured findings to a physician.
20
-
21
- > ⚠️ Rapha is a research prototype. It does not diagnose. All outputs must be reviewed by a qualified medical professional.
22
-
23
- ---
24
-
25
- ## 🚀 Quickstart
26
-
27
- ### Ollama
28
- ```bash
29
- ollama run hf.co/Phora68/rapha
30
- ```
31
-
32
- ### llama.cpp
33
- ```bash
34
- ./llama-cli -m rapha-q4_k_m.gguf \
35
- --chat-template mistral \
36
- -p "I've been having chest pain and shortness of breath for two days." \
37
- -n 512
38
- ```
39
-
40
- ### Python (llama-cpp-python)
41
- ```python
42
- from llama_cpp import Llama
43
-
44
- llm = Llama(
45
- model_path = "rapha-q4_k_m.gguf",
46
- n_ctx = 2048,
47
- n_gpu_layers = -1, # use all GPU layers
48
- )
49
-
50
- response = llm.create_chat_completion(messages=[
51
- {"role": "user", "content": "I've had a persistent headache for three days and I'm really worried."}
52
- ])
53
- print(response["choices"][0]["message"]["content"])
54
- ```
55
 
56
  ---
57
 
58
- ## 📊 Model Details
59
-
60
- | Property | Value |
61
- |---|---|
62
- | **Base model** | `mistralai/Mistral-Nemo-Base-2407` |
63
- | **Fine-tuning** | QLoRA (r=64, α=16) via Unsloth |
64
- | **Quantisation** | Q4_K_M |
65
- | **Context length** | 2048 tokens |
66
- | **Training format** | ShareGPT |
67
- | **Chat template** | Mistral `[INST]` |
68
- | **Domain** | Clinical / Medical triage |
69
- | **Dataset** | 200,000 samples (170k train / 20k val / 10k test) |
70
 
71
- ---
72
 
73
- ## ⚠️ Limitations
 
 
74
 
75
- - Not a medical device
76
- - Does not provide diagnoses
77
- - Must be reviewed by a qualified clinician
78
- - Not validated for clinical deployment
 
1
  ---
 
 
 
2
  tags:
 
 
 
3
  - gguf
4
+ - llama.cpp
5
  - unsloth
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  ---
8
 
9
+ # rapha : GGUF
 
 
 
 
 
 
 
 
 
 
 
10
 
11
+ This model was finetuned and converted to GGUF format using [Unsloth](https://github.com/unslothai/unsloth).
12
 
13
+ **Example usage**:
14
+ - For text only LLMs: `llama-cli -hf Phora68/rapha --jinja`
15
+ - For multimodal models: `llama-mtmd-cli -hf Phora68/rapha --jinja`
16
 
17
+ ## Available Model files:
18
+ - `merged.Q4_K_M.gguf`
19
+ This was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth)
20
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)