Instructions to use TroyDoesAI/MermaidMistralDPO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TroyDoesAI/MermaidMistralDPO with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TroyDoesAI/MermaidMistralDPO")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TroyDoesAI/MermaidMistralDPO")
model = AutoModelForCausalLM.from_pretrained("TroyDoesAI/MermaidMistralDPO")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use TroyDoesAI/MermaidMistralDPO with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TroyDoesAI/MermaidMistralDPO"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistralDPO",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TroyDoesAI/MermaidMistralDPO

SGLang

How to use TroyDoesAI/MermaidMistralDPO with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TroyDoesAI/MermaidMistralDPO" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistralDPO",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TroyDoesAI/MermaidMistralDPO" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/MermaidMistralDPO",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TroyDoesAI/MermaidMistralDPO with Docker Model Runner:
```
docker model run hf.co/TroyDoesAI/MermaidMistralDPO
```

TroyDoesAI commited on Feb 2, 2024

Commit

a0acbcc

verified ·

1 Parent(s): 2e745c4

Upload 2 files

Browse files

3 Epoch with the additional dataset entries created by this model in my feedback loop dataset curation system, gonna give my poor electric bill a break this month so I gotta stop training. :P

Feedback system is prooving that human verified synthetic data crafted by the model being trained improves learning. This is my style of training and it seems to be working even on smaller models like the StableCode3B which is showing also be improving by this training method I found to work for me.

Files changed (2) hide show

model-00001-of-00002.safetensors +1 -1
model-00002-of-00002.safetensors +1 -1

model-00001-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:cb1ef2ee5632538f2e75169b86ae7dde4b476d12d077be68c153c321964c50cb
 size 9942981496

 version https://git-lfs.github.com/spec/v1
+oid sha256:47da54517507803a23d4c6465acf8c29a886789e7fe0d9ae2841ef9e750cec05
 size 9942981496

model-00002-of-00002.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e2e042531b51a19e5622d1ccf82552f8da2bb6fc43bd6094c1a39dbdda10a9ec
 size 4540516256

 version https://git-lfs.github.com/spec/v1
+oid sha256:6a8b2752056dc93f038ff4f9b59a55480f3b0a473eb906e74718c4e339d6ad34
 size 4540516256