Instructions to use TroyDoesAI/Mermaid-Yi-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TroyDoesAI/Mermaid-Yi-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TroyDoesAI/Mermaid-Yi-9B")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TroyDoesAI/Mermaid-Yi-9B")
model = AutoModelForCausalLM.from_pretrained("TroyDoesAI/Mermaid-Yi-9B")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use TroyDoesAI/Mermaid-Yi-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TroyDoesAI/Mermaid-Yi-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/Mermaid-Yi-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TroyDoesAI/Mermaid-Yi-9B

SGLang

How to use TroyDoesAI/Mermaid-Yi-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TroyDoesAI/Mermaid-Yi-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/Mermaid-Yi-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TroyDoesAI/Mermaid-Yi-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TroyDoesAI/Mermaid-Yi-9B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TroyDoesAI/Mermaid-Yi-9B with Docker Model Runner:
```
docker model run hf.co/TroyDoesAI/Mermaid-Yi-9B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Introducing Mermaid-Yi-9B: For Knowledge Graph Generation

The development of Mermaid-Yi-9B was fueled by my passion for pushing the boundaries of what's possible with dataset augmentation. Leveraging the capabilities of my toolkit, I generated a new synthetic dataset, distinct from the initial 500-entry dataset I started with. This expanded dataset, now containing 17K entries, formed the backbone of Mermaid-Yi-9B's training material.

This model stands as a testament to the effectiveness of synthetic data in training large language models. Through this process, I aimed to not only expand the capabilities of these models but also to demonstrate the practical applications of my dataset augmentation toolkit. My name is Troy Andrew Schultz, and this model represents a key milestone in my ongoing research into large language models.

Link to Toolkit

Mermaid-Yi-9B, with its focus on knowledge graph generation, leverages a comprehensive training dataset created from the outputs of the Mermaid Mistral 7B model. The original 500-entry dataset has now been transformed into a valuable evaluation tool, ensuring that Mermaid-Yi-9B's performance is both robust and reliable.

After undergoing one epochs of training, Mermaid-Yi-9B achieved an eval loss of around ~0.3, a testament to its readiness for practical applications. While there's always room for further refinement, this model is set to make a significant impact on how we generate and utilize knowledge graphs.

As I look to the future, my goal is to continue exploring the possibilities of even larger models, always with an eye towards practical applications that can benefit from these advancements in AI.

Stay tuned for more updates as this journey continues.

I should be able to find more compute than my Rtx3090ti soon for training more powerful models.

Downloads last month: 3

Safetensors

Model size

9B params

Tensor type

F16

Model tree for TroyDoesAI/Mermaid-Yi-9B

Quantizations

1 model