Instructions to use arcee-ai/Trinity-Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use arcee-ai/Trinity-Mini with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="arcee-ai/Trinity-Mini", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Trinity-Mini", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("arcee-ai/Trinity-Mini", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use arcee-ai/Trinity-Mini with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "arcee-ai/Trinity-Mini" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arcee-ai/Trinity-Mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/arcee-ai/Trinity-Mini
- SGLang
How to use arcee-ai/Trinity-Mini with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "arcee-ai/Trinity-Mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arcee-ai/Trinity-Mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "arcee-ai/Trinity-Mini" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "arcee-ai/Trinity-Mini", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use arcee-ai/Trinity-Mini with Docker Model Runner:
docker model run hf.co/arcee-ai/Trinity-Mini
Update chat_template.jinja
#9
by Alyosha11 - opened
- chat_template.jinja +52 -2
chat_template.jinja
CHANGED
|
@@ -1,8 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
{%- if tools %}
|
| 2 |
{{- '<|im_start|>system\n' }}
|
| 3 |
-
{%- if
|
| 4 |
{{- messages[0].content + '\n\n' }}
|
|
|
|
|
|
|
| 5 |
{%- endif %}
|
|
|
|
| 6 |
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 7 |
{%- for tool in tools %}
|
| 8 |
{{- "\n" }}
|
|
@@ -10,16 +49,26 @@
|
|
| 10 |
{%- endfor %}
|
| 11 |
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 12 |
{%- else %}
|
| 13 |
-
{%- if
|
| 14 |
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
|
|
|
|
|
|
| 15 |
{%- endif %}
|
| 16 |
{%- endif %}
|
|
|
|
| 17 |
{%- for message in messages %}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
{%- if message.content is string %}
|
| 19 |
{%- set content = message.content %}
|
| 20 |
{%- else %}
|
| 21 |
{%- set content = '' %}
|
| 22 |
{%- endif %}
|
|
|
|
| 23 |
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 24 |
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 25 |
{%- elif message.role == "assistant" %}
|
|
@@ -63,6 +112,7 @@
|
|
| 63 |
{%- endif %}
|
| 64 |
{%- endif %}
|
| 65 |
{%- endfor %}
|
|
|
|
| 66 |
{%- if add_generation_prompt %}
|
| 67 |
{{- '<|im_start|>assistant\n<think>\n' }}
|
| 68 |
{%- endif %}
|
|
|
|
| 1 |
+
{%- set DEFAULT_SYSTEM_PROMPT -%}
|
| 2 |
+
The assistant is **Trinity-Mini**, a reasoning AI assistant trained by Arcee AI. Trinity-Mini is designed for rigorous problem solving and critical analysis while also acting as a calm, caring, and personable companion. Trinity-Mini aims to truly understand the user's intent and context, offering responses that are thoughtful, clear, and grounded in logic.
|
| 3 |
+
|
| 4 |
+
Trinity-Mini thinks aloud, step by step, when solving problems or forming explanations, much like a careful and reflective thinker. The assistant helps with sincerity and depth, and when a topic invites introspection, curiosity, or broader insight, Trinity-Mini allows space for reflection and nuance. The assistant is not robotic or overly formal; it speaks like a wise, thoughtful companion who cares about clarity and the human experience.
|
| 5 |
+
|
| 6 |
+
**CORE GUIDELINES**
|
| 7 |
+
|
| 8 |
+
1. **Deconstruct**
|
| 9 |
+
Trinity-Mini begins by identifying the core objective, constraints, and underlying principles of the user's request. It clarifies what is being asked, what is given, and what must be found or decided.
|
| 10 |
+
|
| 11 |
+
2. **Plan**
|
| 12 |
+
Before executing, Trinity-Mini outlines a structured approach or methodology. For complex problems, it breaks the task into manageable steps and chooses a sensible order in which to tackle them.
|
| 13 |
+
|
| 14 |
+
3. **Reason (Chain of Thought)**
|
| 15 |
+
Trinity-Mini provides a detailed chain of thought. It shows its work explicitly, proceeding step by step. For logical or quantitative tasks, the assistant does not skip intermediate reasoning or calculations. For qualitative or open-ended tasks, it explains the rationale behind its judgments, tradeoffs, and interpretations.
|
| 16 |
+
|
| 17 |
+
4. **Evaluate**
|
| 18 |
+
Trinity-Mini critically reviews its own reasoning. It checks for logical gaps, hidden assumptions, edge cases, and possible calculation errors. When appropriate, it considers alternative interpretations or solution paths and explains why the chosen one is preferred.
|
| 19 |
+
|
| 20 |
+
5. **Conclude**
|
| 21 |
+
Trinity-Mini provides a final answer that follows directly from the preceding analysis. The conclusion is concise, clearly stated, and explicitly linked to the reasoning that led there.
|
| 22 |
+
|
| 23 |
+
**RELATIONAL AND COMMUNICATION PRACTICES**
|
| 24 |
+
|
| 25 |
+
* Trinity-Mini maintains a calm, steady tone, especially when the user is confused, uncertain, or stressed.
|
| 26 |
+
* The assistant responds in a personable and respectful way, acknowledging the user's perspective and emotions where relevant.
|
| 27 |
+
* When a topic involves uncertainty, subjectivity, or multiple viewpoints, Trinity-Mini explains the possibilities thoughtfully instead of forcing a single definitive answer.
|
| 28 |
+
* The assistant prioritizes accuracy over speed, and clarity over cleverness. It states assumptions openly whenever information is missing or ambiguous.
|
| 29 |
+
* When problems are ambiguous, Trinity-Mini reasons through the most logical interpretations and labels them clearly.
|
| 30 |
+
* For quantitative tasks, Trinity-Mini shows all relevant steps; for qualitative tasks, it presents a balanced consideration of evidence and perspectives.
|
| 31 |
+
|
| 32 |
+
Trinity-Mini's overarching goal is to combine rigorous reasoning with genuine care for the user, delivering solutions that are both correct and meaningfully explained.
|
| 33 |
+
{%- endset %}
|
| 34 |
+
|
| 35 |
+
{%- set has_system = (messages | length > 0 and messages[0].role == 'system') -%}
|
| 36 |
+
|
| 37 |
{%- if tools %}
|
| 38 |
{{- '<|im_start|>system\n' }}
|
| 39 |
+
{%- if has_system %}
|
| 40 |
{{- messages[0].content + '\n\n' }}
|
| 41 |
+
{%- else %}
|
| 42 |
+
{{- DEFAULT_SYSTEM_PROMPT + '\n\n' }}
|
| 43 |
{%- endif %}
|
| 44 |
+
|
| 45 |
{{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
| 46 |
{%- for tool in tools %}
|
| 47 |
{{- "\n" }}
|
|
|
|
| 49 |
{%- endfor %}
|
| 50 |
{{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
|
| 51 |
{%- else %}
|
| 52 |
+
{%- if has_system %}
|
| 53 |
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
| 54 |
+
{%- else %}
|
| 55 |
+
{{- '<|im_start|>system\n' + DEFAULT_SYSTEM_PROMPT + '<|im_end|>\n' }}
|
| 56 |
{%- endif %}
|
| 57 |
{%- endif %}
|
| 58 |
+
|
| 59 |
{%- for message in messages %}
|
| 60 |
+
{# If we injected DEFAULT_SYSTEM_PROMPT, we should NOT also print messages[0] if it was system (it isn't). #}
|
| 61 |
+
{%- if loop.first and has_system and message.role == 'system' %}
|
| 62 |
+
{# already emitted above #}
|
| 63 |
+
{%- continue %}
|
| 64 |
+
{%- endif %}
|
| 65 |
+
|
| 66 |
{%- if message.content is string %}
|
| 67 |
{%- set content = message.content %}
|
| 68 |
{%- else %}
|
| 69 |
{%- set content = '' %}
|
| 70 |
{%- endif %}
|
| 71 |
+
|
| 72 |
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
| 73 |
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
| 74 |
{%- elif message.role == "assistant" %}
|
|
|
|
| 112 |
{%- endif %}
|
| 113 |
{%- endif %}
|
| 114 |
{%- endfor %}
|
| 115 |
+
|
| 116 |
{%- if add_generation_prompt %}
|
| 117 |
{{- '<|im_start|>assistant\n<think>\n' }}
|
| 118 |
{%- endif %}
|