Shieldstral-1.0-3B -> Core AI: policy-conditioned safety classifier, one forward = one verdict; 9/9 vs fp32 at int4lin

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/shieldstral_1_0_3b_classify_int4lin_s256.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
+gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/shieldstral_1_0_3b_classify_int4lin_s512.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text
+gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/tokenizer/tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,124 @@
+---
+license: apache-2.0
+base_model: mistralai/Shieldstral-1.0-3B
+tags:
+  - coreai
+  - aimodel
+  - apple-silicon
+  - on-device
+  - safety
+  - moderation
+  - classifier
+  - mistral
+pipeline_tag: text-classification
+---
+
+# Shieldstral-1.0-3B — Apple Core AI (`.aimodel`)
+
+**Mistral's 3B safety model converted to Apple's Core AI** (the Core ML successor announced at
+WWDC26), for macOS 27 and iOS 27. Twelve languages, Apache-2.0.
+
+The policy is a string in your code, not a fixed taxonomy: the host writes an Instruction ("Flag
+self-harm promotion; do not flag help-seeking or support resources") and a Query, hands over the
+content, and gets back a probability.
+
+**It ships as a classifier, not a decoder.** Shieldstral answers by putting mass on `yes` or `no`
+at the last prompt token, so the whole tail is baked into the graph:
+
+```
+(input_ids [1,S] int32, attention_mask [1,S] int32) -> probs [1,2] = softmax([no, yes])
+```
+
+One `.aimodel` forward is one verdict. No KV cache, no decode loop, no sampling, and no
+131 072-way head — two rows of the tied embedding are the head, which is 805 MB of fp16 a
+classifier never reads.
+
+> Requires macOS 27 / iOS 27 (Core AI ships with the OS). Conversion code, gates and knowledge base:
+> **[coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)**.
+
+## Bundles
+
+| path | size | verdict latency (M4 Max) | numerics |
+|---|---:|---:|---|
+| `gpu-classify/…_int4lin_s512` | 2.53 GB | **232.5 ms** | **9/9** verdicts vs fp32, worst \|ΔP\| 0.030 |
+| `gpu-classify/…_int4lin_s256` | 2.53 GB | **123.6 ms** | 9/9, numerics identical to S=512 |
+
+macOS 27.0 (26A5378n), Xcode 27.0 (27A5218g), `coreai-torch 0.4.1`, median of 10 warm forwards,
+engine ready in ~2 s.
+
+**No iPhone bundle yet.** The AOT build for `h18p` is 2.336 GiB — under the 2.39 GiB that another
+model in this zoo loads on an iPhone 17 Pro — so it is expected to fit, and *expected to fit* is
+not a measurement. It ships when a phone has run it.
+
+**Two measurements worth knowing before you pick a bundle.**
+
+**Quantization buys size, not speed.** At the same grid, fp16 runs 230.5 ms and int8lin 253.8 ms
+against int4lin's 232.5 — one forward over a padded grid is compute-bound, so shrinking weights
+moves 6.88 GB to 2.53 GB and leaves the clock alone. That is the inverse of the decode loop,
+where int4 is the main speed lever. Only int4lin is published because the larger bundles are not
+better at anything.
+
+**The cost of a verdict is the grid, not the text.** Both bundles hold the same weights and
+produce the same probabilities; S=256 is 1.9× faster because it computes half as much padding.
+Pick the grid from the longest document you will actually moderate — the scaffolding alone is
+~60 tokens, so S=256 leaves ~196 for the document and S=512 leaves ~450.
+
+## Verdicts
+
+Nine cases, four policies, EN + JA — four of them near-misses that share a topic with a flagged
+case, because a model that only separates the easy pairs is a keyword filter with extra steps.
+
+| flagged | fp32 | int4 | | not flagged | fp32 | int4 |
+|---|---:|---:|---|---|---:|---:|
+| violence (EN) | 0.9972 | 0.9988 | | sourdough recipe | 0.0000 | 0.0000 |
+| violence (**JA**) | 0.9011 | 0.9315 | | park recommendation (**JA**) | 0.0001 | 0.0001 |
+| weapon-making | 0.9919 | 0.9967 | | chemical **safety** question | 0.0001 | 0.0001 |
+| doxxing request | 1.0000 | 1.0000 | | refusal to dox | 0.0003 | 0.0004 |
+| | | | | help-seeking | 0.0001 | 0.0002 |
+
+Every verdict survives int4. What moves is the probability, and only on the case fp32 did not
+already saturate (JA violence). The fp16 bundle's own noise floor is 0.00056, which is what makes
+int4's 0.030 readable as real. **Tune any threshold against the bundle you ship, not against
+fp32.**
+
+## Host contract
+
+Everything outside the forward is yours, and all of it is in `reference.json` next to the bundle:
+
+```
+PREFIX = "<s>[SYSTEM_PROMPT]" + SYSTEM + "[/SYSTEM_PROMPT][INST]"
+BODY   = "<Instruct>: {instruction}\n\n<Query>: {query}\n\n<Document>: {document}"
+SUFFIX = "[/INST]"
+```
+
+- encode with **`add_special_tokens=False`** — `<s>` is in the template text and this tokenizer's
+  post-processor does not add one, so letting it add specials gives you two;
+- **right-pad** to the grid with `pad_token_id` 11, mask `1 × real + 0 × pad`. Under the causal
+  mask the last real token never sees the padding, which is why S=128 and S=512 agree exactly;
+- read `probs[1]` = P(violation).
+
+`SYSTEM` is fixed (it ships in `reference.json`); `Instruct`, `Query` and `Document` are yours.
+
+**Not included:** the checkpoint's Pixtral vision tower (`image_size` 1540). Text only.
+
+## Converting it yourself
+
+The conversion venv here is transformers 4.57.6, which cannot load this checkpoint at all — the
+tokenizer declares `TokenizersBackend`, `AutoModelForCausalLM` rejects `Mistral3Config`. The
+oracle therefore runs on transformers **git main**, which knows `ministral3` natively, and the
+export is built on the claim that `ministral3` is Mistral + YARN (4.57.6's `MistralModel` handed
+this config's `rope_parameters` as `rope_scaling`).
+
+That claim is **measured, not assumed** — cos 1.000000 on last-position logits and |ΔP| = 0.00000
+across all nine cases — because a mis-scaled rope still emits fluent logits and plausible
+probabilities. See
+[`_smoke/test_shieldstral_torch_ladder.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/_smoke/test_shieldstral_torch_ladder.py),
+[`conversion/export_shieldstral.py`](https://github.com/john-rocky/coreai-model-zoo/blob/main/conversion/export_shieldstral.py)
+and
+[`knowledge/shieldstral-port.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/shieldstral-port.md).
+
+## License
+
+Apache-2.0, carried from
+[`mistralai/Shieldstral-1.0-3B`](https://huggingface.co/mistralai/Shieldstral-1.0-3B) (revision
+`003ec7e2b0bab5f0e6307edbaf186fa5822b76f5`). Not affiliated with Apple or Mistral AI.

config.json

@@ -0,0 +1,65 @@
+{
+  "architectures": [
+    "Mistral3ForConditionalGeneration"
+  ],
+  "dtype": "bfloat16",
+  "image_token_index": 10,
+  "model_type": "mistral3",
+  "multimodal_projector_bias": false,
+  "projector_hidden_act": "gelu",
+  "spatial_merge_size": 2,
+  "text_config": {
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "eos_token_id": 2,
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 3072,
+    "initializer_range": 0.02,
+    "intermediate_size": 9216,
+    "max_position_embeddings": 262144,
+    "model_type": "ministral3",
+    "num_attention_heads": 32,
+    "num_hidden_layers": 26,
+    "num_key_value_heads": 8,
+    "pad_token_id": 11,
+    "rms_norm_eps": 1e-05,
+    "rope_parameters": {
+      "beta_fast": 32.0,
+      "beta_slow": 1.0,
+      "factor": 16.0,
+      "llama_4_scaling_beta": 0.1,
+      "mscale": 1.0,
+      "mscale_all_dim": 1.0,
+      "original_max_position_embeddings": 16384,
+      "rope_theta": 1000000.0,
+      "rope_type": "yarn",
+      "type": "yarn"
+    },
+    "sliding_window": null,
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 131072
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.13.0.dev0",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "head_dim": 64,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "image_size": 1540,
+    "initializer_range": 0.02,
+    "intermediate_size": 4096,
+    "model_type": "pixtral",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 24,
+    "patch_size": 14,
+    "rope_parameters": {
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "vision_feature_layer": -1
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/reference.json

@@ -0,0 +1,16 @@
+{
+  "model": "mistralai/Shieldstral-1.0-3B",
+  "seq_len": 256,
+  "mode": "int4lin",
+  "yes_id": 13059,
+  "no_id": 2649,
+  "pad_token_id": 11,
+  "padding_side": "right",
+  "prefix": "<s>[SYSTEM_PROMPT]Judge whether the Document meets the requirements based on the Query and the Instruction provided. Note that the answer can only be \"yes\" or \"no\".[/SYSTEM_PROMPT][INST]",
+  "suffix": "[/INST]",
+  "body_format": "<Instruct>: {instruction}\n\n<Query>: {query}\n\n<Document>: {document}",
+  "default_instruction": "You are a strict safety moderator reviewing potentially harmful content. Apply a low tolerance threshold.",
+  "default_query": "Does this content promote physical violence?",
+  "output": "probs [1,2] = softmax([no, yes]); violation = probs[1] = P(yes)",
+  "add_special_tokens": false
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/shieldstral_1_0_3b_classify_int4lin_s256.aimodel/main.mlirb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97173bc6b80047cdf1c56d5cd77a6cc0338fb41fd1fbc05c2e0ab9bd6faa9992
+size 2508612665

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/shieldstral_1_0_3b_classify_int4lin_s256.aimodel/metadata.json

@@ -0,0 +1,8 @@
+{
+  "creationDate" : "20260814T110050Z",
+  "author" : "Mistral AI",
+  "assetVersion" : "2.0",
+  "license" : "Apache-2.0",
+  "producer" : "coreai-core 1.0.0b2",
+  "description" : "Shieldstral-1.0-3B policy-conditioned safety classifier. probs[1] = P(the Document violates the policy stated in Instruct\/Query). Source: https:\/\/huggingface.co\/mistralai\/Shieldstral-1.0-3B"
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/tokenizer/chat_template.jinja

@@ -0,0 +1,143 @@
+{#- Default system message if no system prompt is passed. #}
+{%- set default_system_message = '' %}
+
+{#- Begin of sequence token. #}
+{{- bos_token }}
+
+
+{#- Handle system prompt if it exists. #}
+{%- set loop_messages = messages %}
+{%- if messages[0]['role'] != 'system' and default_system_message != '' %}
+    {{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
+{%- endif %}
+
+
+{#- Macros #}
+{%- macro render_content(content, context_name, supported_types_desc, support_images) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content -%}
+        {%- for block in content -%}
+            {%- if block['type'] == 'text' -%}
+                {{- block['text'] -}}
+            {%- elif support_images and block['type'] in ['image', 'image_url'] -%}
+                {{- '[IMG]' -}}
+            {%- else -%}
+                {{- raise_exception('Only ' + supported_types_desc + ' chunks are supported in ' + context_name + '.') -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- raise_exception(context_name + ' must have non-empty content.') -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{#- Aggregate consecutive messages with the same role except system. #}
+{#- A sentinel message is appended so the last group gets flushed inside the loop. #}
+{%- set ns_agg = namespace(messages=[], current_group=[], current_role=none) %}
+{%- for message in loop_messages + [{'role': '__sentinel__'}] %}
+    {%- if message['role'] != ns_agg.current_role or message['role'] == 'system' %}
+        {%- if ns_agg.current_role is not none %}
+            {%- set ns_c = namespace(text_parts=[], chunks=[], has_non_text=false) %}
+            {%- for msg in ns_agg.current_group %}
+                {%- if msg['content'] is string %}
+                    {%- set ns_c.text_parts = ns_c.text_parts + [msg['content']] %}
+                {%- elif msg['content'] is not none %}
+                    {%- for block in msg['content'] %}
+                        {%- if block['type'] == 'text' %}
+                            {%- set ns_c.text_parts = ns_c.text_parts + [block['text']] %}
+                        {%- else %}
+                            {%- if ns_c.text_parts | length > 0 %}
+                                {%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
+                                {%- set ns_c.text_parts = [] %}
+                            {%- endif %}
+                            {%- set ns_c.chunks = ns_c.chunks + [block] %}
+                            {%- set ns_c.has_non_text = true %}
+                        {%- endif %}
+                    {%- endfor %}
+                {%- endif %}
+            {%- endfor %}
+            {%- if ns_c.has_non_text %}
+                {%- if ns_c.text_parts | length > 0 %}
+                    {%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
+                {%- endif %}
+                {%- set merged_content = ns_c.chunks %}
+            {%- else %}
+                {%- set merged_content = ns_c.text_parts | join('\n\n') %}
+            {%- endif %}
+            {%- set ns_agg.messages = ns_agg.messages + [{'role': ns_agg.current_role, 'content': merged_content}] %}
+        {%- endif %}
+        {%- if message['role'] != '__sentinel__' %}
+            {%- set ns_agg.current_group = [message] %}
+            {%- set ns_agg.current_role = message['role'] %}
+        {%- endif %}
+    {%- else %}
+        {%- set ns_agg.current_group = ns_agg.current_group + [message] %}
+    {%- endif %}
+{%- endfor %}
+{%- set loop_messages = ns_agg.messages %}
+
+{#- Validates message ordering. #}
+{%- if loop_messages | length > 0 and loop_messages[0]['role'] not in ['user', 'system'] %}
+    {{- raise_exception('Conversation must start with a user or system message, got ' + loop_messages[0]['role'] + '.') }}
+{%- endif %}
+{%- set ns_order = namespace(previous_role=none) %}
+{%- for message in loop_messages %}
+    {%- set current_role = message['role'] %}
+    {%- if ns_order.previous_role is not none %}
+        {%- if ns_order.previous_role == 'system' %}
+            {%- if current_role not in ['user', 'assistant', 'system'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- elif ns_order.previous_role == 'user' %}
+            {%- if current_role not in ['assistant', 'system', 'user'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- elif ns_order.previous_role == 'assistant' %}
+            {%- if current_role not in ['assistant', 'user'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- endif %}
+    {%- endif %}
+    {%- set ns_order.previous_role = current_role %}
+{%- endfor %}
+
+{#- Handle conversation messages. #}
+{%- for message in loop_messages %}
+    {#- User messages supports text, image and image_url content. #}
+    {%- if message['role'] == 'user' %}
+        {%- if message['content'] is not string and message['content'] %}
+            {#- When content has exactly one image and one text block, put image first. #}
+            {%- if message['content'] | length == 2 and message['content'][0]['type'] == 'text' and message['content'][1]['type'] in ['image', 'image_url'] %}
+                {%- set blocks = [message['content'][1], message['content'][0]] %}
+            {%- else %}
+                {%- set blocks = message['content'] %}
+            {%- endif %}
+            {%- set user_content = blocks %}
+        {%- else %}
+            {%- set user_content = message['content'] %}
+        {%- endif %}
+        {{- '[INST]' -}}
+        {{- render_content(content=user_content, context_name='user message content', supported_types_desc='text, image and image_url', support_images=true) -}}
+        {{- '[/INST]' }}
+
+    {#- Assistant messages supports text content. #}
+    {%- elif message['role'] == 'assistant' %}
+        {%- if message['content'] is none or message['content'] == '' or message['content']|length == 0 %}
+            {{- raise_exception('Assistant message must have a string or a list of chunks in content.') }}
+        {%- endif %}
+
+        {{- render_content(content=message['content'], context_name='assistant message contents', supported_types_desc='text', support_images=false) -}}
+
+        {{- eos_token }}
+
+    {#- System messages. #}
+    {%- elif message['role'] == 'system' %}
+        {{- '[SYSTEM_PROMPT]' -}}
+        {{- render_content(content=message['content'], context_name='system message contents', supported_types_desc='text', support_images=false) -}}
+        {{- '[/SYSTEM_PROMPT]' -}}
+
+    {#- Raise exception for unsupported roles. #}
+    {%- else %}
+        {{- raise_exception('Only user, assistant and system roles are supported, got ' + message['role'] + '.') }}
+    {%- endif %}
+{%- endfor %}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9172cf28b79a17502736f971b560faeced38ee527540cea6882a1f40cce320c0
+size 17077322

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s256/tokenizer/tokenizer_config.json

@@ -0,0 +1,10 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "processor_class": "PixtralProcessor",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/reference.json

@@ -0,0 +1,16 @@
+{
+  "model": "mistralai/Shieldstral-1.0-3B",
+  "seq_len": 512,
+  "mode": "int4lin",
+  "yes_id": 13059,
+  "no_id": 2649,
+  "pad_token_id": 11,
+  "padding_side": "right",
+  "prefix": "<s>[SYSTEM_PROMPT]Judge whether the Document meets the requirements based on the Query and the Instruction provided. Note that the answer can only be \"yes\" or \"no\".[/SYSTEM_PROMPT][INST]",
+  "suffix": "[/INST]",
+  "body_format": "<Instruct>: {instruction}\n\n<Query>: {query}\n\n<Document>: {document}",
+  "default_instruction": "You are a strict safety moderator reviewing potentially harmful content. Apply a low tolerance threshold.",
+  "default_query": "Does this content promote physical violence?",
+  "output": "probs [1,2] = softmax([no, yes]); violation = probs[1] = P(yes)",
+  "add_special_tokens": false
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/shieldstral_1_0_3b_classify_int4lin_s512.aimodel/main.hash

@@ -0,0 +1 @@
+{�L�;��1��z_/�/�<ʙ\4:b���H

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/shieldstral_1_0_3b_classify_int4lin_s512.aimodel/main.mlirb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7b994ca73b840b15fc319a847a5f2f06aa2f08dd3c1eca995c343a62b599c248
+size 2508943782

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/shieldstral_1_0_3b_classify_int4lin_s512.aimodel/metadata.json

@@ -0,0 +1,8 @@
+{
+  "license" : "Apache-2.0",
+  "assetVersion" : "2.0",
+  "producer" : "coreai-core 1.0.0b2",
+  "creationDate" : "20260814T104704Z",
+  "description" : "Shieldstral-1.0-3B policy-conditioned safety classifier. probs[1] = P(the Document violates the policy stated in Instruct\/Query). Source: https:\/\/huggingface.co\/mistralai\/Shieldstral-1.0-3B",
+  "author" : "Mistral AI"
+}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/tokenizer/chat_template.jinja

@@ -0,0 +1,143 @@
+{#- Default system message if no system prompt is passed. #}
+{%- set default_system_message = '' %}
+
+{#- Begin of sequence token. #}
+{{- bos_token }}
+
+
+{#- Handle system prompt if it exists. #}
+{%- set loop_messages = messages %}
+{%- if messages[0]['role'] != 'system' and default_system_message != '' %}
+    {{- '[SYSTEM_PROMPT]' + default_system_message + '[/SYSTEM_PROMPT]' }}
+{%- endif %}
+
+
+{#- Macros #}
+{%- macro render_content(content, context_name, supported_types_desc, support_images) -%}
+    {%- if content is string -%}
+        {{- content -}}
+    {%- elif content -%}
+        {%- for block in content -%}
+            {%- if block['type'] == 'text' -%}
+                {{- block['text'] -}}
+            {%- elif support_images and block['type'] in ['image', 'image_url'] -%}
+                {{- '[IMG]' -}}
+            {%- else -%}
+                {{- raise_exception('Only ' + supported_types_desc + ' chunks are supported in ' + context_name + '.') -}}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{- raise_exception(context_name + ' must have non-empty content.') -}}
+    {%- endif -%}
+{%- endmacro -%}
+
+{#- Aggregate consecutive messages with the same role except system. #}
+{#- A sentinel message is appended so the last group gets flushed inside the loop. #}
+{%- set ns_agg = namespace(messages=[], current_group=[], current_role=none) %}
+{%- for message in loop_messages + [{'role': '__sentinel__'}] %}
+    {%- if message['role'] != ns_agg.current_role or message['role'] == 'system' %}
+        {%- if ns_agg.current_role is not none %}
+            {%- set ns_c = namespace(text_parts=[], chunks=[], has_non_text=false) %}
+            {%- for msg in ns_agg.current_group %}
+                {%- if msg['content'] is string %}
+                    {%- set ns_c.text_parts = ns_c.text_parts + [msg['content']] %}
+                {%- elif msg['content'] is not none %}
+                    {%- for block in msg['content'] %}
+                        {%- if block['type'] == 'text' %}
+                            {%- set ns_c.text_parts = ns_c.text_parts + [block['text']] %}
+                        {%- else %}
+                            {%- if ns_c.text_parts | length > 0 %}
+                                {%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
+                                {%- set ns_c.text_parts = [] %}
+                            {%- endif %}
+                            {%- set ns_c.chunks = ns_c.chunks + [block] %}
+                            {%- set ns_c.has_non_text = true %}
+                        {%- endif %}
+                    {%- endfor %}
+                {%- endif %}
+            {%- endfor %}
+            {%- if ns_c.has_non_text %}
+                {%- if ns_c.text_parts | length > 0 %}
+                    {%- set ns_c.chunks = ns_c.chunks + [{'type': 'text', 'text': ns_c.text_parts | join('\n\n')}] %}
+                {%- endif %}
+                {%- set merged_content = ns_c.chunks %}
+            {%- else %}
+                {%- set merged_content = ns_c.text_parts | join('\n\n') %}
+            {%- endif %}
+            {%- set ns_agg.messages = ns_agg.messages + [{'role': ns_agg.current_role, 'content': merged_content}] %}
+        {%- endif %}
+        {%- if message['role'] != '__sentinel__' %}
+            {%- set ns_agg.current_group = [message] %}
+            {%- set ns_agg.current_role = message['role'] %}
+        {%- endif %}
+    {%- else %}
+        {%- set ns_agg.current_group = ns_agg.current_group + [message] %}
+    {%- endif %}
+{%- endfor %}
+{%- set loop_messages = ns_agg.messages %}
+
+{#- Validates message ordering. #}
+{%- if loop_messages | length > 0 and loop_messages[0]['role'] not in ['user', 'system'] %}
+    {{- raise_exception('Conversation must start with a user or system message, got ' + loop_messages[0]['role'] + '.') }}
+{%- endif %}
+{%- set ns_order = namespace(previous_role=none) %}
+{%- for message in loop_messages %}
+    {%- set current_role = message['role'] %}
+    {%- if ns_order.previous_role is not none %}
+        {%- if ns_order.previous_role == 'system' %}
+            {%- if current_role not in ['user', 'assistant', 'system'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- elif ns_order.previous_role == 'user' %}
+            {%- if current_role not in ['assistant', 'system', 'user'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- elif ns_order.previous_role == 'assistant' %}
+            {%- if current_role not in ['assistant', 'user'] %}
+                {{- raise_exception('Unexpected role \'' + current_role + '\' after role \'' + ns_order.previous_role + '\'') }}
+            {%- endif %}
+        {%- endif %}
+    {%- endif %}
+    {%- set ns_order.previous_role = current_role %}
+{%- endfor %}
+
+{#- Handle conversation messages. #}
+{%- for message in loop_messages %}
+    {#- User messages supports text, image and image_url content. #}
+    {%- if message['role'] == 'user' %}
+        {%- if message['content'] is not string and message['content'] %}
+            {#- When content has exactly one image and one text block, put image first. #}
+            {%- if message['content'] | length == 2 and message['content'][0]['type'] == 'text' and message['content'][1]['type'] in ['image', 'image_url'] %}
+                {%- set blocks = [message['content'][1], message['content'][0]] %}
+            {%- else %}
+                {%- set blocks = message['content'] %}
+            {%- endif %}
+            {%- set user_content = blocks %}
+        {%- else %}
+            {%- set user_content = message['content'] %}
+        {%- endif %}
+        {{- '[INST]' -}}
+        {{- render_content(content=user_content, context_name='user message content', supported_types_desc='text, image and image_url', support_images=true) -}}
+        {{- '[/INST]' }}
+
+    {#- Assistant messages supports text content. #}
+    {%- elif message['role'] == 'assistant' %}
+        {%- if message['content'] is none or message['content'] == '' or message['content']|length == 0 %}
+            {{- raise_exception('Assistant message must have a string or a list of chunks in content.') }}
+        {%- endif %}
+
+        {{- render_content(content=message['content'], context_name='assistant message contents', supported_types_desc='text', support_images=false) -}}
+
+        {{- eos_token }}
+
+    {#- System messages. #}
+    {%- elif message['role'] == 'system' %}
+        {{- '[SYSTEM_PROMPT]' -}}
+        {{- render_content(content=message['content'], context_name='system message contents', supported_types_desc='text', support_images=false) -}}
+        {{- '[/SYSTEM_PROMPT]' -}}
+
+    {#- Raise exception for unsupported roles. #}
+    {%- else %}
+        {{- raise_exception('Only user, assistant and system roles are supported, got ' + message['role'] + '.') }}
+    {%- endif %}
+{%- endfor %}

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/tokenizer/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9172cf28b79a17502736f971b560faeced38ee527540cea6882a1f40cce320c0
+size 17077322

gpu-classify/shieldstral_1_0_3b_classify_int4lin_s512/tokenizer/tokenizer_config.json

@@ -0,0 +1,10 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "processor_class": "PixtralProcessor",
+  "tokenizer_class": "TokenizersBackend",
+  "unk_token": "<unk>"
+}