Reconvert with mlx-vlm 0.6.9 (official config schema; restores MoE global_scale/gate.bias dropped by the 0.6.7 conversion)

README.md

@@ -17,53 +17,26 @@ text + image + audio in, text out.
 
 See also: [`ToPo-ToPo/Inkling-Small-mlx-4bit`](https://huggingface.co/ToPo-ToPo/Inkling-Small-mlx-4bit).
 
-## Provenance (self-converted from official weights)
-- Source: [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small) (license: apache-2.0, bf16, 531.9 GB)
-- Tool: `mlx-vlm 0.6.7` — `mlx_vlm.convert --hf-path <staged> --mlx-path . -q --q-bits 2 --q-group-size 64`
-- Effective: **2.506 bits/weight** (77 GiB on disk, ~83.5 GB peak RSS at inference)
-
-## Config translation (already applied here)
-
-The official `config.json` cannot be loaded by `mlx-vlm` as published — its schema uses different key names
-than `mlx_vlm.models.inkling` expects. This repo ships the translated config, so no action is needed:
+## Requires mlx-vlm >= 0.6.9
 
-| Key | Official | Here |
-| --- | --- | --- |
-| `model_type` | `inkling_mm_model` | `inkling` |
-| `text_config.intermediate_size` | 2048 (expert width) | 16384 (dense width) |
-| `text_config.moe_intermediate_size` | — (was `intermediate_size`) | 2048 |
-| `vision_config.text_hidden_size` | `decoder_dmodel` | 4096 |
-| `audio_config.text_hidden_size` | `decoder_dmodel` | 4096 |
-| `vision_config.num_channels` | `n_channels` | 3 |
-| `tokenizer_config.pad_token` / `eos_token` | unset | `<\|endoftext\|>` / `<\|content_model_end_sampling\|>` |
-
-`image_token_id` / `audio_token_id` are left unset so that mlx-vlm's defaults (200054 / 200053) apply —
-those are the placeholders `InklingProcessor` actually emits.
-
-## Loader shim (required)
-
-`mlx-vlm` 0.6.7 and 0.6.8 need two patches at runtime for any Inkling checkpoint. Apply them before `load()`:
+0.6.9 is the first release whose `models/inkling` can load an official Inkling checkpoint through the
+public loader, and the first that implements the MoE `global_scale` / `gate.bias` tensors. On 0.6.7 / 0.6.8
+this repo will not load.
 
 ```python
-from mlx_vlm.models import inkling
-from mlx_vlm.models.inkling import config as inkling_config
-from mlx_vlm import prompt_utils
-
-# 1. models/inkling does not re-export its sub-config classes, which the generic
-#    loader fetches with getattr -> AttributeError on load.
-for name in ("TextConfig", "VisionConfig", "AudioConfig"):
-    setattr(inkling, name, getattr(inkling_config, name))
-
-# 2. prompt_utils.MODEL_CONFIG has no "inkling" entry, so apply_chat_template treats
-#    the model as text-only and drops image/audio parts.
-prompt_utils.MODEL_CONFIG.setdefault("inkling", prompt_utils.MessageFormat.LIST_WITH_IMAGE_FIRST)
+from mlx_vlm import load, generate
+model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")
 ```
 
-When serving over `mlx_vlm.server`, also add Inkling's structural tokens to
-`mlx_vlm.server.responses_state._CONTENT_MARKERS` (`<|message_model|>`, `<|content_text|>`,
-`<|end_message|>`) and set `MLX_VLM_THINKING_START_TOKEN=<|content_thinking|>` /
-`MLX_VLM_THINKING_END_TOKEN=<|end_message|>`, so the reasoning channel and those markers do not
-end up in `content`.
+The config is the **official schema, unmodified** — no key translation and no loader patches are needed.
+
+## Provenance (self-converted from official weights)
+- Source: [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small) (license: apache-2.0, bf16, 531.9 GB)
+- Tool: `mlx-vlm 0.6.9` — `mlx_vlm.convert --hf-path thinkingmachines/Inkling-Small --mlx-path . -q --q-bits 2 --q-group-size 64`
+- Effective: **2.506 bits/weight** (77 GiB on disk)
+- Only edit on top of the conversion: `pad_token` / `eos_token` added to `tokenizer_config.json`
+  (the official `TokenizersBackend` config sets neither, so transformers raises on any padded call).
+  Both point at existing ids — the vocabulary is unchanged.
 
 ## Reasoning effort
 
@@ -71,11 +44,17 @@ The chat template always injects a `Thinking effort level:` system message (defa
 with the OpenAI-compatible `reasoning_effort` — `"none"` / `"minimal"` / `"low"` / `"medium"` / `"high"` /
 `"max"`, or a float in `[0.0, 0.99]`. `"none"` disables thinking entirely.
 
-## Usage
+When serving over `mlx_vlm.server`, note that Inkling wraps its answer in structural tokens
+(`<|message_model|>`, `<|content_text|>`, `<|end_message|>`) which the server's fixed
+`_CONTENT_MARKERS` list does not strip, and that its reasoning channel is
+`<|content_thinking|>` … `<|end_message|><|message_model|>` rather than one of the built-in marker pairs.
+Set `MLX_VLM_THINKING_START_TOKEN` / `MLX_VLM_THINKING_END_TOKEN` accordingly and strip the structural
+tokens, or the reasoning and those markers end up in `content`.
 
-```python
-from mlx_vlm import load, generate
-model, processor = load("ToPo-ToPo/Inkling-Small-mlx-2bit")  # apply the shim above first
-```
+## Revision history
 
-The first load reads 77 GiB, so allow a few minutes and raise any client-side startup timeouts accordingly.
+- **2026-08-04** — reconverted with mlx-vlm 0.6.9. The previous upload had been converted with 0.6.7,
+  whose `models/inkling` did not implement the MoE `mlp.global_scale` (50 keys) and `mlp.gate.bias`
+  (40 keys) present in the official checkpoint, so **those tensors were silently dropped**. It also
+  shipped a translated config (renamed `intermediate_size` / `dense_intermediate_size`, etc.) that 0.6.9
+  rejects. If you pulled this repo before this date, re-download it.

config.json

@@ -3,17 +3,17 @@
         "InklingForConditionalGeneration"
     ],
     "audio_config": {
+        "decoder_dmodel": 4096,
         "n_mel_bins": 80,
         "mel_vocab_size": 16,
         "bias": false,
         "dmel_min_value": -7.0,
         "dmel_max_value": 2.0,
         "use_audio_norm": true,
-        "audio_mode": "dmel",
-        "text_hidden_size": 4096
+        "audio_mode": "dmel"
     },
     "eos_token_id": 200006,
-    "model_type": "inkling",
+    "model_type": "inkling_mm_model",
     "mtp_config": {
         "num_nextn_predict_layers": 8,
         "chain_hidden_post_norm": false,
@@ -104,21 +104,21 @@
         "num_experts_per_tok": 6,
         "n_shared_experts": 2,
         "shared_expert_sink": true,
-        "intermediate_size": 16384,
+        "dense_intermediate_size": 16384,
+        "intermediate_size": 2048,
         "route_scale": 8.0,
         "use_gate_bias": true,
         "gate_activation": "sigmoid",
         "norm_after_topk": true,
-        "use_global_scale": true,
-        "moe_intermediate_size": 2048
+        "use_global_scale": true
     },
     "vision_config": {
         "vision_encoder_type": "hmlp",
+        "decoder_dmodel": 4096,
         "patch_size": 40,
         "temporal_patch_size": 2,
+        "n_channels": 3,
         "n_layers": 4,
-        "use_vision_norm": true,
-        "num_channels": 3,
-        "text_hidden_size": 4096
+        "use_vision_norm": true
     }
 }

model-00001-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27171f0f989f1a71f35c7d8967f8e5f6506608eb46e2326911f0d52b11f97ff5
-size 5187900962
+oid sha256:ab0b0bf0b51bbf38698cc48f4739a721e1a061d27a1d2e04eb1716f0351dbc1c
+size 5187899756

model-00002-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:837e7c88b8f4043486cd9b69d811a3753ef036ea7fe3d1309a21dd4c5678abf5
-size 5365071335
+oid sha256:5a9f1a9c06c07e10f16bbb1a51ac3e6da07bd26383c84ea45e290fe20dc5b2c1
+size 5365073637

model-00003-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dced781d910d705a8ee86b11c7c49d1932a82cb2ff90af7105b618e0ba131c19
-size 4859909583
+oid sha256:cf1b5c741a78af61110e77600ea0ad76db270e9ed3cd4476c3164a2ca6177e5a
+size 4859913000

model-00004-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2f62312f7e5dd3c8f803e1134fc8371b6406147b6b22442d4286a2402007b060
-size 5365071471
+oid sha256:fae89354568e09d0cc59d1f9769e910c9465c9511932eb27e8108dfc667608ec
+size 5365073701

model-00005-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4918dbdb9ca2add617b0c61fa579c80f3999f101c324e9919246831ac5365b6f
-size 4859909727
+oid sha256:4517d652f296829d17b6e3f99434bfdced96af73e315879da943a7b91461e320
+size 4859913099

model-00006-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:295e1bcd71c1f2d308c8a7ba14010e0a52e5416563835a15899fc401f7753f79
-size 5365055039
+oid sha256:5a218f6b938f2934d28ca2039e43bbb0fce598333ac5b4f62921414ea1a74879
+size 5365057335

model-00007-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:812490c0da651ff7dac6d62f1b3c98c29c7e86dc3bc453af998b02c3a2a3d68f
-size 4859926156
+oid sha256:2ec49a07dadf0f152180dc22b2397c4a7efd87bf3fad922e663de253a23378f8
+size 4859929510

model-00008-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0aef9a454632bd934919ab64e0b3fbd4bd60df94e41b63e9b805827664cff6ca
-size 5365055095
+oid sha256:490e72bee117d9a0e965c7132e37eea48524d2c57194204cbfbfb769c387104c
+size 5365057329

model-00009-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:28f63ef9294524befb334d8a4c2ebee35ba0a5e1d02561706d95a6fd478285f5
-size 4859926148
+oid sha256:fb59a3de289772e49364d2f2eed7987a292453cde70ad4a763878cba930ef7d3
+size 4859929496

model-00010-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:81177d8a971d5bedad9c17ad6a55213a2210cf06dcc119d39dd3960371f58bf8
-size 5365055043
+oid sha256:16e61880f3d91e938ae69ee99e6a610f2e1a28e09356ce9b563668dd6c123c5a
+size 5365057337

model-00011-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:90b808b6f4add8fa09db3a73d6c1b8058890f3c2e4a015184607de77b695480f
-size 4859926158
+oid sha256:1f71b52cde8b5745603854ceadf3ce11e9c24be187231d899e5727ece4751386
+size 4859929506

model-00012-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8ffa3548e85648edd71660902223fdbae20c2a99b1b9a3a022654af3ea1ed89a
-size 5365055103
+oid sha256:759f982406433526753a6900fdb51ce0094ef7e9233932a9a780661fd3284a93
+size 5365057339

model-00013-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:108804d405ced5048a9d0932e554e9ee6a208c3d5070eefc28e1acb1a0e7d8b5
-size 4859909783
+oid sha256:32d008bf36054dfde8c21980b2c16b77cf453aa52646b2783f39ab6a146aaf23
+size 4859913141

model-00014-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4b8834eb749e562dcbd18f4c51f0bfa1cd7f5eed90b536b45bf54eaf3c0789c
-size 5365071424
+oid sha256:58dc8f6cbcd31d90acac62e2a925f63b8bd75d6c6ecd4d34c46ed1e923b99c46
+size 5365073676

model-00015-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7b38c9613579e1c87572c858d52b23f8164eedb583da034b10d18c0e28afbe78
-size 4859909735
+oid sha256:db42e0267db293908fbaa136adce7be0ff3aceb48d6555e86c3486a2d8744b6e
+size 4859913091

model-00016-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:75b84c0d634641218f19928945dfeee90e6eb9ab94d0527f977beb411ef47c60
-size 5365071474
+oid sha256:f28c04a2a8d978ad333343afb9d97293f0546b05eabb2d604bb8ef3350abf3fd
+size 5365073726

model-00017-of-00017.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4d738dee26b15bd4fcd42f19ef4f73554f8820de2e118adca4448c70be4ccd11
-size 478896590
+oid sha256:cc68c5326ad5f94ea283c99e1f106ead41104f88d9afcded508bf8551c7b3455
+size 478898863

processor_config.json

@@ -3,25 +3,8 @@
   "audio_token": "<|unused_200053|>",
   "dmel_max_value": 2.0,
   "dmel_min_value": -7.0,
-  "feature_extractor": {
-    "audio_token_duration_s": 0.05,
-    "feature_extractor_type": "InklingFeatureExtractor",
-    "feature_size": 80,
-    "hop_length": 800,
-    "n_fft": 1600,
-    "padding_side": "right",
-    "padding_value": 0.0,
-    "return_attention_mask": true,
-    "sampling_rate": 16000,
-    "window_size": 1600,
-    "window_size_multiplier": 2.0
-  },
   "image_bos_token": "<|content_image|>",
   "image_processor": {
-    "do_convert_rgb": true,
-    "do_normalize": true,
-    "do_rescale": true,
-    "do_resize": true,
     "image_mean": [
       0.48145466,
       0.4578275,
@@ -33,13 +16,10 @@
       0.26130258,
       0.27577711
     ],
-    "resample": 3,
+    "patch_size": 40,
     "rescale_factor": 0.00392156862745098,
     "rescale_image_max_upscaled_long_edge": 2048,
-    "size": {
-      "height": 40,
-      "width": 40
-    }
+    "temporal_patch_size": 2
   },
   "image_token": "<|unused_200054|>",
   "num_dmel_bins": 16,

tokenizer_config.json

@@ -18,7 +18,7 @@
   "eos_token": "<|content_model_end_sampling|>",
   "fix_mistral_regex": false,
   "is_local": true,
-  "local_files_only": false,
+  "local_files_only": true,
   "message_model": "<|message_model|>",
   "message_system": "<|message_system|>",
   "message_tool": "<|message_tool|>",