| --- |
| license: mit |
| library_name: mlx |
| tags: |
| - mlx |
| - function-calling |
| - tool-use |
| - encoder-decoder |
| - apple-silicon |
| --- |
| |
| # Needle MLX |
|
|
| An MLX safetensors format conversion of |
| [Cactus-Compute/needle](https://huggingface.co/Cactus-Compute/needle), a |
| 26M-parameter encoder-decoder function-calling model. |
|
|
| This is not a fine-tune or a quantization. The published checkpoint is loaded |
| using Needle's declared `bfloat16` inference dtype, then stored as MLX |
| safetensors so it preserves the upstream runtime's executable values. |
|
|
| ## What is included |
|
|
| - `model.safetensors`: 31 converted tensors, 26,315,421 parameters. |
| - `needle_mlx.py`: a small custom MLX inference implementation for Needle's |
| architecture. |
| - `tokenizer/`: the upstream SentencePiece tokenizer files. |
| - `config.json` and `manifest.json`: architecture and conversion provenance. |
|
|
| ## Use |
|
|
| ```python |
| from needle_mlx import NeedleModel |
| |
| model = NeedleModel.from_pretrained(".") |
| logits = model.forward([[1, 2, 3]], [[1, 4]]) |
| ``` |
|
|
| This is an MLX-native package, not an MLX-LM or oMLX model yet. Needle uses a |
| custom JAX/Flax architecture, so load it with the included runtime or add a |
| dedicated adapter to the serving runtime. |
|
|
| ## Conversion and verification |
|
|
| The published `needle.pkl` checkpoint was read with a restricted NumPy-only |
| pickle loader, converted to MLX `bfloat16` safetensors, and checked |
| tensor-for-tensor after the same dtype cast used by Needle's upstream runtime. |
|
|
| - Source SHA-256: |
| `40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40` |
| - Converted `model.safetensors` SHA-256: |
| `7b9d5f0d6ddeb7fbb20f4e45f3f616919357e5d08b5778859fdb762a33d60dae` |
| - Verification: 31/31 tensors equal the source after Needle's `bfloat16` |
| runtime cast; an MLX encoder-decoder smoke pass produced logits with shape |
| `(1, 2, 8192)` and generated the upstream weather tool-call example. |
|
|
| The converter source is available at |
| [seeker-cyber-maker/needle-mlx-depicklinator](https://github.com/seeker-cyber-maker/needle-mlx-depicklinator). |
|
|
| ## Credits and license |
|
|
| Needle was created by Cactus Compute. See the |
| [upstream model card](https://huggingface.co/Cactus-Compute/needle) and |
| [source repository](https://github.com/cactus-compute/needle) for the model, |
| training details, and citation. The upstream model card publishes Needle under |
| the MIT license; this converted package retains that license. |
|
|
