File size: 2,374 Bytes
980b1b3 64bce88 980b1b3 64bce88 980b1b3 64bce88 980b1b3 64bce88 980b1b3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | ---
license: mit
library_name: mlx
tags:
- mlx
- function-calling
- tool-use
- encoder-decoder
- apple-silicon
---
# Needle MLX
An MLX safetensors format conversion of
[Cactus-Compute/needle](https://huggingface.co/Cactus-Compute/needle), a
26M-parameter encoder-decoder function-calling model.
This is not a fine-tune or a quantization. The published checkpoint is loaded
using Needle's declared `bfloat16` inference dtype, then stored as MLX
safetensors so it preserves the upstream runtime's executable values.
## What is included
- `model.safetensors`: 31 converted tensors, 26,315,421 parameters.
- `needle_mlx.py`: a small custom MLX inference implementation for Needle's
architecture.
- `tokenizer/`: the upstream SentencePiece tokenizer files.
- `config.json` and `manifest.json`: architecture and conversion provenance.
## Use
```python
from needle_mlx import NeedleModel
model = NeedleModel.from_pretrained(".")
logits = model.forward([[1, 2, 3]], [[1, 4]])
```
This is an MLX-native package, not an MLX-LM or oMLX model yet. Needle uses a
custom JAX/Flax architecture, so load it with the included runtime or add a
dedicated adapter to the serving runtime.
## Conversion and verification
The published `needle.pkl` checkpoint was read with a restricted NumPy-only
pickle loader, converted to MLX `bfloat16` safetensors, and checked
tensor-for-tensor after the same dtype cast used by Needle's upstream runtime.
- Source SHA-256:
`40a32e91d1d4197bf15ba559b74f6727c342dc8746918742fc7d8e2c1f18df40`
- Converted `model.safetensors` SHA-256:
`7b9d5f0d6ddeb7fbb20f4e45f3f616919357e5d08b5778859fdb762a33d60dae`
- Verification: 31/31 tensors equal the source after Needle's `bfloat16`
runtime cast; an MLX encoder-decoder smoke pass produced logits with shape
`(1, 2, 8192)` and generated the upstream weather tool-call example.
The converter source is available at
[seeker-cyber-maker/needle-mlx-depicklinator](https://github.com/seeker-cyber-maker/needle-mlx-depicklinator).
## Credits and license
Needle was created by Cactus Compute. See the
[upstream model card](https://huggingface.co/Cactus-Compute/needle) and
[source repository](https://github.com/cactus-compute/needle) for the model,
training details, and citation. The upstream model card publishes Needle under
the MIT license; this converted package retains that license.
|