alpharomercoma/LFM2.5-1.2B-Instruct-ExecuTorch

LFM2.5-1.2B-Instruct for ExecuTorch

LFM2.5-1.2B-Instruct compiled for the ExecuTorch 1.4.0 LLM runtime with the XNNPACK backend and weights quantised for Arm CPUs, at 2k, 4k, 8k, 16k and 32k. These files replace an earlier export of the same recipe that mostly stopped calling tools and invented searches it had not run. If you downloaded a .pte from this repository before 2026-09-18, replace it.

Files	796 MB (2k) to 827 MB (32k)
Context window	fixed at export; pick the file
Prefill chunk	2048 tokens
Weights	int4, groups of 32, int8 dynamic activations (8da4w), int4 codes solved with GPTQ; int8 embedding
KV cache	fp32, allocated at load: about 805 MB for the full 32k window (6 of 16 layers attend, the rest are short convolutions)

Files

One repository per checkpoint; the backend is a folder and the context window is a file inside it, which is how the exporter publishes and what the app reads.

File	Window	Size
xnnpack/LFM2.5-1.2B-Instruct-8da4w-gptq-2k.pte	2,048	796 MB
xnnpack/LFM2.5-1.2B-Instruct-8da4w-gptq-4k.pte	4,096	798 MB
xnnpack/LFM2.5-1.2B-Instruct-8da4w-gptq-8k.pte	8,192	802 MB
xnnpack/LFM2.5-1.2B-Instruct-8da4w-gptq-16k.pte	16,384	810 MB
xnnpack/LFM2.5-1.2B-Instruct-8da4w-gptq-32k.pte	32,768	827 MB

The window is fixed inside each file and the runtime allocates the whole KV cache at load, so pick the largest one the device can hold. xnnpack/config.json repeats each window in the variants form, which lets a client label the files before downloading one.

What was wrong, and what changed

Two defects, both in the export, neither in XNNPACK's or KleidiAI's kernels.

1. Recurrent state survived the runner's reset. ExecuTorch 1.4.0's LFM2 definition keeps ShortConv.conv_state as a mutable buffer that nothing clears; TextLLMRunner::reset() only sets the position back to zero. Every prompt after the first ran on the previous prompt's convolution state. These exports multiply that state by clamp(input_pos, max=1) in the graph, which is what upstream's Qwen3.5 definition already does for its own recurrent state, and they advertise get_state_reset_at_zero so a host can tell a fixed file from an old one.
2. Round-to-nearest int4 weights. Not the int8 activations: keeping activations at int8 and weights at int8 preserved the behaviour. The int4 codes here are solved with GPTQ (Frantar et al.) on the delegate's own grid: symmetric, codes in -8 to 7, groups of 32, positive scales already exactly representable in bf16 because that is how ExecuTorch 1.4 stores them, with each layer's Hessian taken from its input after the per-token int8 rounding the delegate will apply. Same format, same size, same kernels.

What it does and does not recover

On 141 questions drawn after the calibration set was fixed, none of which anything was tuned on, through the OpenWeights app's own tool loop, one tool offered, on a Galaxy S25 Ultra (Snapdragon 8 Elite, Firebase Test Lab):

	Searched when needed	Correct	Fabricated a search
the earlier export, with the state leak fixed	10%	33%	25%
these files	49%	45%	1%
LFM2.5-1.2B-Instruct-Q4_K_M.gguf for comparison	51%	45%	1%
LFM2.5-1.2B-Instruct-QAD-Q4_0.gguf for comparison	56%	46%	0%

The same pair on one phone, so the two rows cannot be a device difference: round to nearest 11%, 30%, 26% against these files' 48%, 41%, 1%. On a Dimensity 9400 (Poco X8 Pro Max) these files read 44%, 41%, 1%.

Read that table honestly. On that run this export is level with the Q4_K_M GGUF on correctness and about seven points of recall behind Liquid's quantisation-aware QAD-Q4_0 checkpoint, and it searches when it does not need to more often than the file it replaces (29% against 2%), which is the one column where round to nearest looks better and is a consequence of calling at all. On a Dimensity 9400 the same one-tool comparison puts it seven points behind Q4_K_M and twelve behind QAD. What it buys over a GGUF is decode speed, 1.23 to 1.58 times llama.cpp's on the same phone, at the same file size as before.

How much the old export lost depends on the prompt, and so does the ordering. The table above offers one tool. With an app's full sixteen-tool prompt on a Dimensity 9400, one run each, these files searched when needed on 75% and were correct on 46%, against 60% and 40% for round to nearest, 60% and 35% for Q4_K_M and 59% and 37% for QAD-Q4_0: under that prompt this file leads both GGUFs, and the old export still trails but stops refusing to call.

Every figure above is a single run of a single file on one phone. The same file run on three Test Lab units spread about four points of correctness and six of known answers, which is the width to read all of them with.

The window costs memory, and nothing else

The same solved codes exported at all five windows give the same probability to four decimals on all sixteen probe prompts, and the same greedy first token. Choose the window by how much KV cache you can afford at load; a 12 GB phone is comfortable with any of them for this model.

Tokenizer and BOS

tokenizer.json is the base model's. The ExecuTorch 1.4.0 LLM runner does not ask for special tokens, so an app must write <|startoftext|> into the prompt text itself; the runtime's tokenizer encodes the literal to the BOS id (1). EOS is 7. Chat template: the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).

Provenance

Weights are Liquid AI's, from LiquidAI/LFM2.5-1.2B-Instruct, under the LFM Open License 1.0; this repository only changes their packaging. Exported 2026-09-18 with ExecuTorch 1.4.0, torch 2.14, torchao 0.18 on an Apple M-series Mac, from int4 codes solved on an RTX 4090. Calibration was the app's own prompt over held-out questions plus a held-out public draw from GSM8K, IFEval and BFCL, each with the fp32 model's greedy reply, so the decision token is in the data without a label; no question in any table above was in it. The method, every number and what is still unknown are in the OpenWeights repository under docs/research/executorch-state-and-recipes.md.