Accio-Lab/occamy-1.0-FP8

Occamy 1.0 FP8

Blockwise FP8 conversion of Accio-Lab/occamy-1.0. The tokenizer and vision weights are preserved. MTP is available separately.

In the paired evaluation below, FP8 increased token NLL by 0.3378%. Sampled mathematics accuracy was 62/64 versus 63/64 for BF16; science, code, JSON, multi-turn, long-input and tool-roundtrip scores matched the baseline. See the complete comparison below and the validation notes for per-case details.

Format

• FP8 E4M3FN weights, 128 × 128 blocks, FP32 inverse scales, dynamic activation quantization.
• 14 safetensors shards, 36.6 GB total. File size is not a runtime VRAM requirement.
• Conversion: Transformers 5.8.1 Fp8Quantize, PyTorch 2.11.0+cu130. No calibration data or training was used.
• Tensor selection follows the format metadata of Qwen/Qwen3.6-35B-A3B-FP8. None of its weights are included.

Evaluation

SGLang 0.5.13.post1, PyTorch 2.11.0+cu130, FlashInfer 0.6.12. Identical prompts, greedy decoding, seed 42, thinking disabled, four concurrent requests, 16K configured context. Dataset revisions and fixed sample indices are in evaluation/provenance.json; fixtures and raw scored responses are included.

Check	BF16	FP8
GSM8K test sample	63/64	62/64
ARC-Challenge test sample, answer accuracy	59/64	59/64
Python tasks with independent functional tests	12/12	12/12
Strict JSON fixtures	20/20	20/20
Multi-turn memory fixtures	8/8	8/8
Long-input retrieval fixtures	6/6	6/6
Synthetic color/OCR/counting images	11/12	10/12
Native add-tool call, local execution and result follow-up	6/6	6/6
Request errors / truncated primary responses	0 / 0	0 / 0
WikiText test-sample token NLL, 4,080 tokens	2.206196	2.213649

FP8 NLL increased 0.3378%; the perplexity ratio is 1.00748. All scored log probabilities were finite, with matched token counts. No OOM or CUDA errors were observed.

These are sampled regression tests, not full benchmark scores. Per-case results, repeat checks, scoring details and coverage are in VALIDATION-NOTES.md and evaluation/. This release does not include MTP or claim a measured throughput speedup.

Tested serving command

SGLANG_JIT_DEEPGEMM_PRECOMPILE=0 python -m sglang.launch_server \
  --model-path Accio-Lab/occamy-1.0-FP8 --served-model-name occamy \
  --host 127.0.0.1 --port 18400 --dtype bfloat16 \
  --context-length 16384 --max-running-requests 4 \
  --max-total-tokens 65536 --max-mamba-cache-size 16 \
  --mem-fraction-static 0.60 --attention-backend fa3 \
  --mm-attention-backend fa3 --mamba-backend triton \
  --disable-cuda-graph --random-seed 42 \
  --tool-call-parser qwen3_coder --enable-metrics

FP8 is detected from config.json; --dtype bfloat16 applies to preserved weights. Memory settings depend on hardware. The original qwen tool parser does not parse this checkpoint's XML tool format correctly; the tested parser is qwen3_coder.

With the repository downloaded and the server ready:

python evaluation/evaluate.py http://127.0.0.1:18400 sglang-fp8
python evaluation/evaluate_nll.py http://127.0.0.1:18400 sglang-fp8

Reproduction settings and per-case results are in Technical details and evaluation/.

vLLM compatibility

On a single H200, this checkpoint completed all 9 requests and passed all 9 semantic checks, covering text, code, strict JSON, tool calls and result follow-up, and images. The BF16 reference also passed 9/9. There were no request errors or truncated responses. MTP was disabled. This is a small compatibility check, not a broad quality or speed benchmark.

Attribution and license

Original model: Accio-Lab/Occamy-1.0, Apache-2.0; original license included in LICENSE. Format metadata reference: Qwen/Qwen3.6-35B-A3B-FP8. Evaluation datasets retain their own licenses and attribution; see evaluation/DATA-LICENSES.md.