genevera/XORTRON.CriminalComputing.2026.27B.Instruct-AWQ-4bit

XORTRON CriminalComputing 2026 27B Instruct AWQ 4-bit

AWQ-quantized export of darkc0de/XORTRON.CriminalComputing.2026.27B.Instruct prepared for vLLM serving.

Quantization Summary

• Quantization method: awq (AutoRound provider)
• Weight precision: 4-bit (bits=4, group_size=128, sym=true, zero_point=false)
• Quantized block scope: model.language_model.layers
• Calibration config: nsamples=64, iters=0, batch_size=1
• Format variant: gemm

Important Runtime Notes

• For vLLM AWQ in this build, use --dtype float16.
• --dtype auto may resolve to bfloat16 from model config and fail validation.

vLLM Example

vllm serve /path/to/XORTRON.CriminalComputing.2026.27B.Instruct-AWQ-4bit \
  --quantization awq \
  --dtype float16 \
  --trust-remote-code

Files

• config.json
• generation_config.json
• quantization_config.json
• model.safetensors.index.json
• model-00001-of-00010.safetensors ... model-00010-of-00010.safetensors
• tokenizer.json
• tokenizer_config.json
• processor_config.json
• chat_template.jinja

Base Model Attribution

All rights and usage constraints for the base model remain with the original model publisher: darkc0de/XORTRON.CriminalComputing.2026.27B.Instruct.