🛡️ Ordinal LLM — ordinal-2b

2.2B Security-Specialized Language Model with Anti-Hallucination Architecture

⚠️ This is the model architecture and configuration. Trained weights will be uploaded separately after training.

Architecture

Parameter	Value
Parameters	~2.2B
Hidden Size	2560
Layers	28
Attention Heads	20 (GQA: 4 KV heads)
Head Dim	128
Intermediate	6912
Vocab Size	50304
Max Context	8192
Dtype	bfloat16

Anti-Hallucination Features

1. Confidence Head: Per-token reliability score (threshold: 0.7)
2. Retrieval-Augmented Attention: 4 retrieval heads, dim=256
3. Fact Verification Layers: At layers [9, 18, 27]
4. Source Grounding Embeddings: 16 source types

Usage

from transformers import AutoModelForCausalLM, AutoConfig

# Load config
config = AutoConfig.from_pretrained("KaztoRay/ordinal-2b", trust_remote_code=True)

# Load model (after weights are uploaded)
model = AutoModelForCausalLM.from_pretrained("KaztoRay/ordinal-2b", trust_remote_code=True)

Chat Template

<|system|>
You are Ordinal, a cybersecurity AI assistant.<|end_turn|>
<|user|>
What is CVE-2021-44228?<|end_turn|>
<|assistant|>

Training Data

17,000+ instruction/response pairs from verified public databases:

• NVD CVEs (CRITICAL/HIGH/MEDIUM/LOW)
• MITRE ATT&CK (techniques, groups, software)
• CAPEC attack patterns
• CISA KEV (actively exploited)
• GitHub Security Advisories
• 500+ anti-hallucination training examples

Recommended Hardware

Quantization	VRAM Required
FP16	~4 GB
INT8	~2 GB
INT4	~1 GB

Citation

@software{ordinal_llm_2026,
  title={Ordinal LLM: Security-Specialized Language Model},
  author={KaztoRay},
  year={2026},
  url={https://github.com/KaztoRay/Ordinal}
}