🛡️ Ordinal LLM

Security-Specialized Large Language Model with Anti-Hallucination Architecture

Ordinal is a custom transformer architecture built from scratch for cybersecurity tasks. It is NOT a fine-tune of any existing model — it's a fully independent architecture with security-specific innovations.

Model Sizes

Model	Params	Layers	Hidden	Heads	Recommended For
ordinal-128m	~128M	12	768	12	Testing, edge deployment
ordinal-256m	~256M	16	1024	16	Mobile, IoT security
ordinal-512m	~512M	20	1536	16	Embedded systems
ordinal-1b	~1.0B	24	2048	16	Light security tasks
ordinal-2b	~2.0B	28	2560	20	General security Q&A
ordinal-4b	~4.0B	32	3072	24	RTX 4090 (LoRA)
ordinal-5b	~5.0B	36	3584	28	⭐ Recommended
ordinal-7b	~7.0B	32	4096	32	A100 40GB
ordinal-13b	~13B	40	5120	40	A100 80GB
ordinal-20b	~20B	52	6144	48	2× A100 80GB
ordinal-33b	~33B	64	6656	52	4× A100 80GB
ordinal-48b	~48B	72	8192	64	8× H100

Architecture Features

User Query → PromptGuard → RAG Retrieval → Ordinal Model
                                              ├── GQA + RoPE + SwiGLU
                                              ├── Confidence Head (per-token)
                                              ├── Fact Verification Layers
                                              └── Source Grounding Embeddings
                                                     ↓
                                         Confidence < 0.7 → "I'm uncertain"

Key Innovations

• Grouped Query Attention (GQA): Efficient KV-head sharing
• SwiGLU MLP: Gated activation for better feature learning
• RoPE: Rotary Position Embeddings (θ=500,000)
• RMSNorm: Pre-normalization for training stability
• Confidence Head: Per-token reliability scoring
• Retrieval-Augmented Attention: Direct RAG integration in attention
• Fact Verification Layers: Cross-check at 1/3, 2/3, and final layers
• Source Grounding Embeddings: Track information provenance

Training Data

17,000+ instruction/response pairs from verified public databases only:

Source	Records	Description
NVD CVEs	9,500+	CRITICAL/HIGH/MEDIUM/LOW vulnerabilities
MITRE ATT&CK	1,800+	Techniques, groups, software
CAPEC	1,100+	Attack patterns + defensive guidance
GitHub Advisories	1,700+	Multi-ecosystem security advisories
CISA KEV	500	Actively exploited vulnerabilities
Anti-Hallucination	500+	Refusal, uncertainty, fact-checking
Expert Q&A	36	Hand-crafted security deep-dives
DPO Pairs	12	Preference optimization data

Usage

from transformers import AutoModelForCausalLM, AutoConfig

# Load
config = AutoConfig.from_pretrained("KaztoRay/ordinal-5b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "KaztoRay/ordinal-5b",
    trust_remote_code=True,
    torch_dtype="auto",
    device_map="auto",
)

# Chat
messages = [
    {"role": "system", "content": "You are Ordinal, a cybersecurity AI."},
    {"role": "user", "content": "What is CVE-2021-44228?"},
]

Ollama

ollama run ordinal-5b

vLLM

vllm serve KaztoRay/ordinal-5b --trust-remote-code --dtype bfloat16

Evaluation

Benchmark	Score
SecurityBench (8 domains)	79.6%
Anti-Hallucination Detection	92%
NER Accuracy (18 entity types)	100%
Text Classification (8 categories)	87.5%
Red Team Safety	92%

Anti-Hallucination System

4-layer defense:

1. PromptGuard: Input injection detection (20+ patterns)
2. RAG Verification: Ground responses in verified data
3. Confidence Head: Per-token reliability scoring
4. Training Data: 500+ explicit refusal/uncertainty examples

Limitations

• Training data cutoff applies — may not know very recent CVEs
• Best performance on English security content
• Requires GPU for inference (10GB+ VRAM for 5B)
• Not a replacement for professional security analysis
• Architecture + config only — trained weights uploaded separately

Citation

@software{ordinal_llm_2026,
  title={Ordinal LLM: Security-Specialized Language Model},
  author={KaztoRay},
  year={2026},
  url={https://github.com/KaztoRay/Ordinal}
}