PAWN: Playstyle-Agnostic World-model Network for Chess
Collection
Causal transformers trained on random chess games. Testbed for finetuning at small scales. • 4 items • Updated
PAWN-BASE
A causal transformer trained on random chess games, designed as a testbed for finetuning and augmentation methods at small scales.
| Parameters | 35.8M |
| Architecture | Decoder-only transformer (RMSNorm, SwiGLU, RoPE) |
| d_model | 512 |
| Layers | 8 |
| Heads | 8 |
| Best val loss | 3.1058 (step 68,000) |
| Best val accuracy | 6.8% |
Usage
from safetensors.torch import load_file
from pawn.config import CLMConfig
from pawn.model import PAWNCLM
cfg = CLMConfig.base()
model = PAWNCLM(cfg)
model.load_state_dict(load_file("model.safetensors"))
model.eval()
See the PAWN repository (GitHub mirror) for training code and evaluation suite.
License
Apache 2.0
- Downloads last month
- 103
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support