PowderKegIntelligence/PKaI-Nano-1

PKaI Nano 1

PKaI Nano 1 is a 120M-class base language model from PowderKeg Intelligence. It is a compact LLaMA-style decoder model trained from scratch with a Mistral tokenizer and released as a PKaI-native artifact.

This repository contains PKaI-native weights and metadata, not a drop-in transformers.AutoModelForCausalLM package.

Files

• model.safetensors: PKaI base model weights.
• config.json: PKaI model architecture configuration.
• tokenizer.json: PKaI tokenizer metadata.
• tokenizer.model: SentencePiece tokenizer model from mistralai/Mistral-7B-v0.1.
• THIRD_PARTY_NOTICES.txt: tokenizer and training-data provenance notices.
• LICENSE: Apache License, Version 2.0.

Architecture

• Parameters: 112,680,448
• Vocabulary size: 32,000
• Context length: 512
• Layers: 14
• Attention heads: 12
• KV heads: 4
• Embedding size: 768
• Tied embeddings: yes
• QK normalization: yes

Training Data

Publicly disclosed training data sources include:

• HuggingFaceFW/fineweb-edu, released under the Open Data Commons Attribution License (ODC-By) v1.0 and subject to Common Crawl Terms of Use as noted by the dataset card.
• HuggingFaceTB/cosmopedia, released under the Apache License, Version 2.0.

See THIRD_PARTY_NOTICES.txt for source URLs, citations, and attribution notes.

License

Copyright 2026 PowderKeg Intelligence LLC.

The PKaI Nano 1 model artifact is released under the Apache License, Version 2.0. The bundled tokenizer and training-data sources have their own provenance and notices listed in THIRD_PARTY_NOTICES.txt.

Limitations

This is a small base model and has not been instruction-tuned or safety-tuned. It may produce inaccurate, unsafe, biased, or otherwise unsuitable text. Users are responsible for evaluating fitness, safety, and legal compliance for their own use cases.