Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -7,4 +7,61 @@ sdk: static
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
pinned: false
|
| 8 |
---
|
| 9 |
|
| 10 |
+
# 🔺 Project Prism
|
| 11 |
+
|
| 12 |
+
**Project Prism is an experimental AI architecture research project by SmilyAI Labs.**
|
| 13 |
+
|
| 14 |
+
We are investigating a simple question:
|
| 15 |
+
|
| 16 |
+
> **Does one neural network really need to learn knowledge, computation, and working memory in the same way?**
|
| 17 |
+
|
| 18 |
+
Our current research model, **Orion Flagship**, explores an architecture we call **Transformer 2 (T2)**.
|
| 19 |
+
|
| 20 |
+
## Transformer 2
|
| 21 |
+
|
| 22 |
+
Traditional language models largely rely on a homogeneous stack of Transformer layers to learn many different functions at once.
|
| 23 |
+
|
| 24 |
+
T2 explores separating several of these roles:
|
| 25 |
+
|
| 26 |
+
**KNOW** — sparse learned memory for persistent knowledge
|
| 27 |
+
**DO** — reusable neural procedures for computation and transformation
|
| 28 |
+
**THINK** — persistent working state for intermediate reasoning
|
| 29 |
+
**VERIFY** — additional computation for checking and refining results
|
| 30 |
+
|
| 31 |
+
The goal is to investigate whether this separation can improve parameter and compute efficiency while retaining strong general capabilities across language, code, mathematics, and reasoning.
|
| 32 |
+
|
| 33 |
+
## 🌌 Orion Flagship
|
| 34 |
+
|
| 35 |
+
**Orion Flagship** is the primary experimental model being developed under Project Prism.
|
| 36 |
+
|
| 37 |
+
The planned flagship architecture targets approximately:
|
| 38 |
+
|
| 39 |
+
- **7B total parameters**
|
| 40 |
+
- **~1B active parameters**
|
| 41 |
+
- Sparse Knowledge Vault
|
| 42 |
+
- Shared Procedure Banks
|
| 43 |
+
- Persistent Working State
|
| 44 |
+
- Adaptive recurrent deliberation
|
| 45 |
+
- General-purpose language, code, and mathematical reasoning
|
| 46 |
+
|
| 47 |
+
Before scaling to the full model, we are testing the architecture with smaller prototypes and controlled ablations.
|
| 48 |
+
|
| 49 |
+
## 🧪 Current research
|
| 50 |
+
|
| 51 |
+
Our current Mini-T2 experiments are investigating:
|
| 52 |
+
|
| 53 |
+
- whether learned memory develops useful knowledge retrieval;
|
| 54 |
+
- whether Procedure Banks develop reusable computational specialisation;
|
| 55 |
+
- whether Working State improves multi-step reasoning;
|
| 56 |
+
- whether recurrent deliberation improves difficult problems;
|
| 57 |
+
- and whether these components provide measurable advantages over compute-matched Transformer baselines.
|
| 58 |
+
|
| 59 |
+
**Results are experimental. Claims about architectural improvements will be based on controlled evaluations and ablations rather than training loss alone.**
|
| 60 |
+
|
| 61 |
+
---
|
| 62 |
+
|
| 63 |
+
### Project Prism
|
| 64 |
+
|
| 65 |
+
**KNOW · DO · THINK · VERIFY**
|
| 66 |
+
|
| 67 |
+
*Exploring what comes after the conventional Transformer.*
|