---
library_name: transformers
license: apache-2.0
---

# tiny-random-ouro

Tiny random [Ouro](https://huggingface.co/ByteDance/Ouro-1.4B) model for `optimum-intel` OpenVINO CI.

Architecture: `OuroForCausalLM` (Universal Transformer). Generated with random weights; the full
`vocab_size` (49152) and the real tokenizer are kept so the tokenizer round-trips, while all other
dimensions are shrunk. `total_ut_steps=4` is preserved to exercise the Universal-Transformer loop.

| field | value |
|-------|-------|
| hidden_size | 32 |
| intermediate_size | 64 |
| num_hidden_layers | 2 |
| num_attention_heads | 4 |
| num_key_value_heads | 2 |
| head_dim | 8 |
| total_ut_steps | 4 |
| vocab_size | 49152 |

Reproduce with the script in the optimum-intel PR description.