| license: apache-2.0 | |
| tags: | |
| - audio | |
| - speech | |
| - language-model | |
| - auristream | |
| - discrete-diffusion | |
| library_name: transformers | |
| # AuriStream Parallel - Speech Language Model | |
| **AuriStream Parallel** is a discrete diffusion speech language model by **Greta Tuckute** and **Klemen Kotar**. | |
| This repository contains shared model code for AuriStream Parallel checkpoints. | |
| ## Overview | |
| AuriStream Parallel uses: | |
| - bidirectional transformer attention | |
| - grouped token projection (`group_size=4` by default) | |
| - parallel token heads | |
| - partial-masking diffusion objective | |
| ## Usage | |
| Load a checkpoint repository that references this base code: | |
| ```python | |
| from transformers import AutoModel | |
| model = AutoModel.from_pretrained( | |
| "TuKoResearch/AuriStreamParallel100M_Group4_BigAudioDataset_180k", | |
| trust_remote_code=True, | |
| ) | |
| ``` | |
| ## Files | |
| - `configuration_auristream_parallel.py` - Configuration class | |
| - `modeling_auristream_parallel.py` - Model implementation | |