---
library_name: transformers
pipeline_tag: text-generation
base_model: openPangu/openPangu-Embedded-7B
tags:
- diffusion
- parallel-generation
---

# FLUID-7B

FLUID (Flexible Unidirectional Inference Diffusion) is a framework designed to efficiently adapt pre-trained Autoregressive (AR) backbones into parallel diffusion models. By enforcing **Strictly Causal Alignment** and introducing **Elastic Horizons**, FLUID achieves state-of-the-art performance with significantly less training data compared to standard diffusion models.

- **Paper:** [From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons](https://huggingface.co/papers/2605.27387)
- **GitHub Repository:** [Oli-lab-nun/FLUID](https://github.com/Oli-lab-nun/FLUID)

## Key Features

*   **Strictly Causal Alignment**: Unlike bidirectional diffusion, FLUID uses a lower-triangular attention mask to maintain the inductive biases of AR priors. This enables seamless initialization from GPT-style checkpoints like openPangu-Embedded-7B.
*   **Elastic Horizon Modeling**: An entropy-driven mechanism that dynamically modulates denoising strides based on local information density. It "sprints" through predictable text and "downshifts" for complex reasoning.
*   **Training Efficiency**: Achieves superior results on reasoning benchmarks using only 2.7B tokens of adaptation data, outperforming models trained on trillions of tokens.

## Performance

FLUID-7B matches or exceeds top-tier AR and Diffusion baselines across standard benchmarks:

| Model | Type | Tokens | MMLU | GSM8K | MATH500 | HumanEval |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
| LLaMA-3-8B | AR | 15T | 68.4 | 78.3 | 27.4 | 59.8 |
| Qwen-2.5-7B | AR | 18T | 76.6 | 91.6 | 84.8 | 79.2 |
| LLaDA-8B | Diff | 2.0T | 65.5 | 36.2 | 34.2 | 47.6 |
| **FLUID-7B (Ours)** | **Diff** | **2.7B** | **67.8** | **91.9** | **61.8** | **60.4** |

## Acknowledgements

FLUID-7B is adapted from the **openPangu-Embedded-7B** base model. We gratefully acknowledge the developers of openPangu for releasing their model and related resources to the community.

## Citation

```bibtex
@inproceedings{fluid2026,
  title={From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizons},
  author={Anonymous},
  booktitle={Submission to ACL 2026},
  year={2026}
}
```