ByteDance
/

Bernini-Diffusers

@@ -1,179 +0,0 @@
----
-license: apache-2.0
-pipeline_tag: image-text-to-video
----
-<div align="center">
-<h1 align="center">Bernini-Diffusers</h1>
-<h4 align="center">Latent Semantic Planning for Video Diffusion</h4>
-**Chenchen Liu<sup>\*</sup>, Junyi Chen<sup>\*</sup>, Lei Li<sup>\*</sup>, Lu Chi<sup>\*,§</sup>, Mingzhen Sun<sup>\*</sup>, Zhuoying Li<sup>\*</sup>, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan<sup>✉</sup>**
-<sup>\*</sup> Equal contribution&nbsp;&nbsp;<sup>✉</sup> Corresponding author&nbsp;&nbsp;<sup>§</sup> Project lead
-[![arXiv](https://img.shields.io/badge/arXiv-2605.22344-b31b1b.svg)](https://arxiv.org/abs/2605.22344)
-[![Project Page](https://img.shields.io/badge/Project-Page-blue.svg)](https://bernini-ai.github.io/)
-[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Models-yellow)](https://huggingface.co/collections/ByteDance/bernini)
-</div>
-## 🎉 News
-- **[2026-06-10]** We open-sourced the inference code and model weights of the full Bernini (**Bernini**).
-- **[2026-05-22]** We released our paper [Bernini: Latent Semantic Planning for Video Diffusion](https://arxiv.org/abs/2605.22344).
-## ✨ Highlights
-Bernini is a unified framework for video generation and editing that combines an MLLM-based semantic planner with a DiT-based renderer.
-Compared with the renderer-only Bernini-R release, **Bernini-Diffusers** packages the full semantic-planning pipeline: a Qwen2.5-VL planner, Bernini planning weights, and Wan2.2 diffusion components in one self-contained directory. This makes it the recommended release when you need stronger instruction following, multi-step semantic planning, and better handling of complex video editing requests.
-## 🧾 Model card
-| Field | Description |
-|-------|-------------|
-| Model type | Full video generation/editing pipeline with an MLLM-based semantic planner and a DiT-based renderer. |
-| Checkpoint | [`ByteDance/Bernini-Diffusers`](https://huggingface.co/ByteDance/Bernini-Diffusers) |
-| Code | [`ByteDance/Bernini`](https://github.com/bytedance/Bernini) |
-| Recommended use | Complex generation/editing requests that benefit from explicit latent semantic planning and stronger instruction following. |
-| Model behavior | Better at decomposing complex instructions and planning semantic changes before rendering, at the cost of a heavier checkpoint layout than Bernini-R. |
-### Benchmark snapshot
-| Model | EditVerse | OpenVE | OpenS2V | VBench | Bernini-v2v (OS) | Bernini-vr2v (OS) |
-|---|---|---|---|---|---|---|
-| [Bernini 7+14B](https://huggingface.co/ByteDance/Bernini-Diffusers) | 8.02 | 4.03 | 62.30 | 84.37 | 3.49 | 3.48 |
-On video editing, Bernini reaches the first tier among leading closed-source commercial models in our internal arena evaluation based on blind human pairwise comparisons.
-## 📦 Package layout
-This release is a **self-contained diffusers-format directory**. Pass the downloaded `Bernini-Diffusers` directory directly to `--config`.
-```text
-Bernini-Diffusers/
-  bernini/
-  mllm/
-  scheduler/
-  t5_text_encoder/
-  t5_tokenizer/
-  vae/
-  config.json
-  transformer_config.json
-  transformer_2_config.json
-```
-At runtime:
-- `bernini/` provides the Bernini planning checkpoint.
-- `mllm/` provides the Qwen2.5-VL planner assets.
-- `transformer_config.json` and `transformer_2_config.json` define the Wan2.2 diffusion decoder components used by the full pipeline.
-- `t5_text_encoder/`, `t5_tokenizer/`, `vae/`, and `scheduler/` provide the base diffusion modules required for inference.
-## 📥 Download
-```bash
-pip install -U "huggingface_hub"
-hf download ByteDance/Bernini-Diffusers \
-    --local-dir pretrained_models/Bernini-Diffusers
-```
-## 🚀 Usage
-The official inference code is available in the [Bernini repository](https://github.com/bytedance/Bernini).
-### Installation
-```bash
-git clone https://github.com/bytedance/Bernini.git bernini && cd bernini
-pip install -r requirements.txt
-```
-Recommended environment:
-- **Python** 3.11.2
-- **PyTorch** 2.5.1+cu124
-- **CUDA toolkit** 12.4
-- **GPU** Hopper GPUs (H100/H800/H200) are recommended for best performance
-For multi-GPU sequence parallel inference, install VeOmni:
-```bash
-pip install --no-deps git+https://github.com/ByteDance-Seed/VeOmni.git@v0.1.10
-```
-### Load the model
-Pass the downloaded directory directly as `--config`:
-```bash
-python infer_single_gpu.py --config pretrained_models/Bernini-Diffusers \
-    --case assets/testcases/i2i/i2i.json --num_frames 1
-```
-### Prompt enhancer (highly recommended)
-`--use_pe` enhances the prompt through an OpenAI-compatible endpoint and is recommended for best generation quality.
-```bash
-export BERNINI_PE_API_KEY=...      # or OPENAI_API_KEY
-export BERNINI_PE_BASE_URL=...     # or OPENAI_BASE_URL
-export BERNINI_PE_MODEL=...        # vision-capable chat model
-```
-### Gradio demo
-```bash
-# Single GPU
-python gradio_demo.py --config pretrained_models/Bernini-Diffusers --port 7860
-# 8 GPUs, 8-way Ulysses sequence parallel
-torchrun --nproc-per-node 8 gradio_demo.py --ulysses 8 \
-    --config pretrained_models/Bernini-Diffusers \
-    --port 7860 --share
-```
-### Run scripts
-The [`scripts/bernini/`](https://github.com/bytedance/Bernini/tree/master/scripts/bernini) directory in the Bernini repo provides ready-to-run task launchers for the full pipeline:
-- `run_t2i.sh`
-- `run_i2i.sh`
-- `run_t2v.sh`
-- `run_v2v.sh`
-- `run_rv2v.sh`
-- `run_r2v.sh`
-- `run_gradio.sh`
-You can override the model directory with:
-```bash
-export BERNINI_CONFIG=/path/to/Bernini-Diffusers
-```
-## 📑 Citation
-If you use Bernini in your research, please cite:
-```bibtex
-@article{bernini,
-  title   = {Bernini: Latent Semantic Planning for Video Diffusion},
-  author  = {Chenchen Liu and Junyi Chen and Lei Li and Lu Chi and Mingzhen Sun and Zhuoying Li and Yi Fu and Ruoyu Guo and Yiheng Wu and Ge Bai and Zehuan Yuan},
-  journal = {arXiv preprint arXiv:2605.22344},
-  year    = {2026}
-}
-```
-## 🙏 Acknowledgements
-Bernini builds on several outstanding open-source projects:
-- [Wan2.2-T2V-A14B](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B)
-- [Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)
-- [VeOmni](https://github.com/ByteDance-Seed/VeOmni)
-## 📄 License
-Apache License 2.0.