Update README.md
Browse files
README.md
CHANGED
|
@@ -1,192 +1,47 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
tags:
|
| 13 |
-
- image-generation
|
| 14 |
-
- image-editing
|
| 15 |
-
- flux
|
| 16 |
-
pipeline_tag: image-to-image
|
| 17 |
-
library_name: diffusers
|
| 18 |
-
---
|
| 19 |
-
|
| 20 |
-
> **Note:** This is a repackaging of the [black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) model. Only the `flux2-dev.safetensors` file located in the root directory was removed, as it contained the same model as the one defined in the `transformer/` folder. Because of this, `diffusers` was loading the transformer twice, causing out-of-memory (OOM) errors. By removing this file, the duplicate loading was avoided and memory usage during inference was reduced from approximately **178 GB** to **110 GB**, enabling stable execution.
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
`FLUX.2 [dev]` is a 32 billion parameter rectified flow transformer capable of generating, editing and combining images based on text instructions.
|
| 27 |
-
For more information, please read our [blog post](https://bfl.ai/blog/flux-2).
|
| 28 |
-
|
| 29 |
-
# Key Features
|
| 30 |
-
1. State of the art in open text-to-image generation, single-reference editing and multi-reference editing.
|
| 31 |
-
2. No need for finetuning: character, object and style reference without additional training in one model.
|
| 32 |
-
4. Trained using guidance distillation, making `FLUX.2 [dev]` more efficient.
|
| 33 |
-
5. Open weights to drive new scientific research, and empower artists to develop innovative workflows.
|
| 34 |
-
6. Generated outputs can be used for personal, scientific, and commercial purposes, as described in the [FLUX \[dev\] Non-Commercial License](https://github.com/black-forest-labs/flux/blob/main/model_licenses/LICENSE-FLUX1-dev).
|
| 35 |
-
|
| 36 |
-
# Usage
|
| 37 |
-
We provide a reference implementation of `FLUX.2 [dev]`, as well as sampling code, in a dedicated [github repository](https://github.com/black-forest-labs/flux2).
|
| 38 |
-
Developers and creatives looking to build on top of `FLUX.2 [dev]` are encouraged to use this as a starting point.
|
| 39 |
-
|
| 40 |
-
`FLUX.2 [dev]` is also available in both [ComfyUI](https://github.com/comfyanonymous/ComfyUI) and [Diffusers](https://github.com/huggingface/diffusers).
|
| 41 |
-
|
| 42 |
-
### Using with diffusers 🧨
|
| 43 |
-
|
| 44 |
-
For local deployment on a consumer type graphics card, like an RTX 4090 or an RTX 5090, please see the [diffusers docs](https://github.com/black-forest-labs/flux2/blob/main/docs/flux2_dev_hf.md) on our GitHub page.
|
| 45 |
-
|
| 46 |
-
As an example, here's a way to load a 4-bit quantized model with a remote text-encoder on an RTX 4090:
|
| 47 |
-
|
| 48 |
-
```python
|
| 49 |
-
import torch
|
| 50 |
-
from diffusers import Flux2Pipeline
|
| 51 |
-
from diffusers.utils import load_image
|
| 52 |
-
from huggingface_hub import get_token
|
| 53 |
-
import requests
|
| 54 |
-
import io
|
| 55 |
-
|
| 56 |
-
repo_id = "diffusers/FLUX.2-dev-bnb-4bit" #quantized text-encoder and DiT. VAE still in bf16
|
| 57 |
-
device = "cuda:0"
|
| 58 |
-
torch_dtype = torch.bfloat16
|
| 59 |
-
|
| 60 |
-
def remote_text_encoder(prompts):
|
| 61 |
-
response = requests.post(
|
| 62 |
-
"https://remote-text-encoder-flux-2.huggingface.co/predict",
|
| 63 |
-
json={"prompt": prompts},
|
| 64 |
-
headers={
|
| 65 |
-
"Authorization": f"Bearer {get_token()}",
|
| 66 |
-
"Content-Type": "application/json"
|
| 67 |
-
}
|
| 68 |
-
)
|
| 69 |
-
prompt_embeds = torch.load(io.BytesIO(response.content))
|
| 70 |
-
|
| 71 |
-
return prompt_embeds.to(device)
|
| 72 |
-
|
| 73 |
-
pipe = Flux2Pipeline.from_pretrained(
|
| 74 |
-
repo_id, text_encoder=None, torch_dtype=torch_dtype
|
| 75 |
-
).to(device)
|
| 76 |
-
|
| 77 |
-
prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom."
|
| 78 |
-
|
| 79 |
-
#cat_image = load_image("https://huggingface.co/spaces/zerogpu-aoti/FLUX.1-Kontext-Dev-fp8-dynamic/resolve/main/cat.png")
|
| 80 |
-
image = pipe(
|
| 81 |
-
prompt_embeds=remote_text_encoder(prompt),
|
| 82 |
-
#image=[cat_image] #optional multi-image input
|
| 83 |
-
generator=torch.Generator(device=device).manual_seed(42),
|
| 84 |
-
num_inference_steps=50, #28 steps can be a good trade-off
|
| 85 |
-
guidance_scale=4,
|
| 86 |
-
).images[0]
|
| 87 |
-
|
| 88 |
-
image.save("flux2_output.png")
|
| 89 |
-
```
|
| 90 |
-
|
| 91 |
-
**Using the model in BF16 (Requires an H200)**
|
| 92 |
-
|
| 93 |
-
```python
|
| 94 |
-
import torch
|
| 95 |
-
from diffusers.pipelines.flux2.pipeline_flux2 import Flux2Pipeline
|
| 96 |
-
from transformers import Mistral3ForConditionalGeneration
|
| 97 |
-
from diffusers.models.transformers.transformer_flux2 import Flux2Transformer2DModel
|
| 98 |
-
from diffusers.models.autoencoders.autoencoder_kl_flux2 import AutoencoderKLFlux2
|
| 99 |
-
|
| 100 |
-
MODEL_ID = "Aquiles-ai/FLUX.2-dev"
|
| 101 |
-
|
| 102 |
-
text_encoder = Mistral3ForConditionalGeneration.from_pretrained(
|
| 103 |
-
MODEL_ID, subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda"
|
| 104 |
-
)
|
| 105 |
-
|
| 106 |
-
dit = Flux2Transformer2DModel.from_pretrained(
|
| 107 |
-
MODEL_ID, subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda"
|
| 108 |
-
)
|
| 109 |
-
|
| 110 |
-
vae = AutoencoderKLFlux2.from_pretrained(
|
| 111 |
-
MODEL_ID,
|
| 112 |
-
subfolder="vae",
|
| 113 |
-
torch_dtype=torch.bfloat16.to("cuda")
|
| 114 |
-
)
|
| 115 |
-
|
| 116 |
-
pipeline = Flux2Pipeline.from_pretrained(
|
| 117 |
-
MODEL_ID, text_encoder=text_encoder, transformer=dit, vae=vae, dtype=torch.bfloat16
|
| 118 |
-
).to(device="cuda")
|
| 119 |
-
|
| 120 |
-
prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom."
|
| 121 |
-
|
| 122 |
-
output = pipeline(
|
| 123 |
-
prompt=prompt,
|
| 124 |
-
num_inference_steps=50,
|
| 125 |
-
generator=torch.Generator(device="cuda").manual_seed(42),
|
| 126 |
-
guidance_scale=4,
|
| 127 |
-
).images[0]
|
| 128 |
-
|
| 129 |
-
output.save("flux2_output.png")
|
| 130 |
-
```
|
| 131 |
-
|
| 132 |
-
**Using the model with a quantized text encoder:** Even with the text encoder quantized, the model still has high memory requirements. While it may run on an **H100**, the available VRAM would be very tight, so it is recommended to use GPUs with **more than 85 GB of VRAM** to ensure stable execution and avoid OOM issues.
|
| 133 |
-
|
| 134 |
-
```python
|
| 135 |
-
import torch
|
| 136 |
-
from diffusers.pipelines.flux2.pipeline_flux2 import Flux2Pipeline
|
| 137 |
-
from transformers import Mistral3ForConditionalGeneration
|
| 138 |
-
from diffusers.models.transformers.transformer_flux2 import Flux2Transformer2DModel
|
| 139 |
-
from diffusers.models.autoencoders.autoencoder_kl_flux2 import AutoencoderKLFlux2
|
| 140 |
-
|
| 141 |
-
MODEL_ID = "Aquiles-ai/FLUX.2-dev"
|
| 142 |
-
|
| 143 |
-
MODEL_4BIT = "diffusers/FLUX.2-dev-bnb-4bit"
|
| 144 |
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
)
|
| 148 |
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
|
|
|
| 152 |
|
| 153 |
-
|
| 154 |
-
|
| 155 |
-
subfolder="vae",
|
| 156 |
-
torch_dtype=torch.bfloat16.to("cuda")
|
| 157 |
-
)
|
| 158 |
|
| 159 |
-
|
| 160 |
-
|
| 161 |
-
).to(device="cuda")
|
| 162 |
|
| 163 |
-
prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom."
|
| 164 |
|
| 165 |
-
|
| 166 |
-
prompt=prompt,
|
| 167 |
-
num_inference_steps=50,
|
| 168 |
-
generator=torch.Generator(device="cuda").manual_seed(42),
|
| 169 |
-
guidance_scale=4,
|
| 170 |
-
).images[0]
|
| 171 |
|
| 172 |
-
|
| 173 |
-
```
|
| 174 |
|
| 175 |
-
|
|
|
|
| 176 |
|
| 177 |
-
|
|
|
|
| 178 |
|
| 179 |
-
|
| 180 |
-
|
| 181 |
-
2. Post-training mitigation. Subsequently, we undertook multiple rounds of targeted fine-tuning to provide additional mitigation against potential abuse, including both text-to-image (T2I) and image-to-image (I2I) attacks. By inhibiting certain behaviors and suppressing certain concepts in the trained model, these techniques can help to prevent a user generating synthetic CSAM or NCII from a text prompt, or transforming an uploaded image into synthetic CSAM or NCII.
|
| 182 |
-
3. Ongoing evaluation. Throughout this process, we conducted multiple internal and external third-party evaluations of model checkpoints to identify further opportunities for mitigation. External third-party evaluations focused on eliciting CSAM and NCII through adversarial testing with (i) text-only prompts, (ii) a single uploaded reference image with text prompts, and (iii) multiple uploaded reference images with text prompts. Based on this feedback, we conducted further safety fine-tuning to produce our open-weight model (FLUX.2 [dev]).
|
| 183 |
-
4. Release decision. After safety fine-tuning and prior to release, we conducted a final third-party evaluation of the proposed release checkpoint, focused on T2I and I2I generation of synthetic CSAM and NCII, including a comparison with other open-weight T2I and I2I models (total prompts n≈2,800). The final FLUX.2 [dev] checkpoint demonstrated high resilience against violative inputs in complex generation and editing tasks, and demonstrated higher resilience than leading open-weight models across these risk categories. Based on these findings, we approved the release of the FLUX.2 Pro model via API and the release of the open-weight FLUX.2 [dev] model under a non-commercial license to support third-party research and development.
|
| 184 |
-
5. Inference filters. The repository for the FLUX.2 [dev] model includes filters for NSFW and IP-infringing content at input and output. Filters or manual review must be used with the model under the terms of the FLUX.2 [dev] Non-Commercial License. We may approach known deployers of the FLUX.2 [dev] model at random to verify that filters or manual review processes are in place. Additionally, we apply multiple filters to intercept text prompts, uploaded images, and output images on the API for FLUX.2 [pro]. We utilize both in-house and third-party supplied filters to prevent CSAM and NCII outputs, including filters provided by Hive and Microsoft. We provide filters for other categories of potentially harmful content, including gore, which can be adjusted by developers based on their specific risk profile and legitimate use cases.
|
| 185 |
-
6. Content provenance. Content provenance features can help users and platforms better identify, label, and interpret AI-generated content online. The inference code for FLUX.2 [dev] implements an example of pixel-layer watermarking, and this repository includes links to the Coalition for Content Provenance and Authenticity (C2PA) standard for metadata. The API for FLUX.2 Pro applies cryptographically-signed C2PA metadata to output content to indicate that images were produced with our model.
|
| 186 |
-
7. Policies. Use of our models and access to our API are governed by our FLUX [dev] Non-Commercial License (for our non-commercial open-weight users); Developer Terms of Service, Self-Hosted Commercial License Terms, and Usage Policy (for our commercial open-weight model users); and Developer Terms of Service, FLUX API Service Terms, and Usage Policy (for our API users). These prohibit the generation of unlawful content or the use of generated content for unlawful, defamatory, or abusive purposes. Developers and users must consent to these conditions to access the FLUX.2 [dev] model on Hugging Face.
|
| 187 |
-
8. Monitoring. We are monitoring for patterns of violative use after release. We continue to issue and escalate takedown requests to websites, services, or businesses that misuse our models. Additionally, we may ban users or developers who we detect intentionally and repeatedly violate our policies via the FLUX API. Additionally, we provide a dedicated email address (safety@blackforestlabs.ai) to solicit feedback from the community. We maintain a reporting relationship with organizations such as the Internet Watch Foundation and the National Center for Missing and Exploited Children, and welcome ongoing engagement with authorities, developers, and researchers to share intelligence about emerging risks and develop effective mitigations.
|
| 188 |
|
|
|
|
|
|
|
| 189 |
|
| 190 |
-
# License
|
| 191 |
-
This model falls under the [FLUX \[dev\] Non-Commercial License](https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt).
|
| 192 |
|
|
|
|
|
|
|
|
|
| 1 |
+
--- language: - en license: other license_name: flux-dev-non-commercial-license license_link: https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt extra_gated_prompt: >- By clicking "Agree", you agree to the [FLUX [dev] Non-Commercial License Agreement](https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt) and acknowledge the [Acceptable Use Policy](https://bfl.ai/legal/usage-policy). tags: - image-generation - image-editing - flux pipeline_tag: image-to-image library_name: diffusers --- <div align="center"> # ⚡ FLUX.2 [dev] — Optimized & Repackaged [](https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt) [](https://bfl.ai/blog/flux-2) [-brightgreen.svg?style=for-the-badge)](#-vram-optimization-breakthrough) [](https://github.com/huggingface/diffusers) [](#) --- ### 🚀 **Next-Gen Open Text-to-Image & Multi-Reference Editing Engine** *Capable of generating, editing, and combining high-fidelity images with single & multi-reference consistency — zero finetuning required.* [📖 Read Official Blog](https://bfl.ai/blog/flux-2) • [💻 GitHub Repository](https://github.com/black-forest-labs/flux2) • [🤗 Base Model HuggingFace](https://huggingface.co/black-forest-labs/FLUX.2-dev) --- </div> ## 💡 Key Architectural Fix & VRAM Optimization > [!IMPORTANT] > **Duplicate Weight Stripping (OOM Prevention)** > > This repository is an **optimized repackaging** of [`black-forest-labs/FLUX.2-dev`](https://huggingface.co/black-forest-labs/FLUX.2-dev). > - **🚨 The Issue:** The original repository contained a duplicate weight file (`flux2-dev.safetensors`) in the root directory alongside the identical weights in `transformer/`. This caused standard `diffusers` loading routines to initialize the 32B model **twice in VRAM**. > - **🛠️ The Fix:** The redundant root file was safely removed. > - **📈 Performance Gain:** Memory consumption during inference dropped dramatically from **~178 GB** down to **~110 GB**, eliminating out-of-memory (OOM) crashes and enabling stable execution on target hardware. | Metric | Original Repo | Repackaged Repo | Impact | | :--- | :---: | :---: | :---: | | **Root Transformer Duplicate** | Included (`flux2-dev.safetensors`) | ❌ Removed | Zero Redundant Weights | | **Diffusers Loading** | Dual Load (Bugged) | Single Load (Clean) | No Memory Leak / Spikes | | **Peak VRAM Usage** | ~178 GB | **~110 GB** | **38% VRAM Saved (OOM Fixed)** | | **Inference Stability** | Crashing / OOM | **Stable Execution** | ⚡ Ready for Production | --- ## 🎨 Model Teaser & Showcases <div align="center"> <table> <tr> <td align="center"><b>🎨 Text-to-Image Generation</b></td> <td align="center"><b>🪄 Image Editing & Fusion</b></td> </tr> <tr> <td><img src="./teaser_generation.png" alt="Teaser Generation" width="100%"/></td> <td><img src="./teaser_editing.png" alt="Teaser Editing" width="100%"/></td> </tr> </table> </div> --- ## 🔥 Key Features - **🏆 State of the Art Performance:** Superior open-weights benchmarks across text-to-image, single-reference editing, and complex multi-reference image composition. - **✨ Zero-Finetuning In-Context Consistency:** Seamlessly transfer character identity, physical objects, and artistic styles without any additional model training or LoRA fine-tuning. - **⚡ Guidance Distillation:** Optimized training architecture using guidance distillation, delivering faster generation steps and higher efficiency. - **🔬 Open Weights for Innovation:** Full model weights accessible to empower scientific research, custom artistic pipelines, and community development. - **📜 Flexible License Rights:** Generated outputs can be used freely for personal, scientific, and commercial purposes in compliance with the [FLUX [dev] Non-Commercial License](https://github.com/black-forest-labs/flux/blob/main/model_licenses/LICENSE-FLUX1-dev). --- ## 💻 Usage & Deployment We provide a reference implementation, sampling code, and ecosystem integration across **Diffusers** and **ComfyUI**. For complete setup guides, check out the official [GitHub Repository](https://github.com/black-forest-labs/flux2). > [!NOTE] > For local deployment on consumer GPUs (e.g., **RTX 4090** or **RTX 5090**), refer to the [Diffusers Hardware Docs](https://github.com/black-forest-labs/flux2/blob/main/docs/flux2_dev_hf.md). <details open> <summary><b>1️⃣ Option A: RTX 4090 Deployment (4-bit Quantized + Remote Text Encoder)</b></summary> ```python import torch from diffusers import Flux2Pipeline from diffusers.utils import load_image from huggingface_hub import get_token import requests import io # Quantized text-encoder and DiT. VAE remains in bf16 repo_id = "diffusers/FLUX.2-dev-bnb-4bit" device = "cuda:0" torch_dtype = torch.bfloat16 def remote_text_encoder(prompts): response = requests.post( "[https://remote-text-encoder-flux-2.huggingface.co/predict](https://remote-text-encoder-flux-2.huggingface.co/predict)", json={"prompt": prompts}, headers={ "Authorization": f"Bearer {get_token()}", "Content-Type": "application/json" } ) prompt_embeds = torch.load(io.BytesIO(response.content)) return prompt_embeds.to(device) pipe = Flux2Pipeline.from_pretrained( repo_id, text_encoder=None, torch_dtype=torch_dtype ).to(device) prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom." # cat_image = load_image("[https://huggingface.co/spaces/zerogpu-aoti/FLUX.1-Kontext-Dev-fp8-dynamic/resolve/main/cat.png](https://huggingface.co/spaces/zerogpu-aoti/FLUX.1-Kontext-Dev-fp8-dynamic/resolve/main/cat.png)") image = pipe( prompt_embeds=remote_text_encoder(prompt), # image=[cat_image], # optional multi-image input generator=torch.Generator(device=device).manual_seed(42), num_inference_steps=50, # 28 steps can be a good trade-off guidance_scale=4, ).images[0] image.save("flux2_output.png") `
|
| 2 |
+
Python
|
| 3 |
+
`import torch from diffusers.pipelines.flux2.pipeline_flux2 import Flux2Pipeline from transformers import Mistral3ForConditionalGeneration from diffusers.models.transformers.transformer_flux2 import Flux2Transformer2DModel from diffusers.models.autoencoders.autoencoder_kl_flux2 import AutoencoderKLFlux2 MODEL_ID = "Aquiles-ai/FLUX.2-dev" text_encoder = Mistral3ForConditionalGeneration.from_pretrained( MODEL_ID, subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda" ) dit = Flux2Transformer2DModel.from_pretrained( MODEL_ID, subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda" ) vae = AutoencoderKLFlux2.from_pretrained( MODEL_ID, subfolder="vae", torch_dtype=torch.bfloat16.to("cuda") ) pipeline = Flux2Pipeline.from_pretrained( MODEL_ID, text_encoder=text_encoder, transformer=dit, vae=vae, dtype=torch.bfloat16 ).to(device="cuda") prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom." output = pipeline( prompt=prompt, num_inference_steps=50, generator=torch.Generator(device="cuda").manual_seed(42), guidance_scale=4, ).images[0] output.save("flux2_output.png") `
|
| 4 |
+
[!WARNING]
|
| 5 |
+
Even with a quantized text encoder, high memory requirements remain. Execution on an **H100** may be tight on VRAM. **>85 GB VRAM** GPUs are strongly recommended for stability and to avoid OOM issues.
|
| 6 |
+
Python
|
| 7 |
+
`import torch from diffusers.pipelines.flux2.pipeline_flux2 import Flux2Pipeline from transformers import Mistral3ForConditionalGeneration from diffusers.models.transformers.transformer_flux2 import Flux2Transformer2DModel from diffusers.models.autoencoders.autoencoder_kl_flux2 import AutoencoderKLFlux2 MODEL_ID = "Aquiles-ai/FLUX.2-dev" MODEL_4BIT = "diffusers/FLUX.2-dev-bnb-4bit" text_encoder = Mistral3ForConditionalGeneration.from_pretrained( MODEL_4BIT, subfolder="text_encoder", torch_dtype=torch.bfloat16, device_map="cuda" ) dit = Flux2Transformer2DModel.from_pretrained( MODEL_ID, subfolder="transformer", torch_dtype=torch.bfloat16, device_map="cuda" ) vae = AutoencoderKLFlux2.from_pretrained( MODEL_ID, subfolder="vae", torch_dtype=torch.bfloat16.to("cuda") ) pipeline = Flux2Pipeline.from_pretrained( MODEL_ID, text_encoder=text_encoder, transformer=dit, vae=vae, dtype=torch.bfloat16 ).to(device="cuda") prompt = "Realistic macro photograph of a hermit crab using a soda can as its shell, partially emerging from the can, captured with sharp detail and natural colors, on a sunlit beach with soft shadows and a shallow depth of field, with blurred ocean waves in the background. The can has the text `BFL Diffusers` on it and it has a color gradient that start with #FF5733 at the top and transitions to #33FF57 at the bottom." output = pipeline( prompt=prompt, num_inference_steps=50, generator=torch.Generator(device="cuda").manual_seed(42), guidance_scale=4, ).images[0] output.save("flux2_output.png") `
|
| 8 |
+
## 🛡️ Safety & Risk Mitigation Framework
|
| 9 |
+
Black Forest Labs is committed to the responsible development and deployment of our models. Prior to releasing the FLUX.2 family of models, we evaluated and mitigated a number of risks in our model checkpoints and hosted services, including the generation of unlawful content such as child sexual abuse material (CSAM) and nonconsensual intimate imagery (NCII). We implemented a series of pre-release mitigations to help prevent misuse by third parties, with additional post-release mitigations to help address residual risks:
|
| 10 |
+
1.
|
| 11 |
+
**🛡️ Pre-training Mitigation:** We filtered pre-training data for multiple categories of “not safe for work” (NSFW) and known child sexual abuse material (CSAM) to help prevent a user generating unlawful content in response to text prompts or uploaded images. We have partnered with the **Internet Watch Foundation**, an independent nonprofit organization dedicated to preventing online abuse, to filter known CSAM from the training data.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
+
2.
|
| 14 |
+
**🎯 Post-training Mitigation:** Subsequently, we undertook multiple rounds of targeted fine-tuning to provide additional mitigation against potential abuse, including both text-to-image (T2I) and image-to-image (I2I) attacks. By inhibiting certain behaviors and suppressing certain concepts in the trained model, these techniques can help to prevent a user generating synthetic CSAM or NCII from a text prompt, or transforming an uploaded image into synthetic CSAM or NCII.
|
|
|
|
| 15 |
|
| 16 |
+
3.
|
| 17 |
+
**🔍 Ongoing Evaluation:** Throughout this process, we conducted multiple internal and external third-party evaluations of model checkpoints to identify further opportunities for mitigation. External third-party evaluations focused on eliciting CSAM and NCII through adversarial testing with:
|
| 18 |
+
-
|
| 19 |
+
*(i)* text-only prompts
|
| 20 |
|
| 21 |
+
-
|
| 22 |
+
*(ii)* a single uploaded reference image with text prompts
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
-
|
| 25 |
+
*(iii)* multiple uploaded reference images with text prompts.
|
|
|
|
| 26 |
|
|
|
|
| 27 |
|
| 28 |
+
Based on this feedback, we conducted further safety fine-tuning to produce our open-weight model (`FLUX.2 [dev]`).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
+
4.
|
| 31 |
+
**📋 Release Decision:** After safety fine-tuning and prior to release, we conducted a final third-party evaluation of the proposed release checkpoint, focused on T2I and I2I generation of synthetic CSAM and NCII, including a comparison with other open-weight T2I and I2I models (total prompts $n \approx 2,800$). The final `FLUX.2 [dev]` checkpoint demonstrated high resilience against violative inputs in complex generation and editing tasks, and demonstrated higher resilience than leading open-weight models across these risk categories. Based on these findings, we approved the release of the `FLUX.2 Pro` model via API and the release of the open-weight `FLUX.2 [dev]` model under a non-commercial license to support third-party research and development.
|
| 32 |
|
| 33 |
+
5.
|
| 34 |
+
**⚙️ Inference Filters:** The repository for the `FLUX.2 [dev]` model includes filters for NSFW and IP-infringing content at input and output. Filters or manual review must be used with the model under the terms of the FLUX.2 [dev] Non-Commercial License. We may approach known deployers of the `FLUX.2 [dev]` model at random to verify that filters or manual review processes are in place. Additionally, we apply multiple filters to intercept text prompts, uploaded images, and output images on the API for `FLUX.2 [pro]`. We utilize both in-house and third-party supplied filters to prevent CSAM and NCII outputs, including filters provided by **Hive** and **Microsoft**. We provide filters for other categories of potentially harmful content, including gore, which can be adjusted by developers based on their specific risk profile and legitimate use cases.
|
| 35 |
|
| 36 |
+
6.
|
| 37 |
+
**🏷️ Content Provenance:** Content provenance features can help users and platforms better identify, label, and interpret AI-generated content online. The inference code for `FLUX.2 [dev]` implements an example of pixel-layer watermarking, and this repository includes links to the Coalition for Content Provenance and Authenticity (**C2PA**) standard for metadata. The API for `FLUX.2 Pro` applies cryptographically-signed C2PA metadata to output content to indicate that images were produced with our model.
|
| 38 |
|
| 39 |
+
7.
|
| 40 |
+
**📄 Policies:** Use of our models and access to our API are governed by our [FLUX [dev] Non-Commercial License](https://www.google.com/url?sa=E&source=gmail&q=https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt) (for our non-commercial open-weight users); Developer Terms of Service, Self-Hosted Commercial License Terms, and Usage Policy (for our commercial open-weight model users); and Developer Terms of Service, FLUX API Service Terms, and Usage Policy (for our API users). These prohibit the generation of unlawful content or the use of generated content for unlawful, defamatory, or abusive purposes. Developers and users must consent to these conditions to access the `FLUX.2 [dev]` model on Hugging Face.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
8.
|
| 43 |
+
**👁️ Monitoring:** We are monitoring for patterns of violative use after release. We continue to issue and escalate takedown requests to websites, services, or businesses that misuse our models. Additionally, we may ban users or developers who we detect intentionally and repeatedly violate our policies via the FLUX API. Additionally, we provide a dedicated email address (`safety@blackforestlabs.ai`) to solicit feedback from the community. We maintain a reporting relationship with organizations such as the **Internet Watch Foundation** and the **National Center for Missing and Exploited Children**, and welcome ongoing engagement with authorities, developers, and researchers to share intelligence about emerging risks and develop effective mitigations.
|
| 44 |
|
|
|
|
|
|
|
| 45 |
|
| 46 |
+
## 📜 License
|
| 47 |
+
This model falls under the [FLUX [dev] Non-Commercial License](https://www.google.com/url?sa=E&source=gmail&q=https://huggingface.co/black-forest-labs/FLUX.2-dev/blob/main/LICENSE.txt).
|