File size: 887 Bytes
ffa25a3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | ---
library_name: diffusers
---
`diffusers`-native checkpoint for the [google/diffusiongemma-26B-A4B-it](https://huggingface.co/google/diffusiongemma-26B-A4B-it) discrete diffusion LLM.
You can use the model as follows. Note that you need `transformers>=5.12.0` for the underlying `DiffusionGemmaForBlockDiffusion` model.
```python
import torch
from diffusers import DiffusionGemmaPipeline
pipe = DiffusionGemmaPipeline.from_pretrained(
"dg845/DiffusionGemma-diffusers",
torch_dtype=torch.bfloat16,
)
pipe.to("cuda")
# Compile the decoder model for faster inference
pipe.model.model.decoder = torch.compile(pipe.model.model.decoder, mode="reduce-overhead"),
output = pipe(
prompt="Why is the sky blue?",
gen_length=256,
num_inference_steps=48,
cache_implementation="static",
generator=torch.Generator("cuda").manual_seed(42),
)
print(output.texts[0])
``` |