dg845 commited on
Commit
ffa25a3
·
verified ·
1 Parent(s): 760cd1d

Create README.md

Browse files

Add initial description for repo

Files changed (1) hide show
  1. README.md +29 -0
README.md ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: diffusers
3
+ ---
4
+ `diffusers`-native checkpoint for the [google/diffusiongemma-26B-A4B-it](https://huggingface.co/google/diffusiongemma-26B-A4B-it) discrete diffusion LLM.
5
+
6
+ You can use the model as follows. Note that you need `transformers>=5.12.0` for the underlying `DiffusionGemmaForBlockDiffusion` model.
7
+
8
+ ```python
9
+ import torch
10
+ from diffusers import DiffusionGemmaPipeline
11
+
12
+ pipe = DiffusionGemmaPipeline.from_pretrained(
13
+ "dg845/DiffusionGemma-diffusers",
14
+ torch_dtype=torch.bfloat16,
15
+ )
16
+ pipe.to("cuda")
17
+
18
+ # Compile the decoder model for faster inference
19
+ pipe.model.model.decoder = torch.compile(pipe.model.model.decoder, mode="reduce-overhead"),
20
+
21
+ output = pipe(
22
+ prompt="Why is the sky blue?",
23
+ gen_length=256,
24
+ num_inference_steps=48,
25
+ cache_implementation="static",
26
+ generator=torch.Generator("cuda").manual_seed(42),
27
+ )
28
+ print(output.texts[0])
29
+ ```