Instructions to use ByteDance/SDXL-Lightning with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ByteDance/SDXL-Lightning with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("ByteDance/SDXL-Lightning", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps
- Draw Things
- DiffusionBee
Merging lightning issue
How do I merge lightning with an SDXL checkpoint? I tried using kohya but I got errors like these:
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_out_0.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_q.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_attn2_to_v.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_0_proj.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_0_ff_net_2.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_1_attn1_to_k.lora_down.weight sdxl_merge_lora.py:84
INFO no module found for LoRA weight: lora_unet_up_blocks_1_attentions_2_transformer_blocks_1_attn1_to_out_0.lora_down.weight sdxl_merge_lora.py:84
That script has bug.
https://github.com/kohya-ss/sd-scripts/issues/972
Just use ComfyUI. It is way easier:
https://comfyanonymous.github.io/ComfyUI_examples/model_merging/