Diffusers example for low vram?
#2
by bertbobson - opened
I have tried every guide on loading quantized models and cpu offloading for diffusers I found and nothing seems to work for this model.
Kinda sucks to be unable to run this on 24gb vram.
If there is a way, it'd be cool if you could give an example.
kthxbye
Thanks for letting us know. We're looking into it.
I'm not sure if this will be helpful, but there is a document written on how to convert and quantize from safetensors to gguf.
https://github.com/city96/ComfyUI-GGUF/tree/main/tools