Can we adapt safe stable diffusion to ohter versions, e.g., SDV2.0 and SDXL?
The current safe stable diffusion shares weights with the Stable Diffusion v1.5. How to use stable diffusion in other versions?
Hi, in general, safe stable diffusion relies on safety guidance using classifier-free guidance. Thus it is parameter-free and our comment "safe stable diffusion shares weights with the Stable Diffusion v1.5" may be a bit confusing. The pipeline here allows you to just place the SDV2.0 checkpoint path and it should work out of the box. But it can also be adapted to other diffusion model architectures. You can find various models with safety guidance implemented here: https://github.com/ml-research/i2p/tree/main/mitigation and an overview of their I2P benchmark results here: https://arxiv.org/abs/2305.18398
E.g.:
DeepFloyd IF: https://github.com/ml-research/i2p/blob/main/mitigation/safe_if.py
AltDiffusion: https://github.com/ml-research/i2p/blob/main/mitigation/safe_alt.py
Paella: https://github.com/ml-research/i2p/blob/main/mitigation/safe_paella.py
It could also be applied to SDXL but it is not contained in the repo yet. You could either implement it yourself following the other implementations available or wait until we add it. This could be soonish since as far as I know we already used it internally. I will double-check and keep you posted.
Best,
Patrick