import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("8BitStudio/Aniimage-2", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]
Aniimage-2
Aniimage-2 is the second public model in the Aniimage series. The model is a 512x512 anime image generation model trained from scratch using a UNet + VAE + CLIP architecture with 441.9 million parameters. Aniimage-2 has been trained on 1.2M anime images from Danbooru and other sources. It is not based on any existing models, the UNet is trained from scratch.
Model Details
| Resolution | 512x512 |
| Architecture | Latent Diffusion (UNet + VAE + CLIP) |
| Parameters | 441.9M |
| Training Steps | 70,000 |
| Batch Size | 64 |
| Dataset | ~1.2 million curated anime images from Danbooru |
| GPU | NVIDIA RTX 5060 Ti 16GB |
| Scheduler | DPM++ SDE |
| Step count | 50 using DPM++ SDE |
Requirements
- GPU: ~6 GB VRAM minimum (recommend 8+ GB)
- CPU: ~7 GB RAM. Image generation is extremely slow on CPU. (recommend 10+ GB)
Prompting
Aniimage uses plain text captions as well as Danbooru tags.
An example of acceptable captions:
"A smiling anime girl with red hair and a school uniform"
"1girl, solo, smile, red_hair, school_uniform, anime_coloring"
Long prompts and enhanced prompts usually result in worse overall results. Try to keep your prompts short.
Although you can use Danbooru tags, you will get the best results with plain text captions
Adding a negative prompt is required for a good output. The one used during inference in the preview image is "low quality, ugly, blurry, distorted, deformed, bad anatomy, bad proportions, extra limbs, missing limbs, watermark, text, signature, washed out, flat colors, manga panel, disfigured, poorly drawn, jpeg artifacts, cropped, out of frame"
For the best quality, use DPM++ SDE at 50 steps.
Important: If you do not include “NSFW” in the negative prompt, the model may generate NSFW images
Run
Capabilities
Aniimage-2 has significantly higher capabilities than Aniimage-1. Especially when it comes to backgrounds.
|
|
| Aniimage-1 | Aniimage-2 |
| Prompt: Softly lit forest at midday, peaceful atmosphere, no people. | |
Aniimage-2 produces male characters much more reliably than Aniimage-1.
|
|
| Aniimage-1 | Aniimage-2 |
| Prompt: An anime boy with black hair and green eyes. | |
Limitations
Aniimage-2 is bad at generating hands. It may generate hands but they often melt into surfaces if they are similar colors and usually have more or fewer than five fingers. The model generates male characters less reliably than female characters, it may take many images to find a good image like above.
What's Next
Aniimage-3 is currently in the works and will have an updated parameters count, 594.8M versus 441.9M, a 34.6% increase in parameter count. The model will also have 2 million unique images, 66.7% more than the 1.2 million Aniimage-2 has. Aniimage-2 used incorrect Min-SNR loss weighting with its v-prediction objective, causing near-pure-noise timesteps to receive excessive emphasis. This has been corrected for Aniimage-3. Aniimage-3 also uses a much higher-quality dataset with improved filtering. More than 100,000 low-quality images from the Aniimage-2 dataset were removed.
License
Apache 2.0
- Downloads last month
- -