Instructions to use Pinkstack/LiliPixel-V1-190M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Pinkstack/LiliPixel-V1-190M with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Pinkstack/LiliPixel-V1-190M", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("Pinkstack/LiliPixel-V1-190M", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]LiliPixel-V1 (16x16 / 32x32 / 64x64)
This model release is a from-scratch open weight text-to-pixel-art diffusion transformer (191M trainable parameters) that generates pixel "art"/sprite style images at native 16x16, 32x32 or 64x64 resolution. (64 pixel res also has realism toggle) Caption tokens and image tokens share one sequence with full self-attention (single-stream DiT). Text is encoded by a frozen MongoDB/voyage-4-nano (~346M parameters, not included in the weights, downloaded automatically on first use). The model needs roughly just 1GB of memory with lossless BF16 inference.
Model details
| Layers / width / heads | 24 / 768 / 16 |
| Patch size | 1 for 16x16 and 32x32, 2 for 64x64 (shared transformer, separate patch-in/out layers) |
| Pixels | RGBA, values in [-1, 1] |
| Text | frozen encoder features (max 96 tokens, longer prompts are truncated) projected to model width; learned null caption for CFG |
| Objective | rectified flow matching, logit-normal timesteps, velocity target |
| Sampler | Euler, 30/50 steps, CFG 3.0 by default |
| Generation | You should generate at least 4 images at once and select the best looking image |
| Weights | The weights included are HF compatible and are EMA weights |
How to prompt
warning: 16x16 and 32x32 are always with a transparent background, 64x64 is never transparent.
- 16x16 / 32x32 sprites: describe the object and end with
, transparent background. Training captions for images with transparent pixels got this exact phrase appended, so use the same wording. - 64x64 scenes:
a green turtle swimming above a coral reef on deep ocean background(subject, action,on <colour> background). add "realistic photo," at the start of the prompt to enable realism mode. - 64x64 descriptive prompts: the model also saw longer captions describing a subject and its composition.
- Keep your prompts simple: Think of it and explain simply. Very long prompts won't work as well, and anything past 96 tokens is cut off.
- Transparency was only seen at 16x16 and 32x32; 64x64 outputs are opaque unless prompted otherwise, do not use 64x64 for transparent backgrounds, 64x64 is RGB only, not RGBA
Quickstart example
Requires transformers>=4.56 and trust_remote_code=True (the text encoder uses custom code; internet needed on first load).
import torch
from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
"Pinkstack/LiliPixel-V1-190M",
custom_pipeline="Pinkstack/LiliPixel-V1-190M",
trust_remote_code=True, # must be true
torch_dtype=torch.bfloat16,
).to("cuda")
image = pipe(
prompt="cute border collie dog with a purple colar, transparent background", # end the prompt with ", transparent background" for 16,32 pixel sprites, start it with "realistic photo, " for realism images at 64x64.
height=32, width=32, # 16, 32 or 64
num_inference_steps=30, # 50 for 64 pixel images, 30 is quicker but may look worse
guidance_scale=3.0, # 3.0 is used for the banner image, 3.5 or higher is better for more specific prompts eg; "right pointing green arrow"
generator=torch.Generator("cuda").manual_seed(96),
).images[0]
image.save("LiliPixel-image.png")
ComfyUI
LiliPixel is a custom diffusers pipeline with its own text encoder, so ComfyUI's built-in checkpoint loaders can't open it. A single-file custom node wraps the pipeline instead.
- Download
comfyui-workflow/ComfyUI_LiliPixel.pyintoComfyUI/custom_nodes/. - Install the requirements in ComfyUI's Python environment:
pip install -U diffusers "transformers>=4.56" accelerate - Restart ComfyUI and drag
comfyui-workflow/LiliPixel_V1.jsononto the canvas.
The first run needs internet to download the model and the text encoder.
Using the workflow
- Type a short prompt in LiliPixel Generate and pick a resolution (16, 32 or 64).
- With
mode = auto, 16/32 prompts get, transparent backgroundappended for you. At 64, userealisticmode to prependrealistic photo,. steps = 0picks 30 steps for 16/32 and 50 for 64. CFG defaults to 3.0, so raise it to 3.5+ for specific prompts.- Batch size defaults to 4, with each image using
seed + i. Use 8-16 at 64x64, then pick the best. - Previews are upscaled with nearest-neighbour to 512x512 so pixels stay sharp. The Save Image node is muted by default, and Ctrl+M enables it.
- Sprites are saved as transparent PNGs.
alpha_cutoff(default 0.5) gives crisp edges, and 0 keeps soft alpha.
Node outputs
| Output | Description |
|---|---|
image |
RGB, transparency flattened onto the chosen background colour |
image_rgba |
RGBA with alpha preserved (used by the example workflow) |
mask |
Alpha channel, 1 = opaque |
Limitations
- LiliPixel has been trained on a limited amount of pictures and it has a limited parameter size, do not expect the model to generate complex scenes or niche objects/places, for example instead of prompting "Blahaj" prompt "Cute light blue colored shark plushy".
- In 64x64 mode, the model has a higher chance of hallucinating subjects. Ideally bulk-generate 8-16 images at once and select the best looking image.
- As the model is small, ideally generate multiple images with the same prompt across different seeds and select the best looking image, generating images on GPU or even CPU should be near-instant.
- at 16x16 more than 1 subject is complicated for the model, at 32x32 more than 3 subjects (eg; three berries) is complicated for the model. at 64x64 complex scenes may become incoherent and inconsistent.
- the model has not been trained to generate text.
Training
Trained from scratch for 132000 steps (AdamW, peak LR 0.0001, cosine decay, EMA 0.999; the released weights are the EMA weights). Each step draws one batch from one source, with different sampling probabilities for each data source and the following batch sizes (128, 32, 32, 32) | (16x16, 32x32, 64x64 scenes, 64x64 LiliPixel-710K). Captions are randomly dropped 10% of the time for classifier-free guidance.
Training data
- 16x16 sprites: Angais/pixel-art-atlas-1.4m (CC0)
- 32x32 sprites: azyg0s/pixelart-172k (MIT, see the dataset card for the copyright notice)
- 64x64 scenes: Chan-Y/pixelart-308k (CC BY 4.0, by Chan-Y; synthetic FLUX.2-klein-4B images with Gemma-written captions), smart-downscaled from 256x256 to 64x64 with palette merging and block voting, ", pixel art" stripped from captions.
- 64x64 pixel art: Pinkstack/LiliPixel-710K-Flux-64pixelart
Captions come from
caption_composition, falling back tocaption_entity; rows with neither were kept as unconditional examples, capped at 10% of the dataset.
For all sources, transparent background was appended to captions of images with at least 10% transparent pixels. 64x64 never had transparent pixels.
- Downloads last month
- -
