j0no12 commited on
Commit
b489f14
·
verified ·
1 Parent(s): d92743b

Rewrite model card in plain language

Browse files
Files changed (1) hide show
  1. README.md +39 -41
README.md CHANGED
@@ -16,9 +16,9 @@ tags:
16
 
17
  # SolPix
18
 
19
- SolPix is a roughly 49M-parameter text-to-image flow transformer. It generates latents for the SANA 1.1 DC-AE, with text conditioning from Flan-T5 Base.
20
 
21
- ## Model
22
 
23
  | Setting | Value |
24
  |---|---|
@@ -38,18 +38,18 @@ SolPix is a roughly 49M-parameter text-to-image flow transformer. It generates l
38
  | Saved optimizer step | 210,000 |
39
  | Configured training schedule | 5,000,000 steps |
40
 
41
- The text encoder and autoencoder are external dependencies and are excluded from the generator's parameter count. `SolPixTransformer2D` predicts latent velocity; `AutoencoderDCSol` loads the matching Diffusers `AutoencoderDC` to decode sampled latents.
42
 
43
- ## Use
44
 
45
- Download this repository and install its requirements:
46
 
47
  ```bash
48
  python -m pip install -r requirements.txt
49
  python generate.py --prompt "A glass greenhouse in a quiet garden after rain" --output solpix.png
50
  ```
51
 
52
- To choose a local checkpoint, seed, and sampling settings:
53
 
54
  ```bash
55
  python generate.py \
@@ -59,144 +59,142 @@ python generate.py \
59
  --output ./solpix.png
60
  ```
61
 
62
- The helper downloads Flan-T5 Base and the pinned SANA DC-AE revision, then samples with Euler integration and classifier-free guidance. It produces a 512x512 image and uses CUDA when available. CPU inference is supported but slow.
63
 
64
- Sampling follows `x_t = (1 - t) x_clean + t noise`, integrating from `t=1` to `t=0`. Encoder and decoder identifiers and revisions are in `config.json`.
65
 
66
- ## Training
67
 
68
- The split was curated from [MONET v1.2.0](https://huggingface.co/datasets/jasperai/monet) with seed `20260924`: 174,603 training examples and a 9,300-example validation holdout. Training used pre-encoded SANA F32C32 latents and Flan-T5 Base caption states.
69
 
70
- Sources included CC12M, CommonCatalog-CC-BY, COYO, Diffusion-Aesthetic-4K, LAION, and synthetic captions from Flux Klein, Flux Schnell, and Z-Image. Curation applied resolution, aesthetic, NSFW, watermark, and near-duplicate filters. Source records carried CC BY 4.0, Apache 2.0, Google permissive, and MIT license labels. Those labels describe upstream records, not a new license for their contents. Images and dataset shards are not redistributed here.
71
 
72
- The Windows v1.0 continuation used BF16 on one RTX 3080 Ti, with batch size 4 and gradient accumulation 16. The published checkpoint is step 210,000. The documented 1.1 continuation uses the same split and targets step 300,000.
73
 
74
- ## Evaluation and limits
75
 
76
- No formal image-quality or prompt-following benchmark accompanies this checkpoint. The gallery shows generated outputs, not a held-out quality estimate. Expect composition errors, artifacts, and weak rendering of text or fine detail.
77
 
78
- The model has no built-in safety classifier. Dataset filtering does not remove all source biases or unwanted associations. Flan-T5 and SANA DC-AE have their own licenses and usage terms.
79
 
80
- ## Samples
81
 
 
82
 
83
- These are 15 individual images generated by the released SolPix checkpoint, not external examples. Each was sampled at 512×512 with 32 Euler steps and guidance scale 3.5 using the pinned SANA DC-AE decoder. The image files, prompts, seeds, and SHA-256 values are in `samples/`.
84
 
85
  ### Sample 01
86
 
87
  ![SolPix sample 01 - Three Black men sharing french fries at a neighborhood diner, candid documentary photography.](samples/sample_01.png)
88
 
89
- **Prompt:** Three Black men sharing french fries at a neighborhood diner, candid documentary photography.
90
  Seed: 260926
91
 
92
  ### Sample 02
93
 
94
  ![SolPix sample 02 - A red fox standing in fresh snow beneath pine trees at winter dawn, wildlife photography.](samples/sample_02.png)
95
 
96
- **Prompt:** A red fox standing in fresh snow beneath pine trees at winter dawn, wildlife photography.
97
  Seed: 260927
98
 
99
  ### Sample 03
100
 
101
  ![SolPix sample 03 - A glass greenhouse filled with ferns after rain, soft natural light, botanical photograph.](samples/sample_03.png)
102
 
103
- **Prompt:** A glass greenhouse filled with ferns after rain, soft natural light, botanical photograph.
104
  Seed: 260928
105
 
106
  ### Sample 04
107
 
108
  ![SolPix sample 04 - A handmade cobalt blue teapot on a pale stone table, clean studio product photograph.](samples/sample_04.png)
109
 
110
- **Prompt:** A handmade cobalt blue teapot on a pale stone table, clean studio product photograph.
111
  Seed: 260929
112
 
113
  ### Sample 05
114
 
115
  ![SolPix sample 05 - A white sailboat crossing a calm blue bay at golden hour, fine art landscape photograph.](samples/sample_05.png)
116
 
117
- **Prompt:** A white sailboat crossing a calm blue bay at golden hour, fine art landscape photograph.
118
  Seed: 260930
119
 
120
  ### Sample 06
121
 
122
  ![SolPix sample 06 - An orange cat curled on a wooden chair in a sunlit bookshop, cozy editorial photograph.](samples/sample_06.png)
123
 
124
- **Prompt:** An orange cat curled on a wooden chair in a sunlit bookshop, cozy editorial photograph.
125
  Seed: 260931
126
 
127
  ### Sample 07
128
 
129
  ![SolPix sample 07 - A small street cafe reflected in wet pavement at night, warm window light, city photograph.](samples/sample_07.png)
130
 
131
- **Prompt:** A small street cafe reflected in wet pavement at night, warm window light, city photograph.
132
  Seed: 260932
133
 
134
  ### Sample 08
135
 
136
  ![SolPix sample 08 - A wooden lighthouse on a rocky coast under a cloudy sky, atmospheric landscape photograph.](samples/sample_08.png)
137
 
138
- **Prompt:** A wooden lighthouse on a rocky coast under a cloudy sky, atmospheric landscape photograph.
139
  Seed: 260933
140
 
141
  ### Sample 09
142
 
143
  ![SolPix sample 09 - A bowl of ripe peaches on a kitchen counter, morning light, natural still life photograph.](samples/sample_09.png)
144
 
145
- **Prompt:** A bowl of ripe peaches on a kitchen counter, morning light, natural still life photograph.
146
  Seed: 260934
147
 
148
  ### Sample 10
149
 
150
  ![SolPix sample 10 - A snow-covered cabin among tall pine trees at blue hour, quiet winter landscape photograph.](samples/sample_10.png)
151
 
152
- **Prompt:** A snow-covered cabin among tall pine trees at blue hour, quiet winter landscape photograph.
153
  Seed: 260935
154
 
155
  ### Sample 11
156
 
157
  ![SolPix sample 11 - A baker placing fresh bread on a cooling rack in a bright kitchen, documentary photograph.](samples/sample_11.png)
158
 
159
- **Prompt:** A baker placing fresh bread on a cooling rack in a bright kitchen, documentary photograph.
160
  Seed: 260936
161
 
162
  ### Sample 12
163
 
164
  ![SolPix sample 12 - A goldfinch perched on a thin branch among spring blossoms, close-up wildlife photograph.](samples/sample_12.png)
165
 
166
- **Prompt:** A goldfinch perched on a thin branch among spring blossoms, close-up wildlife photograph.
167
  Seed: 260937
168
 
169
  ### Sample 13
170
 
171
  ![SolPix sample 13 - A red bicycle leaning against a brick wall on a leafy neighborhood street, lifestyle photograph.](samples/sample_13.png)
172
 
173
- **Prompt:** A red bicycle leaning against a brick wall on a leafy neighborhood street, lifestyle photograph.
174
  Seed: 260938
175
 
176
  ### Sample 14
177
 
178
  ![SolPix sample 14 - A lemon cake with a slice cut out on a ceramic plate, bright tabletop food photograph.](samples/sample_14.png)
179
 
180
- **Prompt:** A lemon cake with a slice cut out on a ceramic plate, bright tabletop food photograph.
181
  Seed: 260939
182
 
183
  ### Sample 15
184
 
185
  ![SolPix sample 15 - A small observatory beneath a clear star-filled sky, distant mountains, night landscape photograph.](samples/sample_15.png)
186
 
187
- **Prompt:** A small observatory beneath a clear star-filled sky, distant mountains, night landscape photograph.
188
  Seed: 260940
189
 
 
190
 
191
- ## Files
192
-
193
- - `step_00210000.pt`: EMA and raw weights, optimizer state, configuration, and training arguments.
194
- - `solpix/`: transformer, decoder adapter, configuration, data, and training components.
195
- - `generate.py`: prompt-to-image helper.
196
- - `train.py` and `sample_latents.py`: training and latent-sampling entry points.
197
- - `samples/`: 15 generated PNGs and their metadata.
198
- - `config.json`: architecture and external-model manifest.
199
 
200
  ## License
201
 
202
- Repository code, checkpoint weights, configuration, card, and supplied banner are licensed under [Apache 2.0](LICENSE). See [NOTICE](NOTICE) for attribution. This license does not relicense the upstream datasets, Flan-T5, or SANA DC-AE.
 
16
 
17
  # SolPix
18
 
19
+ SolPix generates images from text with a flow transformer of roughly 49M parameters. The transformer works in the latent space of the SANA 1.1 DC-AE, and Flan-T5 Base encodes the prompt. The included helper produces 512x512 images.
20
 
21
+ ## Generator and encoders
22
 
23
  | Setting | Value |
24
  |---|---|
 
38
  | Saved optimizer step | 210,000 |
39
  | Configured training schedule | 5,000,000 steps |
40
 
41
+ The 49M count covers the generator. It excludes the frozen text encoder and autoencoder, which are separate dependencies. `SolPixTransformer2D` predicts latent velocity, and `AutoencoderDCSol` loads the matching Diffusers `AutoencoderDC` to decode the sampled latents.
42
 
43
+ ## Generate an image
44
 
45
+ Download this repository and install `requirements.txt`, then run the helper with a prompt:
46
 
47
  ```bash
48
  python -m pip install -r requirements.txt
49
  python generate.py --prompt "A glass greenhouse in a quiet garden after rain" --output solpix.png
50
  ```
51
 
52
+ You can also set the local checkpoint, seed, and sampling settings:
53
 
54
  ```bash
55
  python generate.py \
 
59
  --output ./solpix.png
60
  ```
61
 
62
+ The helper downloads Flan-T5 Base and the pinned SANA DC-AE revision. It samples with Euler integration and classifier-free guidance, using CUDA if available. CPU inference works but is slow.
63
 
64
+ The flow path is `x_t = (1 - t) x_clean + t noise`; sampling integrates from `t=1` to `t=0`. `config.json` records the encoder and decoder identifiers and their revisions.
65
 
66
+ ## Training data and run
67
 
68
+ The data split comes from [MONET v1.2.0](https://huggingface.co/datasets/jasperai/monet), curated with seed `20260924`. It has 174,603 training examples and a validation holdout of 9,300 examples. Training used pre-encoded SANA F32C32 image latents and Flan-T5 Base caption states.
69
 
70
+ The sources include CC12M and CommonCatalog-CC-BY, COYO, Diffusion-Aesthetic-4K, and LAION. Synthetic captions come from Flux Klein, Flux Schnell, and Z-Image. Curation filters cover resolution and aesthetics, NSFW content, watermarks, and near duplicates.
71
 
72
+ Upstream records carry CC BY 4.0, Apache 2.0, Google permissive, and MIT license labels. Those labels describe the source records; they don't grant a new license for the contents. This repository doesn't redistribute the images or dataset shards.
73
 
74
+ The Windows v1.0 continuation ran in BF16 on one RTX 3080 Ti, with batch size 4 and gradient accumulation 16. The released checkpoint is step 210,000. The documented 1.1 continuation keeps the same split and targets step 300,000.
75
 
76
+ ## Results and limits
77
 
78
+ The released checkpoint has no formal image-quality or prompt-following benchmark. The gallery lets you inspect generated outputs, but it doesn't provide a held-out quality estimate. Expect composition errors and artifacts, with weak rendering of text or fine detail.
79
 
80
+ SolPix has no built-in safety classifier. Filtering the training data doesn't remove all source biases or unwanted associations. Flan-T5 and SANA DC-AE also have their own licenses and usage terms.
81
 
82
+ ## Generated samples
83
 
84
+ These 15 images come from the released SolPix checkpoint. Each uses 512×512 resolution, 32 Euler steps, and guidance scale 3.5 with the pinned SANA DC-AE decoder. `samples/` contains the image files and records their prompts, seeds, and SHA-256 values.
85
 
86
  ### Sample 01
87
 
88
  ![SolPix sample 01 - Three Black men sharing french fries at a neighborhood diner, candid documentary photography.](samples/sample_01.png)
89
 
90
+ Prompt: Three Black men sharing french fries at a neighborhood diner, candid documentary photography.
91
  Seed: 260926
92
 
93
  ### Sample 02
94
 
95
  ![SolPix sample 02 - A red fox standing in fresh snow beneath pine trees at winter dawn, wildlife photography.](samples/sample_02.png)
96
 
97
+ Prompt: A red fox standing in fresh snow beneath pine trees at winter dawn, wildlife photography.
98
  Seed: 260927
99
 
100
  ### Sample 03
101
 
102
  ![SolPix sample 03 - A glass greenhouse filled with ferns after rain, soft natural light, botanical photograph.](samples/sample_03.png)
103
 
104
+ Prompt: A glass greenhouse filled with ferns after rain, soft natural light, botanical photograph.
105
  Seed: 260928
106
 
107
  ### Sample 04
108
 
109
  ![SolPix sample 04 - A handmade cobalt blue teapot on a pale stone table, clean studio product photograph.](samples/sample_04.png)
110
 
111
+ Prompt: A handmade cobalt blue teapot on a pale stone table, clean studio product photograph.
112
  Seed: 260929
113
 
114
  ### Sample 05
115
 
116
  ![SolPix sample 05 - A white sailboat crossing a calm blue bay at golden hour, fine art landscape photograph.](samples/sample_05.png)
117
 
118
+ Prompt: A white sailboat crossing a calm blue bay at golden hour, fine art landscape photograph.
119
  Seed: 260930
120
 
121
  ### Sample 06
122
 
123
  ![SolPix sample 06 - An orange cat curled on a wooden chair in a sunlit bookshop, cozy editorial photograph.](samples/sample_06.png)
124
 
125
+ Prompt: An orange cat curled on a wooden chair in a sunlit bookshop, cozy editorial photograph.
126
  Seed: 260931
127
 
128
  ### Sample 07
129
 
130
  ![SolPix sample 07 - A small street cafe reflected in wet pavement at night, warm window light, city photograph.](samples/sample_07.png)
131
 
132
+ Prompt: A small street cafe reflected in wet pavement at night, warm window light, city photograph.
133
  Seed: 260932
134
 
135
  ### Sample 08
136
 
137
  ![SolPix sample 08 - A wooden lighthouse on a rocky coast under a cloudy sky, atmospheric landscape photograph.](samples/sample_08.png)
138
 
139
+ Prompt: A wooden lighthouse on a rocky coast under a cloudy sky, atmospheric landscape photograph.
140
  Seed: 260933
141
 
142
  ### Sample 09
143
 
144
  ![SolPix sample 09 - A bowl of ripe peaches on a kitchen counter, morning light, natural still life photograph.](samples/sample_09.png)
145
 
146
+ Prompt: A bowl of ripe peaches on a kitchen counter, morning light, natural still life photograph.
147
  Seed: 260934
148
 
149
  ### Sample 10
150
 
151
  ![SolPix sample 10 - A snow-covered cabin among tall pine trees at blue hour, quiet winter landscape photograph.](samples/sample_10.png)
152
 
153
+ Prompt: A snow-covered cabin among tall pine trees at blue hour, quiet winter landscape photograph.
154
  Seed: 260935
155
 
156
  ### Sample 11
157
 
158
  ![SolPix sample 11 - A baker placing fresh bread on a cooling rack in a bright kitchen, documentary photograph.](samples/sample_11.png)
159
 
160
+ Prompt: A baker placing fresh bread on a cooling rack in a bright kitchen, documentary photograph.
161
  Seed: 260936
162
 
163
  ### Sample 12
164
 
165
  ![SolPix sample 12 - A goldfinch perched on a thin branch among spring blossoms, close-up wildlife photograph.](samples/sample_12.png)
166
 
167
+ Prompt: A goldfinch perched on a thin branch among spring blossoms, close-up wildlife photograph.
168
  Seed: 260937
169
 
170
  ### Sample 13
171
 
172
  ![SolPix sample 13 - A red bicycle leaning against a brick wall on a leafy neighborhood street, lifestyle photograph.](samples/sample_13.png)
173
 
174
+ Prompt: A red bicycle leaning against a brick wall on a leafy neighborhood street, lifestyle photograph.
175
  Seed: 260938
176
 
177
  ### Sample 14
178
 
179
  ![SolPix sample 14 - A lemon cake with a slice cut out on a ceramic plate, bright tabletop food photograph.](samples/sample_14.png)
180
 
181
+ Prompt: A lemon cake with a slice cut out on a ceramic plate, bright tabletop food photograph.
182
  Seed: 260939
183
 
184
  ### Sample 15
185
 
186
  ![SolPix sample 15 - A small observatory beneath a clear star-filled sky, distant mountains, night landscape photograph.](samples/sample_15.png)
187
 
188
+ Prompt: A small observatory beneath a clear star-filled sky, distant mountains, night landscape photograph.
189
  Seed: 260940
190
 
191
+ ## Repository files
192
 
193
+ - `step_00210000.pt` contains EMA and raw weights, optimizer state, configuration, and training arguments.
194
+ - `solpix/` has the transformer and decoder adapter, along with configuration, data, and training components.
195
+ - `generate.py` turns a prompt into an image. `train.py` and `sample_latents.py` are the training and latent-sampling entry points.
196
+ - `samples/` holds the 15 generated PNGs and their metadata. `config.json` records the architecture and external-model manifest.
 
 
 
 
197
 
198
  ## License
199
 
200
+ [Apache 2.0](LICENSE) covers the repository code and checkpoint weights, as well as the configuration, card, and supplied banner. [NOTICE](NOTICE) contains the attribution. The upstream datasets, Flan-T5, and SANA DC-AE keep their own licenses.