---
license: apache-2.0
tags:
  - biology
  - diffusion
  - single-cell
  - transcriptomics
  - microscopy
  - image-to-rna
  - conditional-generation
  - scgpt
  - vit
datasets:
  - altoslabs/scGeneScope
pipeline_tag: feature-extraction
model-index:
  - name: PhenoSeq
    results:
      - task:
          type: feature-extraction
          name: Image-conditioned RNA-seq generation
        metrics:
          - type: loss
            value: 0.1683
            name: Validation MSE Loss (epoch 87)
---

# PhenoSeq: Image-Conditioned Diffusion for Single-Cell Transcriptomics

[![HuggingFace](https://img.shields.io/badge/🤗%20HuggingFace-Sentinal4D/PhenoSeq-yellow)](https://huggingface.co/Sentinal4D/PhenoSeq)
[![Paper](https://img.shields.io/badge/Paper-OpenReview-blue)](https://openreview.net/forum?id=ACHMa1e8J1)
[![GitHub](https://img.shields.io/badge/GitHub-reednaidoo/PhenoSeq-black?logo=github)](https://github.com/reednaidoo/PhenoSeq)

PhenoSeq is a Gaussian diffusion model that **generates scGPT RNA-seq embeddings conditioned on ViT-L microscopy imaging features**. Given fluorescence microscopy images of a cell or well, it predicts a 512-dimensional scGPT embedding representing the transcriptomic state of individual cells — enabling image-to-transcriptome translation at single-cell resolution.

## Model summary

| Property | Value |
|---|---|
| Input | ViT-L imaging features — `(N, 5120)` per sample (5 fluorescence channels × 1,024 dims) |
| Output | scGPT embeddings — `(n_cells, 512)` |
| Architecture | Cross-attention diffusion denoiser |
| Diffusion steps | 1,000 (cosine schedule) |
| Inference steps | 50 (DDIM, default) |
| Model parameters | ~168 M |
| Training dataset | [scGeneScope](https://huggingface.co/datasets/altoslabs/scGeneScope) |
| Best val MSE loss | 0.1683 (epoch 87) |

## Architecture

The denoiser uses a **cross-attention transformer** stack:

1. **Imaging encoder** — 2-layer self-attention transformer projects `(B, N, 5120)` → `(B, N, 1024)` context.
2. **RNA + time input** — noisy scGPT embedding and sinusoidal time embedding are projected to `model_dim`.
3. **Cross-attention blocks** (×6) — RNA queries attend to imaging context, with self-attention and adaptive layer normalization conditioned on the timestep.
4. **Output projection** — predicts noise `ε ∈ ℝ^{512}` for the denoising objective.

The diffusion process uses a cosine beta schedule over T=1,000 steps with EMA weight averaging (decay 0.9999).

## Quick start

```bash
pip install torch numpy huggingface_hub
python example.py
```

```python
import numpy as np
from pipeline import PhenoSeqPipeline

pipe = PhenoSeqPipeline.from_pretrained("Sentinal4D/PhenoSeq")

# img_features: ViT-L embeddings — (n_cells, n_imaging_cells=16, 5120)
img_features = np.random.randn(8, 16, 5120).astype(np.float32)
rna_predictions = pipe(img_features)   # → (8, 512)
```

## Inputs and outputs

### Input: imaging features

`img_features` must be ViT-L image embeddings extracted from **5 fluorescence channels** using a ViT-L/14 backbone, resulting in 5 × 1,024 = 5,120 dimensions per imaging cell. Each sample/well typically provides **N = 16** imaging cells (evenly spaced from the available pool) that form the conditioning context.

Shape: `(B, N, 5120)` where `B` is the number of target RNA cells to predict.

> **Imaging normalisation** — `img_norm.npz` contains the per-feature mean and std computed from the training split of scGeneScope. These are applied automatically by `PhenoSeqPipeline`. If you work with a different dataset you will need to recompute and supply your own normalisation stats.

### Output: RNA embeddings

Shape: `(B, 512)` — scGPT-space embeddings un-normalized back to the original scGPT embedding scale. These can be used directly for downstream tasks such as cell-type classification (see `classify_improved.py`), clustering, or trajectory inference.

## Imaging normalisation stats

The pipeline requires `img_norm.npz` (per-feature mean and std from the training split). This file is distributed alongside `best_model.pt` in this repo. If you retrain or use different data, regenerate it:

```bash
python save_img_norm.py --config config.yaml --output img_norm.npz
```

## Full inference on scGeneScope data

For large-scale inference over the cached scGeneScope data (`.npz` per sample):

```bash
# Fast (DDIM, 50 steps)
python infer.py --checkpoint best_model.pt --ddim_steps 50

# Val split only
python infer.py --checkpoint best_model.pt --split val --output_dir results/predictions

# Full DDPM sampling (slower, ~1000 steps)
python infer.py --checkpoint best_model.pt --ddim_steps 0
```

Output: one `{Sample_ID}.npz` per sample under `results/predictions/`, with key `X` of shape `(n_cells, 512)`.

## Downstream: cell-type classification

Predicted RNA embeddings can be evaluated with the included classifier:

```bash
python classify_improved.py
```


## Training

The model was trained from scratch on scGeneScope using:

```bash
python train.py --config config.yaml
```

## Data

Training and evaluation data come from the [scGeneScope](https://huggingface.co/datasets/altoslabs/scGeneScope) dataset (Altos Labs):

- **Imaging features**: ViT-L/14 embeddings extracted from 5-channel fluorescence microscopy images, stored as `.h5ad` files.
- **RNA-seq features**: scGPT cell embeddings (512-dim) from paired single-cell RNA-seq, stored as `.h5ad` files.
- Samples are matched by `Sample_ID` at well level.

Prepare the local cache before training:

```bash
python prepare_data.py
```

## Repository structure

```
PhenoSeq/
├── pipeline.py          ← self-contained inference pipeline (start here)
├── best_model.pt        ← trained checkpoint with EMA weights & RNA norm stats
├── img_norm.npz         ← imaging normalisation stats (mean/std, training split)
├── config.yaml          ← full training configuration
├── infer.py             ← batch inference over cached scGeneScope data
├── save_img_norm.py     ← helper to (re)generate img_norm.npz
├── train.py             ← training entry point
├── prepare_data.py      ← extract imaging/RNA features → .npz cache
├── models/
│   ├── denoiser.py      ← cross-attention denoiser
│   ├── diffusion.py     ← Gaussian diffusion process (forward + reverse)
│   └── lit_module.py    ← PyTorch Lightning training wrapper
└── data/
    └── dataset.py       ← paired imaging-RNA dataset & dataloader
```

## Requirements

```
torch>=2.0
anndata>=0.10
numpy>=1.24
scipy>=1.10
PyYAML>=6.0
tqdm>=4.65
# Optional: huggingface_hub (for from_pretrained with a Hub repo id)
```

Install:
```bash
pip install -r requirements.txt
```

## License

Apache 2.0 — see [LICENSE](LICENSE).

## Citation

If you use PhenoSeq, please cite our paper:

```bibtex
@inproceedings{naidoo2026phenoseq,
  title     = {Cell Painting Generates Single-Cell Transcriptomics via Conditional Diffusion},
  author    = {Naidoo, Reed and Hu, Jingyu and Tripodi, Giuseppe and Bakal, Chris and Chakraborti, Tapabrata},
  booktitle = {ICML 2026 Workshop on Multi-modal Foundation Models and Large Language Models for Life Sciences (FM4LS)},
  year      = {2026},
  url       = {https://openreview.net/forum?id=ACHMa1e8J1}
}
```

Please also cite the training dataset:

```bibtex
@dataset{scgenescope,
  author    = {Altos Labs},
  title     = {scGeneScope},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/datasets/altoslabs/scGeneScope}
}
```