README.md

---
frameworks:
- ""
language:
- en
- zh
license: apache-2.0
tags:
- OneScience
- life-science
- genomic-sequence-modeling
- DNA/RNA
tasks: []
---

<p align="center">
  <strong><span style="font-size: 30px;">Evo2</span></strong>
</p>

# Model Introduction

Evo2 is a large-scale foundation model for genomic sequences. It can generate, complete, and predict functions for DNA/RNA sequences. The model supports pretraining on large-scale genomic datasets and can be rapidly adapted to downstream tasks such as regulatory element prediction and variant effect assessment, providing a unified sequence modeling tool for genomics research.

# Model Description

Evo2 is built on a hybrid architecture that combines Hyena operators with convolutional gating, and supports context windows at the million-token scale. The model is pretrained on multi-species genomic data containing trillions of base pairs. It takes raw nucleotide sequences as input and supports autoregressive generation, sequence classification, and embedding extraction. Evo2 is compatible with multi-GPU distributed training and flexible data mixing, achieves leading performance on benchmarks such as GenBench, and is suitable for pretraining, fine-tuning, and downstream functional validation.

# Applicable Scenarios

| Scenario | Description |
| --- | --- |
| DNA/RNA sequence generation | Generate subsequent genomic sequence fragments from an input prompt |
| FASTA sequence prediction | Read FASTA input and output sequence prediction results |
| Local offline inference validation | Use the example inputs in this repository and a local checkpoint to verify the inference workflow end to end |
| Mini training workflow validation | Use `data/data_mini/genome_data/` to validate the training scripts, data loading, and checkpoint workflow |
| Cluster training adaptation | Use `scripts/train_slurm.py` to run training in a Slurm/cluster environment |
| Data preprocessing | Generate the `.bin/.idx` data required by the Byte-Level tokenizer from FASTA/JSON data |


# Usage

## 1. Using OneCode

You can try intelligent one-click AI4S programming through the OneCode online environment:

[Try intelligent one-click AI4S programming](https://gitee.com/link?target=https%3A%2F%2Fweb-2069360198568017922-iaaj.ksai.scnet.cn%3A58043%2Fhome)

## 2. Manual Installation and Usage

**Hardware Requirements**

- Running on a GPU or DCU is recommended.
- A CPU can be used for import checks and small-configuration connectivity validation, but full training and inference will be slow.
- DCU users need to install DTK in advance. DTK 25.04.2 or later is recommended, or the OneScience-recommended version that matches the current cluster.

### Download the Model Package

```bash
modelscope download --model OneScience/evo2 --local_dir ./evo2
cd evo2
```

### Install the Runtime Environment
#### DCU Environment

```bash
# Activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[bio] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

If the scripts need to access the OneScience source code, set:

```bash
export ONESCIENCE_ROOT=/path/to/onescience
```

When run from this directory, the scripts automatically add the project root and `model/` to `sys.path`. You can also manually verify that the local package can be imported:

```bash
export PYTHONPATH=$(pwd)/model:${PYTHONPATH:-}
python -c "import evo2; print('evo2 import ok')"
```

## 3. Quick Start

### Quick Checks

View the entry-point arguments:

```bash
python scripts/infer.py --help
python scripts/predict.py --help
python scripts/train.py --help
python scripts/train_slurm.py --help
```

## 4. Data and Weights

The example training data has been placed at:

```text
data/data_mini/genome_data/
```

It contains:

* `chr20.fa`, `chr21.fa`, `chr22.fa`, and their compressed versions
* Merged FASTA: `chr20_21_22.fa`
* Preprocessed binary data for training, validation, and testing: `preprocessed_data/`

The `dataset_prefix` in `config/genome_data_config.yaml` is a relative path. During actual runs, it should be used together with:

```bash
--dataset-dir data/data_mini/genome_data
```

By default, the 7B NeMo checkpoint is placed inside the repository:

```text
checkpoints/evo2_nemo_7b/
```

The inference and prediction entry points read this directory by default; training does not load a checkpoint by default. To fine-tune from existing weights or resume training, explicitly pass `--ckpt-dir checkpoints/evo2_nemo_7b` in the training command. You can also use `EVO2_CKPT_DIR` or `--ckpt-dir` to override it with another checkpoint path:

```bash
export EVO2_CKPT_DIR=/path/to/evo2_nemo_7b
```

The weight files are located under `/path/to/evo2/checkpoints/evo2_nemo_7b/weights`.

## 5. Prompt-Based Generation Inference

`infer.py` writes results to `--output-file`, but it does not automatically create the parent directory. Create the output directory before the first run:

```bash
mkdir -p outputs
```

Minimal example:

```bash
python scripts/infer.py \
  --prompt "ATGCGT" \
  --output-file outputs/evo2_generation.txt
```

Common parameters:

| Parameter | Default | Description |
| --- | --- | --- |
| `--ckpt-dir` | `checkpoints/evo2_nemo_7b` | NeMo checkpoint directory; can be overridden with `EVO2_CKPT_DIR` |
| `--prompt` | Built-in E. coli classification-label prompt | Input sequence or text prompt |
| `--max-new-tokens` | `1024` | Maximum number of tokens to generate |
| `--temperature` | `1.0` | Sampling temperature |
| `--top-k` | `0` | top-k sampling |
| `--top-p` | `0.0` | top-p sampling |
| `--tensor-parallel-size` | `1` | Tensor parallel size |
| `--output-file` | Empty | Write to a file when specified; otherwise print logs |

## 6. FASTA Prediction

The directory includes an example FASTA:

```text
data/predict_example.fa
```

Run prediction:

```bash
python scripts/predict.py \
  --fasta data/predict_example.fa \
  --output-dir outputs/predict
```

`predict.py` automatically creates `--output-dir`. If `--fasta` is not passed, it reads `data/predict_example.fa` by default; if `--output-dir` is not passed, it writes output to `outputs/predict` by default.

Common parameters:

| Parameter | Default | Description |
| --- | --- | --- |
| `--fasta` | `data/predict_example.fa` | Input FASTA file |
| `--ckpt-dir` | `checkpoints/evo2_nemo_7b` | NeMo checkpoint directory; can be overridden with `EVO2_CKPT_DIR` |
| `--output-dir` | `outputs/predict` | Prediction result directory |
| `--batch-size` | `1` | Prediction batch size |
| `--model-size` | `7b` | Model specification; available values are determined by NeMo `HYENA_MODEL_OPTIONS` |
| `--output-log-prob-seqs` | Disabled | Output sequence log probability |

`data/data_mini/genome_data/chr20.fa`, `chr21.fa`, and `chr22.fa` are relatively large chromosome-level FASTA files. Direct prediction may consume substantial GPU memory. It is recommended to split them into shorter fragments first, or use a multi-GPU parallel configuration.

## 7. Training

The training entry point requires one of two options: pass `-d/--dataset-config`, or use `--mock-data`. For the built-in example data, `config/genome_data_config.yaml` and `data/data_mini/genome_data` are recommended.

Example for training the 1B architecture from scratch or validating the training workflow:

```bash
python scripts/train.py \
  -d config/genome_data_config.yaml \
  --dataset-dir data/data_mini/genome_data \
  --model-size 1b \
  --result-dir results_1b \
  --devices 8 \
  --num-nodes 1 \
  --seq-length 8192 \
  --micro-batch-size 2 \
  --lr 0.0001 \
  --warmup-steps 5 \
  --max-steps 1000 \
  --clip-grad 1 \
  --wd 0.01 \
  --activation-checkpoint-recompute-num-layers 1 \
  --val-check-interval 50 \
  --limit-val-batches 2
```

This example does not load a checkpoint and is suitable for validating the training workflow when 1B weights are unavailable. To fine-tune from a 1B NeMo checkpoint, additionally pass `--ckpt-dir /path/to/evo2_nemo_1b`.

Example for training the 7B long-context architecture from scratch or validating the training workflow:

```bash
python scripts/train.py \
  -d config/genome_data_config.yaml \
  --dataset-dir data/data_mini/genome_data \
  --model-size 7b_arc_longcontext \
  --result-dir results_7b \
  --devices 8 \
  --num-nodes 1 \
  --seq-length 1024 \
  --micro-batch-size 1 \
  --lr 0.0001 \
  --warmup-steps 5 \
  --max-steps 1000 \
  --clip-grad 1 \
  --wd 0.01 \
  --activation-checkpoint-recompute-num-layers 1 \
  --val-check-interval 50 \
  --limit-val-batches 2
```

This example does not load a checkpoint. If you need to fine-tune from an existing 7B NeMo checkpoint or resume training, additionally pass `--ckpt-dir checkpoints/evo2_nemo_7b` or another checkpoint path.

In a Slurm/cluster environment, replace the entry point with:

```bash
python scripts/train_slurm.py ...
```

Common training parameters:

The table below covers the main parameters used in the training examples above and in `scripts/train_evo2_1b.sh` and `scripts/train_evo2_7b.sh`. For more advanced parameters, run `python scripts/train.py --help`.

| Parameter | Default | Description |
| --- | --- | --- |
| `-d, --dataset-config` | Required, or use `--mock-data` | Training data configuration YAML |
| `--mock-data` | Disabled | Do not read real data; use mock data for connectivity testing. Mutually exclusive with `-d/--dataset-config` |
| `--dataset-dir` | `EVO2_DATASET_DIR` or `data/data_mini/genome_data` | Data root directory |
| `--ckpt-dir` | Empty | Initial checkpoint directory; pass explicitly when fine-tuning from existing weights or resuming training |
| `--model-size` | `7b` | Model specification. Common values include `1b`, `7b`, and `7b_arc_longcontext`; use `test` for testing |
| `--devices` | `1` | Number of devices on a single node |
| `--num-nodes` | `1` | Number of nodes |
| `--tensor-parallel-size` | `1` | Tensor parallel size |
| `--pipeline-model-parallel-size` | `1` | Pipeline model parallel size |
| `--context-parallel-size` | `1` | Context parallel size |
| `--sequence-parallel` | Disabled | Enable sequence parallel, usually used together with tensor parallelism |
| `--seq-length` | `8192` | Training sequence length |
| `--micro-batch-size` | `1` | Micro batch size |
| `--global-batch-size` | Automatically inferred | Global batch size |
| `--grad-acc-batches` | `1` | Number of gradient accumulation batches, used to infer the global batch size |
| `--lr` | `3e-4` | Learning rate, explicitly set to `0.0001` in the examples |
| `--min-lr` | `3e-5` | Minimum learning rate of the cosine scheduler |
| `--warmup-steps` | `2500` | Number of warmup steps, explicitly set to `5` in the examples |
| `--constant-steps` | `80000` | Number of steps for which the learning rate stays constant |
| `--max-steps` | `500000` | Number of training steps |
| `--wd` | `0.01` | Optimizer weight decay |
| `--clip-grad` | `1.0` | Gradient clipping threshold |
| `--workers` | `8` | Number of DataLoader workers |
| `--activation-checkpoint-recompute-num-layers` | Empty | Override the number of activation recomputation layers; explicitly set to `1` in the examples |
| `--val-check-interval` | Empty | Validation and checkpoint check interval; explicitly set to `50` in the examples |
| `--limit-val-batches` | `20` | Maximum number of batches to run per validation; explicitly set to `2` in the examples |
| `--result-dir` | `./results` | Log and result directory |
| `--experiment-name` | `evo2` | Experiment name, which affects the log directory |
| `--disable-checkpointing` | Not passed by default; checkpoint callback enabled | Pass this parameter during connectivity tests to disable the checkpoint callback |
| `--no-save-last-checkpoint` | Save the last checkpoint by default | Disable saving the last checkpoint; often used together with `--disable-checkpointing` for quick tests |
| `--save-top-k` | `5` | Number of best checkpoints to save |
| `--ckpt-async-save` | Disabled | Enable asynchronous checkpoint saving |

## 8. Data Preprocessing

The current example training data has already been preprocessed. To process new FASTA or JSON data, use:

```bash
bash scripts/tools/data_process/preprocess_data_fasta.sh
bash scripts/tools/data_process/preprocess_data_json.sh
```

Related configurations:

```text
config/genome_preprocess_config.yaml
config/genome_data_config.yaml
```

# Official OneScience Information

| Platform | OneScience Main Repository | Skills Repository |
| --- | --- | --- |
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |

# Citations and License

- Original Evo2 paper: [Genome modelling and design across all domains of life with Evo 2](https://doi.org/10.1038/s41586-026-10176-5).
- The original Evo2 code uses the Apache License 2.0. See `LICENSE` in this repository. Original license source: [ArcInstitute/evo2 LICENSE](https://github.com/ArcInstitute/evo2/blob/main/LICENSE).
- The Evo2 7B model weights are sourced from Hugging Face [`arcinstitute/evo2_7b`](https://huggingface.co/arcinstitute/evo2_7b). In this repository, `checkpoints/evo2_nemo_7b` is the converted NeMo checkpoint format. The original weight license is Apache-2.0.
- The example dataset `data/data_mini/genome_data/` was downloaded and preprocessed from the hg38 `chr20`, `chr21`, and `chr22` chromosome FASTA files provided by the UCSC Genome Browser. It is used only to validate the training and inference workflows. See the data download directory: [UCSC hg38 chromosomes](https://hgdownload.soe.ucsc.edu/goldenpath/hg38/chromosomes/).
- Use of UCSC hg38 data must comply with the UCSC Genome Browser data usage terms. See: [UCSC Terms of Use](https://genome.ucsc.edu/license/).
- If you use this repository, the model weights, or generated results in research, it is recommended to cite the original Evo2 paper, the model weight source, the UCSC hg38 data source, and relevant OneScience project information, and to add citations for downstream analysis tools or datasets according to the actual task.