README.md

## CVRP: A Rice Image Dataset with High-Quality Annotations for Image Segmentation and Plant Phenomics Research

##### Multi-cultivar and multi-view rice plant image dataset (CVRP) consists of 2,303 field images with their annotated masks and 123 indoor images of individual panicles.



### Annotation Workflow
##### To optimize the process of annotation, we combine deep learning methods with manual curation. The workflow comprises two stages: manual annotation and model-based prediction followed by manual curation. 
![alt text](img/workflow.png)
### Getting Started
##### Recommend python 3.7, CUDA v11.3 and Pytorch 1.10.0.
***Clone***

```bash
pip install huggingface_hub
huggingface-cli download CVRPDataset/Model --local-dir /your/path/to/save/Model
cd Model
git clone https://github.com/open-mmlab/mmsegmentation.git -b v1.1.2 mmsegmentation

pip install -U openmim
mim install mmengine
mim install mmcv==2.0.0

cd mmsegmentation
pip install -v -e .
pip install "mmdet>=3.0.0rc4"
```
***UI***
##### We create a web user interface for annotation based on gradio.
```bash
pip install gradio
python app.py
```
##### The UI :
1. Users can upload an image or use a sample image at ①.Then, they can select one of four models at ②. We recommend **Mask2Former**. After that, click *Run*.
2. We provide two forms of segmentation results for download at ③.
![alt text](img/app.png)

***Train and Test***
1. ***Creating a Dataset***

Here is an example if you want to make own dataset.

① Directory structure of the dataset:
<pre>
   📁 CVRPDataset/
   ├─📁 images/
   └─📁 labelme_jsons/
</pre>
② Convert labelme files to mask:
   ```bash
   python run/labelme2mask.py
   ```
   now, the structure looks like:
<pre>
   📁 CVRPDataset/
   ├─📁 img_dir/
   └─📁 ann_dir/
</pre>
③ Split the training set and test set.
   ```bash
   python run/split_dataset.py
   ```
  now, the structure looks like:
<pre>
   📁 CVRPDataset/
   ├─📁 img_dir/
   │  ├─📁 train/
   │  └─📁 val/
   └─📁 ann_dir/
      ├─📁 train/
      └─📁 val/
</pre>

   If you have annotations in RGB format, you need 
   ```bash
   python run/transform.py
   ```
   and then follow split the train set and evaluation set according to the format in ③.
   You can also download our training set and validation set [here](http://61.155.111.202:18080/cvrp).

2. ***Dataset Configs***

```bash
cd mmsegmentation/mmseg/datasets
rm -rf __init__.py                  # delete original file
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP.py
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/__init__.py  
cd ../../configs/_base_/datasets
wget https://huggingface.co/CVRPDataset/Model/resolve/main/dataset_configs/CVRP_pipeline.py
```
① If you want to register your own dataset, import and register datasets in `mmseg/datasets/__init__.py`
   ```python
   from .CVRP import CVRPDataset
   ```
   Add,
   ```python
   # other datasets
   __all__ = ['CVRPDataset']
   ```
② Register dataset class in `mmseg/datasets/CVRP.py'
   ```python
   class CVRPDataset(BaseSegDataset):
   METAINFO = {
        'classes':['background','panicle'],
        'palette':[[127,127,127],[200,0,0]]
    }
   ```
   
③ Modify pipeline of data process in `config/_base_/datasets/CVRP_pipeline.py`
   ```python
    dataset_type = 'CVRPDataset' 
    data_root = 'CVRPDataset/'
   ```
   you'll need to specify the paths for the train and evalution data directories.
  ```python
    # train_dataloader: 
    data_prefix=dict(img_path='img_dir/train', seg_map_path='ann_dir/train')
    # val_dataloader:
    data_prefix=dict(img_path='img_dir/val', seg_map_path='ann_dir/val')
  ```

3. ***Model Configs***

You can generate model config files using *run_configs.py*

```bash
cd mmsegmentation
mkdir 'work_dirs' 'CVRP_configs' 'outputs' 'CVRPDataset'
python ../run/run_configs.py --model_name deeplabv3plus -m configs/deeplabv3plus/deeplabv3plus_r101-d8_4xb4-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name knet -m configs/knet/knet-s3_swin-l_upernet_8xb2-adamw-80k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name mask2former -m configs/mask2former/mask2former_swin-l-in22k-384x384-pre_8xb2-160k_ade20k-640x640.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs

python ../run/run_configs.py --model_name segformer -m configs/segformer/segformer_mit-b5_8xb2-160k_ade20k-512x512.py -d configs/_base_/datasets/CVRP_pipeline.py -c 2 -w work_dirs -s CVRP_configs
```
 Also, you can download model config files [here](https://huggingface.co/CVRPDataset/Model/tree/main/model_configs). 
 ```bash
cd CVRP_configs
wget https://huggingface.co/CVRPDataset/Model/resolve/main/model_configs/CVRP_mask2former.py
 ```

4. ***Train***
```bash
cd mmsegmentation 
python tools/train.py CVRP_configs/CVRP_mask2former.py
```
Also, you can download checkpoint [here](https://huggingface.co/CVRPDataset/Model/tree/main/checkpoint). 
```bash
cd work_dirs
mkdir CVRP_mask2former
cd CVRP_mask2former
wget https://huggingface.co/CVRPDataset/Model/resolve/main/checkpoint/Mask2Former.pth
```

5. ***Test***
```bash
python ../run/test.py -d CVRPDataset/val -m CVRP_configs/CVRP_mask2former.py -pth work_dirs/CVRP_mask2former/Mask2Former.pth -o outputs/CVRP_mask2former
```

### LabelMe
##### If you need to manually adjust the annotation, you can use LabelMe.
```bash
python run/mask2json.py
pip install labelme==3.16.7
labelme
```

```bash
python run/json2png.py
```
### Data and Code Availability
The CVRP dataset and accompanying codes are publicly available from Hugging Face at https://huggingface.co/datasets/CVRPDataset/CVRP & https://huggingface.co/CVRPDataset/Model, 
and Bioinformatics service in Nanjing Agricultural University at http://bic.njau.edu.cn/CVRP.html.

### Acknowledgements
We thank Mr.Zhitao Zhu, Dr. Weijie Tang, and Dr. Yunhui Zhang for their technical support.