| --- |
| license: apache-2.0 |
| language: |
| - en |
| - zh |
| tags: |
| - OneScience |
| - Earth Science |
| - Weather Forecast |
| - Short-to-Medium-Range Weather Forecast |
| - ERA5 |
| frameworks: PyTorch |
| datasets: |
| - OneScience/ERA5 |
| --- |
| <p align="center"> |
| <strong> |
| <span style="font-size: 30px;">FengWu</span> |
| </strong> |
| </p> |
| |
| # Model Introduction |
|
|
| FengWu is a global medium-range weather forecast foundation model jointly released by the Shanghai Artificial Intelligence Laboratory and multiple universities. It has been adopted by organizations such as the Hong Kong Observatory for operational weather forecasting. |
|
|
| Paper: FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead |
|
|
| https://arxiv.org/abs/2304.02948 |
|
|
| # Model Description |
|
|
| The FengWu model is built on a multi-modal and multi-task deep learning approach, without relying on traditional physical equations. It is trained entirely on ERA5 reanalysis data. |
|
|
| # Use Cases |
|
|
| | Scenario | Description | |
| | :---: | :--- | |
| | Weather Forecast Training | Train FengWu using ERA5 HDF5 data | |
| | Local Quick Validation | Use synthetic data to verify data loading, model training & inference, and inference result visualization. | |
| | ModelScope / OneCode Execution | Download as a standalone model package, install dependencies, and run scripts directly. | |
| | Multi-GPU Training | Launch multi-process training via `torchrun`. | |
|
|
| # Usage Guide |
|
|
| ## 1. OneCode Usage |
|
|
| Experience intelligent one-click AI4S programming through the OneCode online environment: |
|
|
| [Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home) |
|
|
| ## 2. Manual Installation and Usage |
|
|
| **Hardware Requirements** |
|
|
| - A GPU or DCU is recommended. |
| - CPU can be used for import and small-scale connectivity verification; full training and inference will be slow. |
| - DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended. |
|
|
| ### Download the Model Package |
|
|
| ```bash |
| modelscope download --model OneScience/FengWu --local_dir ./FengWu |
| cd FengWu |
| ``` |
|
|
| ### Install the Runtime Environment |
|
|
| **DCU Environment** |
|
|
| ```bash |
| # Please activate DTK and CONDA first |
| conda create -n onescience311 python=3.11 -y |
| conda activate onescience311 |
| # uv installation is supported |
| pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai |
| ``` |
|
|
| **GPU Environment** |
| ```bash |
| # Please activate CONDA first |
| conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12 |
| conda activate onescience311 |
| # uv installation is supported |
| pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai |
| ``` |
|
|
| ### Training Data Introduction |
|
|
| The OneScience community provides ERA5 data for training (due to file size limits, the current repository contains a slice of the full dataset). Users can download it with the command below and confirm that the data path in `conf/config.yaml` is set correctly: |
|
|
| ```bash |
| modelscope download --dataset OneScience/ERA5 --local_dir ./data |
| ``` |
|
|
| ### Training |
|
|
| Single GPU: |
|
|
| ```bash |
| python scripts/train.py |
| ``` |
|
|
| Multi-GPU: |
|
|
| ```bash |
| torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py |
| ``` |
|
|
| Training will save `model_bak.pth` under `data/checkpoints/`. |
|
|
| ### Training Weights |
|
|
| This repository provides weights trained on 39 years of ERA5 reanalysis data in the `weights/` folder. The weight files will be uploaded soon and are expected to be available in the near future. |
|
|
| ### Inference |
|
|
| ```bash |
| python scripts/inference.py |
| ``` |
|
|
| Inference results will be saved to `result/output/`. |
|
|
| ### Evaluation and Visualization |
|
|
| ```bash |
| python scripts/result.py |
| ``` |
|
|
| You can specify a date and variable at the end of `result.py` for visualization. |
|
|
| # OneScience Official Information |
|
|
| | Platform | OneScience Main Repository | Skills Repository | |
| | --- | --- | --- | |
| | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills | |
| | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills | |
|
|
| # Citation & License |
|
|
| - This repository is a reproduction of the original FengWu paper. |
|
|
