zhangrenchao commited on
Commit
b437af8
·
verified ·
1 Parent(s): a13f4b9

Add English model card

Browse files
Files changed (1) hide show
  1. README.md +183 -0
README.md ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - OneScience
7
+ - Earth Science
8
+ - Heliophysics
9
+ - Sun-Earth Space Forecasting
10
+ - Spatiotemporal Foundation Model
11
+ - Spectral Gating
12
+ - Long-Short Attention
13
+ frameworks: PyTorch
14
+ datasets:
15
+ - SDO/AIA
16
+ - SDO/HMI
17
+ ---
18
+
19
+ <p align="center">
20
+ <strong><span style="font-size: 30px;">Surya</span></strong>
21
+ </p>
22
+
23
+ # Model Introduction
24
+
25
+ Surya is a spatiotemporal foundation model for heliophysics and space weather research. It learns solar dynamics from multi-instrument SDO observations. The model combines spectral gating with long-short attention to predict future solar images from two historical time steps and supports autoregressive multi-step forecasting.
26
+
27
+ Paper: Surya: Foundation Model for Heliophysics
28
+ https://ntrs.nasa.gov/citations/20250008498
29
+
30
+ # Model Description
31
+
32
+ Surya was proposed by institutions including NASA, IBM Research, and the University of Alabama in Huntsville. The model uses eight AIA channels and five HMI products from SDO as training data. It is suitable for one-step prediction and autoregressive multi-step forecasting of solar images.
33
+
34
+ # Use Cases
35
+
36
+ | Scenario | Description |
37
+ | :---: | :--- |
38
+ | Solar dynamics forecasting | Predict future multichannel solar images from two historical time steps. |
39
+ | Space weather research | Analyze solar activity and errors at different forecast lead times. |
40
+ | SDO multi-instrument modeling | Jointly process multichannel observations from AIA and HMI. |
41
+ | Solar flare forecasting | Fine-tune model representations to predict M-class and X-class solar flares within future time periods. |
42
+ | Solar active region segmentation | Fine-tune on magnetograms to segment solar active regions and polarity inversion lines. |
43
+ | Local quick validation | Use synthetic data to validate training, inference, evaluation, and visualization. |
44
+ | Multi-GPU training | Launch distributed data-parallel training with `torchrun`. |
45
+
46
+ # Usage Guide
47
+
48
+ ## 1. OneCode Usage
49
+
50
+ Experience intelligent one-click AI4S programming through the OneCode online environment:
51
+
52
+ [Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
53
+
54
+ ## 2. Manual Installation and Usage
55
+
56
+ **Hardware Requirements**
57
+
58
+ - A GPU or DCU is recommended.
59
+ - CPU can be used for pipeline validation with the current default small configuration.
60
+ - Training the original paper configuration with `4096x4096` resolution and 366 million parameters requires large-scale multi-GPU computing resources.
61
+
62
+ ### Download the Model Package
63
+
64
+ ```bash
65
+ hf download OneScience-Group/Surya --local-dir ./Surya
66
+ cd Surya
67
+ ```
68
+
69
+ ### Install the Runtime Environment
70
+
71
+ **DCU Environment**
72
+
73
+ ```bash
74
+ conda create -n onescience311 python=3.11 -y
75
+ conda activate onescience311
76
+ pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
77
+ ```
78
+
79
+ **GPU Environment**
80
+
81
+ ```bash
82
+ conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
83
+ conda activate onescience311
84
+ pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
85
+ ```
86
+
87
+ ### Training Data Introduction
88
+
89
+ The paper uses SDO data from 2010 to 2024, including AIA 94, 131, 171, 193, 211, 304, 335, and 1600 Å, as well as HMI magnetic-field and Doppler-velocity products. The data are standardized to a 12-minute interval and spatially registered. The original training data are approximately 257 TB.
90
+
91
+ This model package uses synthetic data by default:
92
+
93
+ ```bash
94
+ python scripts/fake_data.py
95
+ ```
96
+
97
+ Each NPZ file contains at least `inputs`, `targets`, and `activity`. `inputs` has shape `[N, 2, 13, H, W]`, `targets` has shape `[N, S, 13, H, W]`, and `activity` stores the solar activity intensity at each forecast time step. When using real data, prepare files with the same fields and modify `conf/config.yaml`.
98
+
99
+ When using real data, do not run `fake_data.py`. Save temporally aligned, spatially registered, channel-organized, and signum-log normalized data to `data/`, replacing the synthetic data files:
100
+
101
+ ```text
102
+ data/train.npz
103
+ data/test.npz
104
+ ```
105
+
106
+ Each NPZ file contains at least:
107
+
108
+ ```text
109
+ inputs: float32 [N,2,13,H,W]
110
+ targets: float32 [N,S,13,H,W]
111
+ activity: float32 [N,S]
112
+ ```
113
+
114
+ The 13 channels correspond in order to eight AIA channels and five HMI products, and `S` is consistent with `forecast_steps` in `conf/config.yaml`. Modify `conf/config.yaml` according to the image size, number of forecast steps, channel statistics, and data paths of the real data. After data preparation, continue to use the unified training, inference, and evaluation commands below; use script arguments to override the default paths if other file locations are required.
115
+
116
+ ### Training
117
+
118
+ Single GPU:
119
+
120
+ ```bash
121
+ python scripts/train.py
122
+ ```
123
+
124
+ Multiple GPUs:
125
+
126
+ ```bash
127
+ torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py
128
+ ```
129
+
130
+ Training outputs:
131
+
132
+ ```text
133
+ result/checkpoints/surya.pt
134
+ result/training/metrics.json
135
+ ```
136
+
137
+ The training outputs include a model checkpoint for subsequent multi-step forecasting and training metrics that reflect different training stages, overall loss, and learning-rate changes, facilitating training-state preservation and model optimization analysis. Results on synthetic data are only for code-flow validation and do not represent paper results.
138
+
139
+ ### Training Weights
140
+
141
+ This repository provides weights trained on SDO/AIA and SDO/HMI solar observations in the `weight/` folder. The weight files will be uploaded soon and are expected to be available in the near future.
142
+
143
+ ### Inference
144
+
145
+ ```bash
146
+ python scripts/inference.py
147
+ ```
148
+
149
+ Inference results are output to:
150
+
151
+ ```text
152
+ result/output/forecast.npz
153
+ ```
154
+
155
+ ### Evaluation and Visualization
156
+
157
+ ```bash
158
+ python scripts/result.py
159
+ ```
160
+
161
+ Evaluation and visualization outputs are written to:
162
+
163
+ ```text
164
+ result/evaluation/metrics.json
165
+ result/evaluation/rollout_forecast_skill.png
166
+ result/evaluation/solar_dynamics_forecast.png
167
+ result/evaluation/solar_activity_evolution.png
168
+ result/evaluation/sdo_channel_error.png
169
+ result/evaluation/persistence_skill.png
170
+ ```
171
+
172
+ The evaluation results comprehensively measure multi-step forecast errors, forecast skill relative to a persistence baseline, performance across AIA/HMI channels, and changes in solar activity intensity. Forecast images and lead-time curves show the evolution of solar dynamics and the long-term forecast stability of the model. The results follow the autoregressive forecasting protocol from the paper; results on synthetic data are only for validating the Surya workflow and do not represent performance on the paper's downstream tasks.
173
+
174
+ # Official OneScience Resources
175
+
176
+ | Platform | OneScience Main Repository | Skills Repository |
177
+ | --- | --- | --- |
178
+ | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
179
+ | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
180
+
181
+ # Citation and License
182
+
183
+ This repository is a reproduction of the original Surya paper.