OneScience commited on
Commit
21e6c23
·
verified ·
1 Parent(s): c7eefde

Upload folder using huggingface_hub

Browse files
README.md ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ tags:
6
+ - OneScience
7
+ - fluid dynamics
8
+ - airfoil aerodynamic prediction
9
+ frameworks: PyTorch
10
+ ---
11
+ <p align="center">
12
+ <strong>
13
+ <span style="font-size: 30px;">Transolver-Airfoil-Design</span>
14
+ </strong>
15
+ </p>
16
+
17
+ # Model Overview
18
+
19
+ Transolver-Airfoil-Design is a two-dimensional external-flow prediction model for airfoils, built on Transolver by Tsinghua University's THUML group. It rapidly predicts flow-field distributions and aerodynamic performance around airfoils.
20
+
21
+ Paper: [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://arxiv.org/abs/2402.02366)
22
+
23
+ # Model Description
24
+ Transolver-Airfoil-Design uses a Transformer architecture with Physics-Attention and is adapted for training on the unstructured-mesh AirfRANS airfoil dataset. It predicts velocity fields, pressure fields, and drag coefficients for airfoil geometries.
25
+
26
+ ## Use Cases
27
+
28
+ | Use Case | Description |
29
+ | :--- | :--- |
30
+ | Airfoil aerodynamic design | Rapidly predict two-dimensional external flow around airfoils to screen candidate geometries |
31
+ | Industrial simulation acceleration | Accelerate large-scale simulations using PDE surrogate modeling on complex geometries |
32
+
33
+ # Usage
34
+
35
+ ## 1. OneCode
36
+
37
+ Use the online OneCode environment for an intelligent, one-click AI for Science (AI4S) programming experience:
38
+
39
+ [Launch OneCode for one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
40
+
41
+ ## 2. Manual Setup
42
+
43
+ **Hardware Requirements**
44
+
45
+ - A GPU or DCU is recommended.
46
+ - A CPU can be used for import checks and small-scale pipeline validation, but full training and inference will be slow.
47
+ - DCU users must install DTK in advance. DTK 25.04.2 or later, or the OneScience-recommended version for the target cluster, is recommended.
48
+
49
+ ### Download the Model Package
50
+
51
+ ```bash
52
+ modelscope download --model OneScience/Transolver-Airfoil-Design --local_dir ./Transolver-Airfoil-Design
53
+ cd Transolver-Airfoil-Design
54
+ ```
55
+
56
+ ### Set Up the Runtime Environment
57
+
58
+
59
+ **DCU Environment**
60
+
61
+ ```bash
62
+ # Activate DTK and Conda first
63
+ conda create -n onescience311 python=3.11 -y
64
+ conda activate onescience311
65
+ # Installation with uv is also supported
66
+ pip install onescience[cfd-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
67
+ ```
68
+
69
+ **GPU Environment**
70
+ ```bash
71
+ # Activate Conda first
72
+ conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
73
+ conda activate onescience311
74
+ # Installation with uv is also supported
75
+ pip install onescience[cfd-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
76
+ ```
77
+
78
+ ### Training Data
79
+ The OneScience community provides the `airfrans` dataset for training. Download it with the command below and verify that the data path in `config/config.yaml` is configured correctly:
80
+
81
+ ```bash
82
+ modelscope download --dataset OneScience/airfrans --local_dir ./data
83
+ ```
84
+
85
+ ### Training
86
+
87
+ ```bash
88
+ python scripts/train.py
89
+ ```
90
+
91
+ By default, training saves the following checkpoint:
92
+
93
+ ```text
94
+ ./weight/Transolver.pth
95
+ ```
96
+
97
+ ### Model Weights
98
+ This repository will provide model weights pretrained on ShapeNetCar data in the `weights/` directory. The weights will be uploaded soon.
99
+
100
+ ### Inference
101
+
102
+ ```bash
103
+ python scripts/inference.py
104
+ ```
105
+
106
+ ### Evaluation and Visualization
107
+
108
+ ```bash
109
+ python scripts/result.py
110
+ ```
111
+
112
+ # Official OneScience Resources
113
+
114
+ | Platform | OneScience Repository | Skills Repository |
115
+ | --- | --- | --- |
116
+ | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
117
+ | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
118
+
119
+ # Citations and License
120
+
121
+ - Original Transolver paper: [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://arxiv.org/pdf/2402.02366).
122
+ - This repository retains source attribution and has been adapted for automated execution through OneScience and ModelScope.
conf/config.yaml ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ model:
2
+ name: "Transolver"
3
+ specific_params:
4
+ Transolver: &transolver_params
5
+ n_hidden: 32
6
+ n_layers: 2
7
+ space_dim: 7
8
+ fun_dim: 0
9
+ n_head: 4
10
+ mlp_ratio: 2
11
+ out_dim: 4
12
+ slice_num: 8
13
+ unified_pos: 0
14
+ build_graph: false
15
+ r: 0.05
16
+ max_neighbors: 32
17
+ Transolver_plus: *transolver_params
18
+
19
+ datapipe:
20
+ name: "AirfRANS"
21
+ backend: "fake_airfrans"
22
+ verbose: false
23
+ source:
24
+ data_dir: "./data/fake_airfrans"
25
+ stats_dir: "./data/stats"
26
+ data:
27
+ splits:
28
+ task: "fake"
29
+ train_name: "train"
30
+ val_name: "val"
31
+ test_name: "test"
32
+ val_split_ratio: 0.25
33
+ sampling:
34
+ sample_strategy: null
35
+ n_boot: 128
36
+ surf_ratio: 0.25
37
+ crop: null
38
+ subsampling: 64
39
+ dataloader:
40
+ batch_size: 1
41
+ num_workers: 0
42
+
43
+ training:
44
+ device: "cpu"
45
+ max_epoch: 1
46
+ lr: 0.001
47
+ gpuid: 0
48
+ patience: 20
49
+ progress_log_interval: 5
50
+ loss_criterion: "MSE_weighted"
51
+ loss_weight: 1.0
52
+ checkpoint_dir: "./weight"
53
+ result_dir: "./result"
54
+ n_test: 2
55
+ seed: 42
56
+
57
+ fake_data:
58
+ num_train: 4
59
+ num_val: 2
60
+ num_test: 2
61
+ num_nodes: 64
62
+ input_dim: 7
63
+ output_dim: 4
64
+ surface_ratio: 0.25
configuration.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"framework":"Pytorch","task":"other","model":"Transolver-Airfoil-Design","pipeline_tag":"cfd-airfoil-surrogate"}
model/Transolver2D.py ADDED
@@ -0,0 +1,136 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn as nn
3
+ import numpy as np
4
+ try:
5
+ from timm.layers import trunc_normal_
6
+ except ImportError:
7
+ from torch.nn.init import trunc_normal_
8
+ from onescience.modules.mlp.MLP import StandardMLP
9
+ from onescience.modules.transformer.Transolver_block import Transolver_block
10
+
11
+ class Transolver2D(nn.Module):
12
+ """
13
+ Transolver2D 模型。
14
+
15
+ 该模型专为处理二维物理场问题(如 2D CFD, 弹性力学等)而设计。
16
+ 架构与 Transolver3D 类似,但针对 2D 空间特性进行了配置。
17
+ 它包含预处理 MLP 映射和多层 Transolver Block 堆叠,用于提取非结构化或结构化网格上的物理特征。
18
+
19
+ Args:
20
+ space_dim (int): 空间维度。默认值: 1。
21
+ n_layers (int): Transolver Block 的层数。默认值: 5。
22
+ n_hidden (int): 隐藏层特征维度。默认值: 256。
23
+ dropout (float): Dropout 概率。默认值: 0。
24
+ n_head (int): 注意力头数。默认值: 8。
25
+ act (str): 激活函数类型。默认值: 'gelu'。
26
+ mlp_ratio (float): MLP 膨胀比率。默认值: 1。
27
+ fun_dim (int): 输入物理场的特征维度。默认值: 1。
28
+ out_dim (int): 输出特征维度。默认值: 1。
29
+ slice_num (int): 物理注意力中的切片数量。默认值: 32。
30
+ ref (int): 统一位置编码的参考网格分辨率。默认值: 8。
31
+ unified_pos (bool): 是否使用统一位置编码。默认值: False。
32
+
33
+ 形状:
34
+ 输入 data: 包含 x (物理状态) 和 pos (坐标) 的数据对象。
35
+ 输出: (N, out_dim),预测的物理场。
36
+ """
37
+ def __init__(self,
38
+ space_dim=1,
39
+ n_layers=5,
40
+ n_hidden=256,
41
+ dropout=0,
42
+ n_head=8,
43
+ act='gelu',
44
+ mlp_ratio=1,
45
+ fun_dim=1,
46
+ out_dim=1,
47
+ slice_num=32,
48
+ ref=8,
49
+ unified_pos=False
50
+ ):
51
+ super(Transolver2D, self).__init__()
52
+ self.__name__ = 'Transolver'
53
+ self.ref = ref
54
+ self.unified_pos = unified_pos
55
+
56
+ if self.unified_pos:
57
+ input_dim = fun_dim + space_dim + self.ref * self.ref
58
+ else:
59
+ input_dim = fun_dim + space_dim
60
+
61
+ self.preprocess = StandardMLP(
62
+ input_dim=input_dim,
63
+ hidden_dims=[n_hidden * 2],
64
+ output_dim=n_hidden,
65
+ activation=act,
66
+ use_bias=True,
67
+ use_skip_connection=False
68
+ )
69
+ self.n_hidden = n_hidden
70
+ self.space_dim = space_dim
71
+
72
+ self.blocks = nn.ModuleList([
73
+ Transolver_block(
74
+ num_heads=n_head,
75
+ hidden_dim=n_hidden,
76
+ dropout=dropout,
77
+ act=act,
78
+ mlp_ratio=mlp_ratio,
79
+ out_dim=out_dim,
80
+ slice_num=slice_num,
81
+ last_layer=(_ == n_layers - 1),
82
+ geotype='unstructured'
83
+ )
84
+ for _ in range(n_layers)
85
+ ])
86
+
87
+ self.initialize_weights()
88
+ self.placeholder = nn.Parameter((1 / (n_hidden)) * torch.rand(n_hidden, dtype=torch.float))
89
+
90
+ def initialize_weights(self):
91
+ self.apply(self._init_weights)
92
+
93
+ def _init_weights(self, m):
94
+ if isinstance(m, nn.Linear):
95
+ trunc_normal_(m.weight, std=0.02)
96
+ if isinstance(m, nn.Linear) and m.bias is not None:
97
+ nn.init.constant_(m.bias, 0)
98
+ elif isinstance(m, (nn.LayerNorm, nn.BatchNorm1d)):
99
+ nn.init.constant_(m.bias, 0)
100
+ nn.init.constant_(m.weight, 1.0)
101
+
102
+ def get_grid(self, my_pos):
103
+ batchsize = my_pos.shape[0]
104
+
105
+ gridx = torch.tensor(np.linspace(-2, 4, self.ref), dtype=torch.float)
106
+ gridx = gridx.reshape(1, self.ref, 1, 1).repeat([batchsize, 1, self.ref, 1])
107
+ gridy = torch.tensor(np.linspace(-1.5, 1.5, self.ref), dtype=torch.float)
108
+ gridy = gridy.reshape(1, 1, self.ref, 1).repeat([batchsize, self.ref, 1, 1])
109
+
110
+ grid_ref = torch.cat((gridx, gridy), dim=-1).to(my_pos.device).reshape(batchsize, self.ref ** 2, 2) # B Ref^2 2
111
+
112
+ pos = torch.sqrt(
113
+ torch.sum((my_pos[:, :, None, :] - grid_ref[:, None, :, :]) ** 2,
114
+ dim=-1)). \
115
+ reshape(batchsize, my_pos.shape[1], self.ref * self.ref).contiguous()
116
+ return pos
117
+
118
+ def forward(self, data):
119
+ x, fx, T = data.x, None, None
120
+ x = x[None, :, :] # [1, N, C]
121
+
122
+ if self.unified_pos:
123
+ new_pos = self.get_grid(data.pos[None, :, :])
124
+ x = torch.cat((x, new_pos), dim=-1)
125
+
126
+ if fx is not None:
127
+ fx = torch.cat((x, fx), -1)
128
+ fx = self.preprocess(fx)
129
+ else:
130
+ fx = self.preprocess(x)
131
+ fx = fx + self.placeholder[None, None, :]
132
+
133
+ for block in self.blocks:
134
+ fx = block(fx)
135
+
136
+ return fx[0]
model/Transolver2D_plus.py ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn as nn
3
+ import numpy as np
4
+ try:
5
+ from timm.layers import trunc_normal_
6
+ except ImportError:
7
+ from torch.nn.init import trunc_normal_
8
+
9
+ from onescience.modules.mlp.MLP import StandardMLP
10
+ from onescience.modules.transformer.Transolver_block import Transolver_block
11
+
12
+ class Transolver2D_plus(nn.Module):
13
+ """
14
+ Transolver2D_plus 模型。
15
+
16
+ 这是 Transolver2D 的增强版本。
17
+ 主要的区别在于它使用了 'unstructured_plus' 类型的注意力机制,这种机制通常包含更复杂的切片或特征交互逻辑(如 Gumbel Softmax 采样等),以增强模型对不规则网格的捕捉能力。
18
+
19
+ Args:
20
+ space_dim (int): 空间维度。默认值: 1。
21
+ n_layers (int): Block 层数。默认值: 5。
22
+ n_hidden (int): 隐藏层维度。默认值: 256。
23
+ dropout (float): Dropout 概率。默认值: 0。
24
+ n_head (int): 注意力头数。默认值: 8。
25
+ act (str): 激活函数。默认值: 'gelu'。
26
+ mlp_ratio (float): MLP 膨胀比率。默认值: 1。
27
+ fun_dim (int): 输入物理场特征维度。默认值: 1。
28
+ out_dim (int): 输出特征维度。默认值: 1。
29
+ slice_num (int): 切片数量。默认值: 32。
30
+ ref (int): 统一位置编码参考分辨率。默认值: 8。
31
+ unified_pos (bool): 是否使用统一位置编码。默认值: False。
32
+ """
33
+ def __init__(self,
34
+ space_dim=1,
35
+ n_layers=5,
36
+ n_hidden=256,
37
+ dropout=0,
38
+ n_head=8,
39
+ act='gelu',
40
+ mlp_ratio=1,
41
+ fun_dim=1,
42
+ out_dim=1,
43
+ slice_num=32,
44
+ ref=8,
45
+ unified_pos=False
46
+ ):
47
+ super(Transolver2D_plus, self).__init__()
48
+ self.__name__ = 'Transolver'
49
+ self.ref = ref
50
+ self.unified_pos = unified_pos
51
+
52
+ if self.unified_pos:
53
+ input_dim = fun_dim + space_dim + self.ref * self.ref
54
+ else:
55
+ input_dim = fun_dim + space_dim
56
+
57
+ self.preprocess = StandardMLP(
58
+ input_dim=input_dim,
59
+ hidden_dims=[n_hidden * 2],
60
+ output_dim=n_hidden,
61
+ activation=act,
62
+ use_bias=True,
63
+ use_skip_connection=False
64
+ )
65
+
66
+ self.n_hidden = n_hidden
67
+ self.space_dim = space_dim
68
+
69
+ self.blocks = nn.ModuleList([
70
+ Transolver_block(
71
+ num_heads=n_head,
72
+ hidden_dim=n_hidden,
73
+ dropout=dropout,
74
+ act=act,
75
+ mlp_ratio=mlp_ratio,
76
+ out_dim=out_dim,
77
+ slice_num=slice_num,
78
+ last_layer=(_ == n_layers - 1),
79
+ geotype='unstructured_plus'
80
+ )
81
+ for _ in range(n_layers)
82
+ ])
83
+
84
+ self.initialize_weights()
85
+ self.placeholder = nn.Parameter((1 / (n_hidden)) * torch.rand(n_hidden, dtype=torch.float))
86
+
87
+ def initialize_weights(self):
88
+ self.apply(self._init_weights)
89
+
90
+ def _init_weights(self, m):
91
+ if isinstance(m, nn.Linear):
92
+ trunc_normal_(m.weight, std=0.02)
93
+ if isinstance(m, nn.Linear) and m.bias is not None:
94
+ nn.init.constant_(m.bias, 0)
95
+ elif isinstance(m, (nn.LayerNorm, nn.BatchNorm1d)):
96
+ nn.init.constant_(m.bias, 0)
97
+ nn.init.constant_(m.weight, 1.0)
98
+
99
+ def get_grid(self, my_pos):
100
+ batchsize = my_pos.shape[0]
101
+
102
+ gridx = torch.tensor(np.linspace(-2, 4, self.ref), dtype=torch.float)
103
+ gridx = gridx.reshape(1, self.ref, 1, 1).repeat([batchsize, 1, self.ref, 1])
104
+ gridy = torch.tensor(np.linspace(-1.5, 1.5, self.ref), dtype=torch.float)
105
+ gridy = gridy.reshape(1, 1, self.ref, 1).repeat([batchsize, self.ref, 1, 1])
106
+
107
+ grid_ref = torch.cat((gridx, gridy), dim=-1).to(my_pos.device).reshape(batchsize, self.ref ** 2, 2)
108
+
109
+ pos = torch.sqrt(
110
+ torch.sum((my_pos[:, :, None, :] - grid_ref[:, None, :, :]) ** 2,
111
+ dim=-1)). \
112
+ reshape(batchsize, my_pos.shape[1], self.ref * self.ref).contiguous()
113
+ return pos
114
+
115
+ def forward(self, data):
116
+ x, fx, T = data.x, None, None
117
+ x = x[None, :, :]
118
+
119
+ if self.unified_pos:
120
+ new_pos = self.get_grid(data.pos[None, :, :])
121
+ x = torch.cat((x, new_pos), dim=-1)
122
+
123
+ if fx is not None:
124
+ fx = torch.cat((x, fx), -1)
125
+ fx = self.preprocess(fx)
126
+ else:
127
+ fx = self.preprocess(x)
128
+ fx = fx + self.placeholder[None, None, :]
129
+
130
+ for block in self.blocks:
131
+ fx = block(fx)
132
+
133
+ return fx[0]
model/__init__.py ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ from .Transolver2D import Transolver2D
2
+ from .Transolver2D_plus import Transolver2D_plus
3
+
4
+ __all__ = ["Transolver2D", "Transolver2D_plus"]
scripts/__pycache__/train.cpython-311.pyc ADDED
Binary file (18.5 kB). View file
 
scripts/fake_data.py ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import sys
5
+ from pathlib import Path
6
+
7
+ import numpy as np
8
+ import torch
9
+ from torch_geometric.data import Data
10
+
11
+ ROOT = Path(__file__).resolve().parents[1]
12
+ sys.path.insert(0, str(ROOT))
13
+
14
+ from onescience.utils.YParams import YParams
15
+
16
+
17
+ def resolve_path(path: str) -> Path:
18
+ value = Path(path).expanduser()
19
+ return value if value.is_absolute() else ROOT / value
20
+
21
+
22
+ def make_sample(num_nodes: int, input_dim: int, output_dim: int, surface_ratio: float, seed: int) -> Data:
23
+ generator = torch.Generator().manual_seed(seed)
24
+ pos = torch.rand((num_nodes, 2), generator=generator)
25
+ freestream = torch.ones((num_nodes, 2))
26
+ freestream[:, 1] = torch.linspace(-0.2, 0.2, num_nodes)
27
+ sdf = torch.linalg.norm(pos - 0.5, dim=1, keepdim=True) - 0.25
28
+ normals = torch.nn.functional.normalize(pos - 0.5, dim=1)
29
+ x = torch.cat([pos, freestream, sdf, normals], dim=1)
30
+ if x.size(1) != input_dim:
31
+ raise ValueError(f"fake input_dim must be 7 for this AirfRANS feature layout, got {input_dim}")
32
+
33
+ y0 = torch.sin(np.pi * pos[:, :1]) + 0.1 * freestream[:, :1]
34
+ y1 = torch.cos(np.pi * pos[:, 1:2]) + 0.1 * freestream[:, 1:2]
35
+ pressure = pos[:, :1] - pos[:, 1:2]
36
+ viscosity = torch.abs(sdf) + 0.01
37
+ y = torch.cat([y0, y1, pressure, viscosity], dim=1)
38
+ if y.size(1) != output_dim:
39
+ raise ValueError(f"fake output_dim must be 4 for this AirfRANS target layout, got {output_dim}")
40
+
41
+ surf = torch.zeros(num_nodes, dtype=torch.bool)
42
+ surf_count = max(1, int(num_nodes * surface_ratio))
43
+ surf[:surf_count] = True
44
+ perm = torch.randperm(num_nodes, generator=generator)
45
+ return Data(pos=pos[perm], x=x[perm].float(), y=y[perm].float(), surf=surf[perm])
46
+
47
+
48
+ def main() -> int:
49
+ cfg = YParams(str(ROOT / "conf" / "config.yaml"), "fake_data")
50
+ cfg_data = YParams(str(ROOT / "conf" / "config.yaml"), "datapipe")
51
+ cfg_train = YParams(str(ROOT / "conf" / "config.yaml"), "training")
52
+ data_dir = resolve_path(cfg_data.source.data_dir)
53
+ stats_dir = resolve_path(cfg_data.source.stats_dir)
54
+ data_dir.mkdir(parents=True, exist_ok=True)
55
+ stats_dir.mkdir(parents=True, exist_ok=True)
56
+
57
+ split_counts = {
58
+ cfg_data.data.splits.train_name: int(cfg.num_train),
59
+ cfg_data.data.splits.val_name: int(cfg.num_val),
60
+ cfg_data.data.splits.test_name: int(cfg.num_test),
61
+ }
62
+ manifest: dict[str, list[str]] = {}
63
+ all_x, all_y = [], []
64
+ seed = int(cfg_train.seed)
65
+
66
+ for split, count in split_counts.items():
67
+ names = []
68
+ for idx in range(count):
69
+ name = f"{split}_{idx:04d}"
70
+ sample = make_sample(
71
+ num_nodes=int(cfg.num_nodes),
72
+ input_dim=int(cfg.input_dim),
73
+ output_dim=int(cfg.output_dim),
74
+ surface_ratio=float(cfg.surface_ratio),
75
+ seed=seed + len(all_x),
76
+ )
77
+ torch.save(sample, data_dir / f"{name}.pt")
78
+ names.append(name)
79
+ if split == cfg_data.data.splits.train_name:
80
+ all_x.append(sample.x.numpy())
81
+ all_y.append(sample.y.numpy())
82
+ manifest[split] = names
83
+
84
+ (data_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")
85
+ train_x = np.concatenate(all_x, axis=0)
86
+ train_y = np.concatenate(all_y, axis=0)
87
+ np.save(stats_dir / "mean_in.npy", train_x.mean(axis=0).astype(np.float32))
88
+ np.save(stats_dir / "std_in.npy", (train_x.std(axis=0) + 1e-6).astype(np.float32))
89
+ np.save(stats_dir / "mean_out.npy", train_y.mean(axis=0).astype(np.float32))
90
+ np.save(stats_dir / "std_out.npy", (train_y.std(axis=0) + 1e-6).astype(np.float32))
91
+ print(f"Fake AirfRANS graph data written to {data_dir}")
92
+ return 0
93
+
94
+
95
+ if __name__ == "__main__":
96
+ raise SystemExit(main())
scripts/inference.py ADDED
@@ -0,0 +1,177 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ import logging
5
+ import random
6
+ import sys
7
+ from pathlib import Path
8
+
9
+ import numpy as np
10
+ import torch
11
+ import torch.nn as nn
12
+
13
+ ROOT = Path(__file__).resolve().parents[1]
14
+ sys.path.insert(0, str(ROOT))
15
+
16
+ from onescience.distributed.manager import DistributedManager
17
+ from onescience.utils.YParams import YParams
18
+
19
+ from train import build_datapipe, build_model, resolve_path
20
+
21
+
22
+ def setup_logging() -> logging.Logger:
23
+ logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
24
+ return logging.getLogger("transolver-inference")
25
+
26
+
27
+ def load_checkpoint(path: Path, device: torch.device):
28
+ if not path.exists():
29
+ raise FileNotFoundError(f"Checkpoint not found: {path}. Run scripts/train.py first or provide a compatible weight.")
30
+ try:
31
+ return torch.load(path, map_location=device, weights_only=False)
32
+ except TypeError:
33
+ return torch.load(path, map_location=device)
34
+
35
+
36
+ def run_fake_inference(cfg_model, cfg_data, cfg_train, logger: logging.Logger, device: torch.device) -> int:
37
+ model_params = cfg_model.specific_params[cfg_model.name]
38
+ datapipe = build_datapipe(cfg_data, model_params, distributed=False)
39
+ test_loader = datapipe._loader(datapipe.test_dataset, False)[0]
40
+
41
+ model = build_model(cfg_model).to(device)
42
+ checkpoint_path = resolve_path(cfg_train.checkpoint_dir) / f"{cfg_model.name}.pth"
43
+ checkpoint = load_checkpoint(checkpoint_path, device)
44
+ model.load_state_dict(checkpoint["model_state_dict"])
45
+ model.eval()
46
+
47
+ result_dir = resolve_path(cfg_train.result_dir)
48
+ result_dir.mkdir(parents=True, exist_ok=True)
49
+ mean_out, std_out = datapipe.coef_norm[2], datapipe.coef_norm[3]
50
+ preds, targets = [], []
51
+ criterion = nn.MSELoss()
52
+
53
+ with torch.no_grad():
54
+ for data in test_loader:
55
+ data = data.to(device)
56
+ out = model(data)
57
+ pred = out.cpu().numpy() * (std_out + 1e-8) + mean_out
58
+ target = data.y.cpu().numpy() * (std_out + 1e-8) + mean_out
59
+ preds.append(pred)
60
+ targets.append(target)
61
+
62
+ pred_arr = np.concatenate(preds, axis=0)
63
+ target_arr = np.concatenate(targets, axis=0)
64
+ mse = float(criterion(torch.from_numpy(pred_arr), torch.from_numpy(target_arr)))
65
+ np.savez(result_dir / "predictions.npz", pred=pred_arr, target=target_arr, mse=np.array(mse, dtype=np.float32))
66
+ (result_dir / "score.json").write_text(json.dumps({"model_name": cfg_model.name, "mse": mse}, indent=2), encoding="utf-8")
67
+ logger.info("Saved fake inference outputs to %s", result_dir)
68
+ logger.info("MSE: %.6f", mse)
69
+ return 0
70
+
71
+
72
+ def run_airfrans_inference(cfg_model, cfg_data, cfg_train, logger: logging.Logger, device: torch.device) -> int:
73
+ import pyvista as pv
74
+ import scipy.stats as sc
75
+ from tqdm import tqdm
76
+ from onescience.utils.transolver.metrics import (
77
+ Airfoil_test,
78
+ Compute_coefficients,
79
+ Infer_test,
80
+ NumpyEncoder,
81
+ rel_err,
82
+ )
83
+
84
+ model_params = cfg_model.specific_params[cfg_model.name]
85
+ hparams = model_params
86
+ hparams["subsampling"] = cfg_data.data.subsampling
87
+ datapipe = build_datapipe(cfg_data, model_params, distributed=False)
88
+ coef_norm = datapipe.coef_norm
89
+ test_loader = datapipe.test_dataloader()
90
+ test_names = datapipe.test_dataset.data_list_names
91
+
92
+ model = build_model(cfg_model).to(device)
93
+ checkpoint_path = resolve_path(cfg_train.checkpoint_dir) / f"{cfg_model.name}.pth"
94
+ checkpoint = load_checkpoint(checkpoint_path, device)
95
+ model.load_state_dict(checkpoint["model_state_dict"])
96
+ model.eval()
97
+
98
+ path_in = resolve_path(cfg_data.source.data_dir)
99
+ path_out = resolve_path(cfg_train.result_dir) / cfg_data.data.splits.task
100
+ path_out.mkdir(parents=True, exist_ok=True)
101
+ sample_count = min(int(cfg_train.n_test), len(test_names))
102
+ chosen = sorted(random.sample(range(len(test_names)), k=sample_count)) if sample_count else []
103
+
104
+ criterion = nn.MSELoss(reduction="none")
105
+ scores_vol, scores_surf, scores_force, scores_p, scores_wss = [], [], [], [], []
106
+ times, true_coefs, pred_coefs = [], [], []
107
+
108
+ for j, data in enumerate(tqdm(test_loader, desc="Testing")):
109
+ sim_name = test_names[j]
110
+ u_inf, angle = float(sim_name.split("_")[2]), float(sim_name.split("_")[3])
111
+ outs, elapsed = Infer_test(device, [model], [hparams], data, coef_norm=coef_norm)
112
+ times.append(elapsed)
113
+
114
+ intern = pv.read(path_in / sim_name / f"{sim_name}_internal.vtu")
115
+ aerofoil = pv.read(path_in / sim_name / f"{sim_name}_aerofoil.vtp")
116
+ tc, true_intern, true_airfoil = Compute_coefficients([intern], [aerofoil], data.surf.cpu(), u_inf, angle, keep_vtk=True)
117
+ tc, true_airfoil = tc[0], true_airfoil[0]
118
+ intern_pred, aerofoil_pred = Airfoil_test(intern, aerofoil, outs, coef_norm, data.surf.cpu())
119
+ pc, intern_pred_vtk, aerofoil_pred_vtk = Compute_coefficients(intern_pred, aerofoil_pred, data.surf.cpu(), u_inf, angle, keep_vtk=True)
120
+ true_coefs.append(tc)
121
+ pred_coefs.append(pc)
122
+
123
+ out = outs[0]
124
+ scores_vol.append(criterion(out[~data.surf], data.y[~data.surf]).mean(dim=0).cpu().numpy())
125
+ scores_surf.append(criterion(out[data.surf], data.y[data.surf]).mean(dim=0).cpu().numpy())
126
+ scores_force.append(rel_err(tc, pc[0]))
127
+ scores_wss.append(rel_err(true_airfoil.point_data["wallShearStress"], aerofoil_pred_vtk[0].point_data["wallShearStress"]).mean(axis=0))
128
+ scores_p.append(rel_err(true_airfoil.point_data["p"], aerofoil_pred_vtk[0].point_data["p"]).mean(axis=0))
129
+ if j in chosen:
130
+ intern_pred_vtk[0].save(path_out / f"{sim_name}_pred_internal.vtu")
131
+ aerofoil_pred_vtk[0].save(path_out / f"{sim_name}_pred_aerofoil.vtp")
132
+
133
+ true_coefs_arr = np.array(true_coefs)
134
+ pred_coefs_arr = np.array(pred_coefs)
135
+ spear = [
136
+ sc.spearmanr(true_coefs_arr[:, 0], pred_coefs_arr[:, 0, 0])[0],
137
+ sc.spearmanr(true_coefs_arr[:, 1], pred_coefs_arr[:, 0, 1])[0],
138
+ ]
139
+ score = {
140
+ "model_name": cfg_model.name,
141
+ "mean_time": np.array(times).mean(axis=0),
142
+ "std_time": np.array(times).std(axis=0),
143
+ "mean_score_vol": np.array(scores_vol).mean(axis=0),
144
+ "mean_score_surf": np.array(scores_surf).mean(axis=0),
145
+ "mean_score_force": np.array(scores_force).mean(axis=0),
146
+ "mean_rel_p": np.array(scores_p).mean(axis=0),
147
+ "mean_rel_wss": np.array(scores_wss).mean(axis=0),
148
+ "spearman_coef": np.array(spear),
149
+ }
150
+ (path_out / f"score_{cfg_model.name}.json").write_text(json.dumps(score, indent=2, cls=NumpyEncoder), encoding="utf-8")
151
+ logger.info("Saved AirfRANS inference outputs to %s", path_out)
152
+ return 0
153
+
154
+
155
+ def main() -> int:
156
+ config_path = ROOT / "conf" / "config.yaml"
157
+ cfg_model = YParams(str(config_path), "model")
158
+ cfg_data = YParams(str(config_path), "datapipe")
159
+ cfg_train = YParams(str(config_path), "training")
160
+ logger = setup_logging()
161
+ DistributedManager.initialize()
162
+ device_name = getattr(cfg_train, "device", "auto")
163
+ if device_name == "cpu":
164
+ device = torch.device("cpu")
165
+ elif device_name.startswith("cuda"):
166
+ device = torch.device(device_name if torch.cuda.is_available() else "cpu")
167
+ else:
168
+ device = torch.device(f"cuda:{cfg_train.gpuid}" if torch.cuda.is_available() else "cpu")
169
+ if cfg_data.backend == "fake_airfrans":
170
+ return run_fake_inference(cfg_model, cfg_data, cfg_train, logger, device)
171
+ if cfg_data.backend == "airfrans":
172
+ return run_airfrans_inference(cfg_model, cfg_data, cfg_train, logger, device)
173
+ raise ValueError(f"Unsupported datapipe.backend: {cfg_data.backend}")
174
+
175
+
176
+ if __name__ == "__main__":
177
+ raise SystemExit(main())
scripts/result.py ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import json
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+
8
+ ROOT = Path(__file__).resolve().parents[1]
9
+ RESULT_DIR = ROOT / "result"
10
+
11
+
12
+ def main() -> int:
13
+ score_path = RESULT_DIR / "score.json"
14
+ pred_path = RESULT_DIR / "predictions.npz"
15
+ if score_path.exists():
16
+ print(score_path.read_text(encoding="utf-8"))
17
+ return 0
18
+ if not pred_path.exists():
19
+ raise FileNotFoundError(f"No result found at {score_path} or {pred_path}. Run scripts/inference.py first.")
20
+ data = np.load(pred_path)
21
+ mse = float(np.mean((data["pred"] - data["target"]) ** 2))
22
+ summary = {"mse": mse, "num_nodes": int(data["pred"].shape[0])}
23
+ print(json.dumps(summary, indent=2))
24
+ return 0
25
+
26
+
27
+ if __name__ == "__main__":
28
+ raise SystemExit(main())
scripts/train.py ADDED
@@ -0,0 +1,270 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ import random
5
+ import sys
6
+ import time
7
+ from pathlib import Path
8
+
9
+ import numpy as np
10
+ import torch
11
+ import torch.distributed as dist
12
+ import torch.nn as nn
13
+ from torch.nn.parallel import DistributedDataParallel
14
+ from torch.utils.data.distributed import DistributedSampler
15
+ from torch_geometric.loader import DataLoader as PyGDataLoader
16
+
17
+ ROOT = Path(__file__).resolve().parents[1]
18
+ sys.path.insert(0, str(ROOT))
19
+
20
+ from model import Transolver2D, Transolver2D_plus
21
+ from onescience.distributed.manager import DistributedManager
22
+ from onescience.utils.YParams import YParams
23
+
24
+
25
+ def resolve_path(path: str) -> Path:
26
+ value = Path(path).expanduser()
27
+ return value if value.is_absolute() else ROOT / value
28
+
29
+
30
+ def setup_logging(rank: int) -> logging.Logger:
31
+ logging.basicConfig(
32
+ level=logging.INFO if rank == 0 else logging.WARNING,
33
+ format="%(asctime)s - %(levelname)s - %(message)s",
34
+ datefmt="%Y-%m-%d %H:%M:%S",
35
+ )
36
+ return logging.getLogger("transolver-train")
37
+
38
+
39
+ def load_graph(path: Path):
40
+ try:
41
+ return torch.load(path, weights_only=False)
42
+ except TypeError:
43
+ return torch.load(path)
44
+
45
+
46
+ class FakeAirfRANSDataset(torch.utils.data.Dataset):
47
+ def __init__(self, data_dir: Path, stats_dir: Path, split: str, coef_norm=None):
48
+ manifest_path = data_dir / "manifest.json"
49
+ if not manifest_path.exists():
50
+ raise FileNotFoundError(f"Fake manifest not found: {manifest_path}. Run scripts/fake_data.py first.")
51
+ import json
52
+
53
+ manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
54
+ self.data_dir = data_dir
55
+ self.names = manifest[split]
56
+ self.coef_norm = coef_norm or (
57
+ np.load(stats_dir / "mean_in.npy"),
58
+ np.load(stats_dir / "std_in.npy"),
59
+ np.load(stats_dir / "mean_out.npy"),
60
+ np.load(stats_dir / "std_out.npy"),
61
+ )
62
+
63
+ def __len__(self) -> int:
64
+ return len(self.names)
65
+
66
+ def __getitem__(self, index: int):
67
+ data = load_graph(self.data_dir / f"{self.names[index]}.pt")
68
+ mean_in, std_in, mean_out, std_out = self.coef_norm
69
+ data.x = (data.x - torch.as_tensor(mean_in, dtype=data.x.dtype)) / torch.as_tensor(std_in + 1e-8, dtype=data.x.dtype)
70
+ data.y = (data.y - torch.as_tensor(mean_out, dtype=data.y.dtype)) / torch.as_tensor(std_out + 1e-8, dtype=data.y.dtype)
71
+ return data
72
+
73
+
74
+ class FakeAirfRANSDatapipe:
75
+ def __init__(self, cfg_data, distributed: bool):
76
+ self.cfg_data = cfg_data
77
+ self.distributed = distributed
78
+ data_dir = resolve_path(cfg_data.source.data_dir)
79
+ stats_dir = resolve_path(cfg_data.source.stats_dir)
80
+ self.train_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.train_name)
81
+ self.coef_norm = self.train_dataset.coef_norm
82
+ self.val_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.val_name, self.coef_norm)
83
+ self.test_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.test_name, self.coef_norm)
84
+
85
+ def _loader(self, dataset, shuffle: bool):
86
+ sampler = DistributedSampler(dataset, shuffle=shuffle) if self.distributed else None
87
+ return (
88
+ PyGDataLoader(
89
+ dataset,
90
+ batch_size=self.cfg_data.dataloader.batch_size,
91
+ num_workers=self.cfg_data.dataloader.num_workers,
92
+ pin_memory=torch.cuda.is_available(),
93
+ shuffle=shuffle and sampler is None,
94
+ sampler=sampler,
95
+ ),
96
+ sampler,
97
+ )
98
+
99
+ def train_dataloader(self):
100
+ return self._loader(self.train_dataset, True)
101
+
102
+ def val_dataloader(self):
103
+ return self._loader(self.val_dataset, False)
104
+
105
+
106
+ def build_model(cfg_model):
107
+ model_name = cfg_model.name
108
+ if model_name not in cfg_model.specific_params:
109
+ raise ValueError(f"Model '{model_name}' is missing from model.specific_params")
110
+ params = cfg_model.specific_params[model_name]
111
+ model_cls = Transolver2D_plus if model_name == "Transolver_plus" else Transolver2D
112
+ if model_name not in ("Transolver", "Transolver_plus"):
113
+ raise NotImplementedError("This refactored package localizes only Transolver and Transolver_plus.")
114
+ return model_cls(
115
+ n_hidden=params.n_hidden,
116
+ n_layers=params.n_layers,
117
+ space_dim=params.space_dim,
118
+ fun_dim=params.fun_dim,
119
+ n_head=params.n_head,
120
+ mlp_ratio=params.mlp_ratio,
121
+ out_dim=params.out_dim,
122
+ slice_num=params.slice_num,
123
+ unified_pos=bool(params.unified_pos),
124
+ )
125
+
126
+
127
+ def build_datapipe(cfg_data, model_params, distributed: bool):
128
+ cfg_data.model_hparams = model_params
129
+ if cfg_data.backend == "airfrans":
130
+ from onescience.datapipes.cfd import AirfRANSDatapipe
131
+
132
+ return AirfRANSDatapipe(params=cfg_data, distributed=distributed)
133
+ if cfg_data.backend == "fake_airfrans":
134
+ return FakeAirfRANSDatapipe(cfg_data, distributed=distributed)
135
+ raise ValueError(f"Unsupported datapipe.backend: {cfg_data.backend}")
136
+
137
+
138
+ def masked_mean(losses: torch.Tensor, mask: torch.Tensor) -> torch.Tensor:
139
+ if mask.any():
140
+ return losses[mask].mean()
141
+ return losses.mean() * 0.0
142
+
143
+
144
+ def save_checkpoint(model, optimizer, scheduler, epoch: int, loss: float, checkpoint_dir: Path, model_name: str) -> None:
145
+ checkpoint_dir.mkdir(parents=True, exist_ok=True)
146
+ model_to_save = model.module if hasattr(model, "module") else model
147
+ torch.save(
148
+ {
149
+ "model_state_dict": model_to_save.state_dict(),
150
+ "optimizer_state_dict": optimizer.state_dict(),
151
+ "scheduler_state_dict": scheduler.state_dict(),
152
+ "epoch": epoch,
153
+ "loss": loss,
154
+ },
155
+ checkpoint_dir / f"{model_name}.pth",
156
+ )
157
+
158
+
159
+ def main() -> int:
160
+ config_path = ROOT / "conf" / "config.yaml"
161
+ cfg_model = YParams(str(config_path), "model")
162
+ cfg_data = YParams(str(config_path), "datapipe")
163
+ cfg_train = YParams(str(config_path), "training")
164
+
165
+ torch.manual_seed(int(cfg_train.seed))
166
+ np.random.seed(int(cfg_train.seed))
167
+ random.seed(int(cfg_train.seed))
168
+
169
+ DistributedManager.initialize()
170
+ manager = DistributedManager()
171
+ logger = setup_logging(manager.rank)
172
+ device_name = getattr(cfg_train, "device", "auto")
173
+ if device_name == "cpu":
174
+ device = torch.device("cpu")
175
+ elif device_name.startswith("cuda"):
176
+ device = torch.device(device_name if torch.cuda.is_available() else "cpu")
177
+ else:
178
+ device = torch.device(f"cuda:{cfg_train.gpuid}" if torch.cuda.is_available() and manager.world_size == 1 else manager.device)
179
+
180
+ model_params = cfg_model.specific_params[cfg_model.name]
181
+ datapipe = build_datapipe(cfg_data, model_params, distributed=(manager.world_size > 1))
182
+ train_loader, train_sampler = datapipe.train_dataloader()
183
+ val_loader, val_sampler = datapipe.val_dataloader()
184
+ if len(train_loader) == 0 or len(val_loader) == 0:
185
+ raise RuntimeError("Train and validation loaders must both be non-empty.")
186
+
187
+ model = build_model(cfg_model).to(device)
188
+ if manager.world_size > 1:
189
+ model = DistributedDataParallel(model, device_ids=[manager.local_rank], output_device=manager.local_rank)
190
+
191
+ optimizer = torch.optim.Adam(model.parameters(), lr=float(cfg_train.lr))
192
+ scheduler = torch.optim.lr_scheduler.OneCycleLR(
193
+ optimizer,
194
+ max_lr=float(cfg_train.lr),
195
+ total_steps=max(1, len(train_loader) * int(cfg_train.max_epoch)),
196
+ )
197
+ criterion = nn.L1Loss(reduction="none") if cfg_train.loss_criterion == "MAE" else nn.MSELoss(reduction="none")
198
+ use_weighted_loss = cfg_train.loss_criterion == "MSE_weighted"
199
+ checkpoint_dir = resolve_path(cfg_train.checkpoint_dir)
200
+ best_valid_loss = float("inf")
201
+ best_epoch = -1
202
+
203
+ if manager.rank == 0:
204
+ params = sum(p.numel() for p in model.parameters() if p.requires_grad)
205
+ logger.info("Training %s with %.3fM parameters on %s", cfg_model.name, params / 1e6, device)
206
+
207
+ for epoch in range(int(cfg_train.max_epoch)):
208
+ start = time.time()
209
+ if train_sampler is not None:
210
+ train_sampler.set_epoch(epoch)
211
+ if val_sampler is not None:
212
+ val_sampler.set_epoch(epoch)
213
+
214
+ model.train()
215
+ train_loss = 0.0
216
+ for data in train_loader:
217
+ data = data.to(device)
218
+ optimizer.zero_grad(set_to_none=True)
219
+ out = model(data)
220
+ losses = criterion(out, data.y)
221
+ surf_loss = masked_mean(losses, data.surf)
222
+ vol_loss = masked_mean(losses, ~data.surf)
223
+ loss = vol_loss + float(cfg_train.loss_weight) * surf_loss if use_weighted_loss else losses.mean()
224
+ loss.backward()
225
+ optimizer.step()
226
+ scheduler.step()
227
+ train_loss += float(loss.detach().cpu())
228
+ train_loss /= len(train_loader)
229
+
230
+ model.eval()
231
+ valid_loss = 0.0
232
+ with torch.no_grad():
233
+ for data in val_loader:
234
+ data = data.to(device)
235
+ out = model(data)
236
+ losses = criterion(out, data.y)
237
+ surf_loss = masked_mean(losses, data.surf)
238
+ vol_loss = masked_mean(losses, ~data.surf)
239
+ loss = vol_loss + float(cfg_train.loss_weight) * surf_loss if use_weighted_loss else losses.mean()
240
+ if manager.world_size > 1:
241
+ dist.all_reduce(loss, op=dist.ReduceOp.AVG)
242
+ valid_loss += float(loss.detach().cpu())
243
+ valid_loss /= len(val_loader)
244
+
245
+ if manager.rank == 0:
246
+ logger.info(
247
+ "Epoch %d/%d train=%.6f valid=%.6f time=%.2fs",
248
+ epoch + 1,
249
+ int(cfg_train.max_epoch),
250
+ train_loss,
251
+ valid_loss,
252
+ time.time() - start,
253
+ )
254
+ if valid_loss < best_valid_loss:
255
+ best_valid_loss = valid_loss
256
+ best_epoch = epoch
257
+ save_checkpoint(model, optimizer, scheduler, epoch, valid_loss, checkpoint_dir, cfg_model.name)
258
+ logger.info("Saved checkpoint to %s", checkpoint_dir / f"{cfg_model.name}.pth")
259
+ if epoch - best_epoch > int(cfg_train.patience):
260
+ logger.info("Early stopping after %d epochs without improvement.", int(cfg_train.patience))
261
+ break
262
+
263
+ if manager.world_size > 1 and dist.is_available() and dist.is_initialized():
264
+ dist.barrier()
265
+ dist.destroy_process_group()
266
+ return 0
267
+
268
+
269
+ if __name__ == "__main__":
270
+ raise SystemExit(main())
weight/.gitkeep ADDED
File without changes