Upload folder using huggingface_hub
Browse files- README.md +122 -0
- conf/config.yaml +64 -0
- configuration.json +1 -0
- model/Transolver2D.py +136 -0
- model/Transolver2D_plus.py +133 -0
- model/__init__.py +4 -0
- scripts/__pycache__/train.cpython-311.pyc +0 -0
- scripts/fake_data.py +96 -0
- scripts/inference.py +177 -0
- scripts/result.py +28 -0
- scripts/train.py +270 -0
- weight/.gitkeep +0 -0
README.md
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
tags:
|
| 6 |
+
- OneScience
|
| 7 |
+
- fluid dynamics
|
| 8 |
+
- airfoil aerodynamic prediction
|
| 9 |
+
frameworks: PyTorch
|
| 10 |
+
---
|
| 11 |
+
<p align="center">
|
| 12 |
+
<strong>
|
| 13 |
+
<span style="font-size: 30px;">Transolver-Airfoil-Design</span>
|
| 14 |
+
</strong>
|
| 15 |
+
</p>
|
| 16 |
+
|
| 17 |
+
# Model Overview
|
| 18 |
+
|
| 19 |
+
Transolver-Airfoil-Design is a two-dimensional external-flow prediction model for airfoils, built on Transolver by Tsinghua University's THUML group. It rapidly predicts flow-field distributions and aerodynamic performance around airfoils.
|
| 20 |
+
|
| 21 |
+
Paper: [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://arxiv.org/abs/2402.02366)
|
| 22 |
+
|
| 23 |
+
# Model Description
|
| 24 |
+
Transolver-Airfoil-Design uses a Transformer architecture with Physics-Attention and is adapted for training on the unstructured-mesh AirfRANS airfoil dataset. It predicts velocity fields, pressure fields, and drag coefficients for airfoil geometries.
|
| 25 |
+
|
| 26 |
+
## Use Cases
|
| 27 |
+
|
| 28 |
+
| Use Case | Description |
|
| 29 |
+
| :--- | :--- |
|
| 30 |
+
| Airfoil aerodynamic design | Rapidly predict two-dimensional external flow around airfoils to screen candidate geometries |
|
| 31 |
+
| Industrial simulation acceleration | Accelerate large-scale simulations using PDE surrogate modeling on complex geometries |
|
| 32 |
+
|
| 33 |
+
# Usage
|
| 34 |
+
|
| 35 |
+
## 1. OneCode
|
| 36 |
+
|
| 37 |
+
Use the online OneCode environment for an intelligent, one-click AI for Science (AI4S) programming experience:
|
| 38 |
+
|
| 39 |
+
[Launch OneCode for one-click AI4S programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
|
| 40 |
+
|
| 41 |
+
## 2. Manual Setup
|
| 42 |
+
|
| 43 |
+
**Hardware Requirements**
|
| 44 |
+
|
| 45 |
+
- A GPU or DCU is recommended.
|
| 46 |
+
- A CPU can be used for import checks and small-scale pipeline validation, but full training and inference will be slow.
|
| 47 |
+
- DCU users must install DTK in advance. DTK 25.04.2 or later, or the OneScience-recommended version for the target cluster, is recommended.
|
| 48 |
+
|
| 49 |
+
### Download the Model Package
|
| 50 |
+
|
| 51 |
+
```bash
|
| 52 |
+
modelscope download --model OneScience/Transolver-Airfoil-Design --local_dir ./Transolver-Airfoil-Design
|
| 53 |
+
cd Transolver-Airfoil-Design
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
### Set Up the Runtime Environment
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
**DCU Environment**
|
| 60 |
+
|
| 61 |
+
```bash
|
| 62 |
+
# Activate DTK and Conda first
|
| 63 |
+
conda create -n onescience311 python=3.11 -y
|
| 64 |
+
conda activate onescience311
|
| 65 |
+
# Installation with uv is also supported
|
| 66 |
+
pip install onescience[cfd-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
**GPU Environment**
|
| 70 |
+
```bash
|
| 71 |
+
# Activate Conda first
|
| 72 |
+
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
|
| 73 |
+
conda activate onescience311
|
| 74 |
+
# Installation with uv is also supported
|
| 75 |
+
pip install onescience[cfd-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 76 |
+
```
|
| 77 |
+
|
| 78 |
+
### Training Data
|
| 79 |
+
The OneScience community provides the `airfrans` dataset for training. Download it with the command below and verify that the data path in `config/config.yaml` is configured correctly:
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
modelscope download --dataset OneScience/airfrans --local_dir ./data
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
### Training
|
| 86 |
+
|
| 87 |
+
```bash
|
| 88 |
+
python scripts/train.py
|
| 89 |
+
```
|
| 90 |
+
|
| 91 |
+
By default, training saves the following checkpoint:
|
| 92 |
+
|
| 93 |
+
```text
|
| 94 |
+
./weight/Transolver.pth
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
### Model Weights
|
| 98 |
+
This repository will provide model weights pretrained on ShapeNetCar data in the `weights/` directory. The weights will be uploaded soon.
|
| 99 |
+
|
| 100 |
+
### Inference
|
| 101 |
+
|
| 102 |
+
```bash
|
| 103 |
+
python scripts/inference.py
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
### Evaluation and Visualization
|
| 107 |
+
|
| 108 |
+
```bash
|
| 109 |
+
python scripts/result.py
|
| 110 |
+
```
|
| 111 |
+
|
| 112 |
+
# Official OneScience Resources
|
| 113 |
+
|
| 114 |
+
| Platform | OneScience Repository | Skills Repository |
|
| 115 |
+
| --- | --- | --- |
|
| 116 |
+
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
|
| 117 |
+
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
|
| 118 |
+
|
| 119 |
+
# Citations and License
|
| 120 |
+
|
| 121 |
+
- Original Transolver paper: [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://arxiv.org/pdf/2402.02366).
|
| 122 |
+
- This repository retains source attribution and has been adapted for automated execution through OneScience and ModelScope.
|
conf/config.yaml
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
model:
|
| 2 |
+
name: "Transolver"
|
| 3 |
+
specific_params:
|
| 4 |
+
Transolver: &transolver_params
|
| 5 |
+
n_hidden: 32
|
| 6 |
+
n_layers: 2
|
| 7 |
+
space_dim: 7
|
| 8 |
+
fun_dim: 0
|
| 9 |
+
n_head: 4
|
| 10 |
+
mlp_ratio: 2
|
| 11 |
+
out_dim: 4
|
| 12 |
+
slice_num: 8
|
| 13 |
+
unified_pos: 0
|
| 14 |
+
build_graph: false
|
| 15 |
+
r: 0.05
|
| 16 |
+
max_neighbors: 32
|
| 17 |
+
Transolver_plus: *transolver_params
|
| 18 |
+
|
| 19 |
+
datapipe:
|
| 20 |
+
name: "AirfRANS"
|
| 21 |
+
backend: "fake_airfrans"
|
| 22 |
+
verbose: false
|
| 23 |
+
source:
|
| 24 |
+
data_dir: "./data/fake_airfrans"
|
| 25 |
+
stats_dir: "./data/stats"
|
| 26 |
+
data:
|
| 27 |
+
splits:
|
| 28 |
+
task: "fake"
|
| 29 |
+
train_name: "train"
|
| 30 |
+
val_name: "val"
|
| 31 |
+
test_name: "test"
|
| 32 |
+
val_split_ratio: 0.25
|
| 33 |
+
sampling:
|
| 34 |
+
sample_strategy: null
|
| 35 |
+
n_boot: 128
|
| 36 |
+
surf_ratio: 0.25
|
| 37 |
+
crop: null
|
| 38 |
+
subsampling: 64
|
| 39 |
+
dataloader:
|
| 40 |
+
batch_size: 1
|
| 41 |
+
num_workers: 0
|
| 42 |
+
|
| 43 |
+
training:
|
| 44 |
+
device: "cpu"
|
| 45 |
+
max_epoch: 1
|
| 46 |
+
lr: 0.001
|
| 47 |
+
gpuid: 0
|
| 48 |
+
patience: 20
|
| 49 |
+
progress_log_interval: 5
|
| 50 |
+
loss_criterion: "MSE_weighted"
|
| 51 |
+
loss_weight: 1.0
|
| 52 |
+
checkpoint_dir: "./weight"
|
| 53 |
+
result_dir: "./result"
|
| 54 |
+
n_test: 2
|
| 55 |
+
seed: 42
|
| 56 |
+
|
| 57 |
+
fake_data:
|
| 58 |
+
num_train: 4
|
| 59 |
+
num_val: 2
|
| 60 |
+
num_test: 2
|
| 61 |
+
num_nodes: 64
|
| 62 |
+
input_dim: 7
|
| 63 |
+
output_dim: 4
|
| 64 |
+
surface_ratio: 0.25
|
configuration.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"framework":"Pytorch","task":"other","model":"Transolver-Airfoil-Design","pipeline_tag":"cfd-airfoil-surrogate"}
|
model/Transolver2D.py
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import torch
|
| 2 |
+
import torch.nn as nn
|
| 3 |
+
import numpy as np
|
| 4 |
+
try:
|
| 5 |
+
from timm.layers import trunc_normal_
|
| 6 |
+
except ImportError:
|
| 7 |
+
from torch.nn.init import trunc_normal_
|
| 8 |
+
from onescience.modules.mlp.MLP import StandardMLP
|
| 9 |
+
from onescience.modules.transformer.Transolver_block import Transolver_block
|
| 10 |
+
|
| 11 |
+
class Transolver2D(nn.Module):
|
| 12 |
+
"""
|
| 13 |
+
Transolver2D 模型。
|
| 14 |
+
|
| 15 |
+
该模型专为处理二维物理场问题(如 2D CFD, 弹性力学等)而设计。
|
| 16 |
+
架构与 Transolver3D 类似,但针对 2D 空间特性进行了配置。
|
| 17 |
+
它包含预处理 MLP 映射和多层 Transolver Block 堆叠,用于提取非结构化或结构化网格上的物理特征。
|
| 18 |
+
|
| 19 |
+
Args:
|
| 20 |
+
space_dim (int): 空间维度。默认值: 1。
|
| 21 |
+
n_layers (int): Transolver Block 的层数。默认值: 5。
|
| 22 |
+
n_hidden (int): 隐藏层特征维度。默认值: 256。
|
| 23 |
+
dropout (float): Dropout 概率。默认值: 0。
|
| 24 |
+
n_head (int): 注意力头数。默认值: 8。
|
| 25 |
+
act (str): 激活函数类型。默认值: 'gelu'。
|
| 26 |
+
mlp_ratio (float): MLP 膨胀比率。默认值: 1。
|
| 27 |
+
fun_dim (int): 输入物理场的特征维度。默认值: 1。
|
| 28 |
+
out_dim (int): 输出特征维度。默认值: 1。
|
| 29 |
+
slice_num (int): 物理注意力中的切片数量。默认值: 32。
|
| 30 |
+
ref (int): 统一位置编码的参考网格分辨率。默认值: 8。
|
| 31 |
+
unified_pos (bool): 是否使用统一位置编码。默认值: False。
|
| 32 |
+
|
| 33 |
+
形状:
|
| 34 |
+
输入 data: 包含 x (物理状态) 和 pos (坐标) 的数据对象。
|
| 35 |
+
输出: (N, out_dim),预测的物理场。
|
| 36 |
+
"""
|
| 37 |
+
def __init__(self,
|
| 38 |
+
space_dim=1,
|
| 39 |
+
n_layers=5,
|
| 40 |
+
n_hidden=256,
|
| 41 |
+
dropout=0,
|
| 42 |
+
n_head=8,
|
| 43 |
+
act='gelu',
|
| 44 |
+
mlp_ratio=1,
|
| 45 |
+
fun_dim=1,
|
| 46 |
+
out_dim=1,
|
| 47 |
+
slice_num=32,
|
| 48 |
+
ref=8,
|
| 49 |
+
unified_pos=False
|
| 50 |
+
):
|
| 51 |
+
super(Transolver2D, self).__init__()
|
| 52 |
+
self.__name__ = 'Transolver'
|
| 53 |
+
self.ref = ref
|
| 54 |
+
self.unified_pos = unified_pos
|
| 55 |
+
|
| 56 |
+
if self.unified_pos:
|
| 57 |
+
input_dim = fun_dim + space_dim + self.ref * self.ref
|
| 58 |
+
else:
|
| 59 |
+
input_dim = fun_dim + space_dim
|
| 60 |
+
|
| 61 |
+
self.preprocess = StandardMLP(
|
| 62 |
+
input_dim=input_dim,
|
| 63 |
+
hidden_dims=[n_hidden * 2],
|
| 64 |
+
output_dim=n_hidden,
|
| 65 |
+
activation=act,
|
| 66 |
+
use_bias=True,
|
| 67 |
+
use_skip_connection=False
|
| 68 |
+
)
|
| 69 |
+
self.n_hidden = n_hidden
|
| 70 |
+
self.space_dim = space_dim
|
| 71 |
+
|
| 72 |
+
self.blocks = nn.ModuleList([
|
| 73 |
+
Transolver_block(
|
| 74 |
+
num_heads=n_head,
|
| 75 |
+
hidden_dim=n_hidden,
|
| 76 |
+
dropout=dropout,
|
| 77 |
+
act=act,
|
| 78 |
+
mlp_ratio=mlp_ratio,
|
| 79 |
+
out_dim=out_dim,
|
| 80 |
+
slice_num=slice_num,
|
| 81 |
+
last_layer=(_ == n_layers - 1),
|
| 82 |
+
geotype='unstructured'
|
| 83 |
+
)
|
| 84 |
+
for _ in range(n_layers)
|
| 85 |
+
])
|
| 86 |
+
|
| 87 |
+
self.initialize_weights()
|
| 88 |
+
self.placeholder = nn.Parameter((1 / (n_hidden)) * torch.rand(n_hidden, dtype=torch.float))
|
| 89 |
+
|
| 90 |
+
def initialize_weights(self):
|
| 91 |
+
self.apply(self._init_weights)
|
| 92 |
+
|
| 93 |
+
def _init_weights(self, m):
|
| 94 |
+
if isinstance(m, nn.Linear):
|
| 95 |
+
trunc_normal_(m.weight, std=0.02)
|
| 96 |
+
if isinstance(m, nn.Linear) and m.bias is not None:
|
| 97 |
+
nn.init.constant_(m.bias, 0)
|
| 98 |
+
elif isinstance(m, (nn.LayerNorm, nn.BatchNorm1d)):
|
| 99 |
+
nn.init.constant_(m.bias, 0)
|
| 100 |
+
nn.init.constant_(m.weight, 1.0)
|
| 101 |
+
|
| 102 |
+
def get_grid(self, my_pos):
|
| 103 |
+
batchsize = my_pos.shape[0]
|
| 104 |
+
|
| 105 |
+
gridx = torch.tensor(np.linspace(-2, 4, self.ref), dtype=torch.float)
|
| 106 |
+
gridx = gridx.reshape(1, self.ref, 1, 1).repeat([batchsize, 1, self.ref, 1])
|
| 107 |
+
gridy = torch.tensor(np.linspace(-1.5, 1.5, self.ref), dtype=torch.float)
|
| 108 |
+
gridy = gridy.reshape(1, 1, self.ref, 1).repeat([batchsize, self.ref, 1, 1])
|
| 109 |
+
|
| 110 |
+
grid_ref = torch.cat((gridx, gridy), dim=-1).to(my_pos.device).reshape(batchsize, self.ref ** 2, 2) # B Ref^2 2
|
| 111 |
+
|
| 112 |
+
pos = torch.sqrt(
|
| 113 |
+
torch.sum((my_pos[:, :, None, :] - grid_ref[:, None, :, :]) ** 2,
|
| 114 |
+
dim=-1)). \
|
| 115 |
+
reshape(batchsize, my_pos.shape[1], self.ref * self.ref).contiguous()
|
| 116 |
+
return pos
|
| 117 |
+
|
| 118 |
+
def forward(self, data):
|
| 119 |
+
x, fx, T = data.x, None, None
|
| 120 |
+
x = x[None, :, :] # [1, N, C]
|
| 121 |
+
|
| 122 |
+
if self.unified_pos:
|
| 123 |
+
new_pos = self.get_grid(data.pos[None, :, :])
|
| 124 |
+
x = torch.cat((x, new_pos), dim=-1)
|
| 125 |
+
|
| 126 |
+
if fx is not None:
|
| 127 |
+
fx = torch.cat((x, fx), -1)
|
| 128 |
+
fx = self.preprocess(fx)
|
| 129 |
+
else:
|
| 130 |
+
fx = self.preprocess(x)
|
| 131 |
+
fx = fx + self.placeholder[None, None, :]
|
| 132 |
+
|
| 133 |
+
for block in self.blocks:
|
| 134 |
+
fx = block(fx)
|
| 135 |
+
|
| 136 |
+
return fx[0]
|
model/Transolver2D_plus.py
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import torch
|
| 2 |
+
import torch.nn as nn
|
| 3 |
+
import numpy as np
|
| 4 |
+
try:
|
| 5 |
+
from timm.layers import trunc_normal_
|
| 6 |
+
except ImportError:
|
| 7 |
+
from torch.nn.init import trunc_normal_
|
| 8 |
+
|
| 9 |
+
from onescience.modules.mlp.MLP import StandardMLP
|
| 10 |
+
from onescience.modules.transformer.Transolver_block import Transolver_block
|
| 11 |
+
|
| 12 |
+
class Transolver2D_plus(nn.Module):
|
| 13 |
+
"""
|
| 14 |
+
Transolver2D_plus 模型。
|
| 15 |
+
|
| 16 |
+
这是 Transolver2D 的增强版本。
|
| 17 |
+
主要的区别在于它使用了 'unstructured_plus' 类型的注意力机制,这种机制通常包含更复杂的切片或特征交互逻辑(如 Gumbel Softmax 采样等),以增强模型对不规则网格的捕捉能力。
|
| 18 |
+
|
| 19 |
+
Args:
|
| 20 |
+
space_dim (int): 空间维度。默认值: 1。
|
| 21 |
+
n_layers (int): Block 层数。默认值: 5。
|
| 22 |
+
n_hidden (int): 隐藏层维度。默认值: 256。
|
| 23 |
+
dropout (float): Dropout 概率。默认值: 0。
|
| 24 |
+
n_head (int): 注意力头数。默认值: 8。
|
| 25 |
+
act (str): 激活函数。默认值: 'gelu'。
|
| 26 |
+
mlp_ratio (float): MLP 膨胀比率。默认值: 1。
|
| 27 |
+
fun_dim (int): 输入物理场特征维度。默认值: 1。
|
| 28 |
+
out_dim (int): 输出特征维度。默认值: 1。
|
| 29 |
+
slice_num (int): 切片数量。默认值: 32。
|
| 30 |
+
ref (int): 统一位置编码参考分辨率。默认值: 8。
|
| 31 |
+
unified_pos (bool): 是否使用统一位置编码。默认值: False。
|
| 32 |
+
"""
|
| 33 |
+
def __init__(self,
|
| 34 |
+
space_dim=1,
|
| 35 |
+
n_layers=5,
|
| 36 |
+
n_hidden=256,
|
| 37 |
+
dropout=0,
|
| 38 |
+
n_head=8,
|
| 39 |
+
act='gelu',
|
| 40 |
+
mlp_ratio=1,
|
| 41 |
+
fun_dim=1,
|
| 42 |
+
out_dim=1,
|
| 43 |
+
slice_num=32,
|
| 44 |
+
ref=8,
|
| 45 |
+
unified_pos=False
|
| 46 |
+
):
|
| 47 |
+
super(Transolver2D_plus, self).__init__()
|
| 48 |
+
self.__name__ = 'Transolver'
|
| 49 |
+
self.ref = ref
|
| 50 |
+
self.unified_pos = unified_pos
|
| 51 |
+
|
| 52 |
+
if self.unified_pos:
|
| 53 |
+
input_dim = fun_dim + space_dim + self.ref * self.ref
|
| 54 |
+
else:
|
| 55 |
+
input_dim = fun_dim + space_dim
|
| 56 |
+
|
| 57 |
+
self.preprocess = StandardMLP(
|
| 58 |
+
input_dim=input_dim,
|
| 59 |
+
hidden_dims=[n_hidden * 2],
|
| 60 |
+
output_dim=n_hidden,
|
| 61 |
+
activation=act,
|
| 62 |
+
use_bias=True,
|
| 63 |
+
use_skip_connection=False
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
self.n_hidden = n_hidden
|
| 67 |
+
self.space_dim = space_dim
|
| 68 |
+
|
| 69 |
+
self.blocks = nn.ModuleList([
|
| 70 |
+
Transolver_block(
|
| 71 |
+
num_heads=n_head,
|
| 72 |
+
hidden_dim=n_hidden,
|
| 73 |
+
dropout=dropout,
|
| 74 |
+
act=act,
|
| 75 |
+
mlp_ratio=mlp_ratio,
|
| 76 |
+
out_dim=out_dim,
|
| 77 |
+
slice_num=slice_num,
|
| 78 |
+
last_layer=(_ == n_layers - 1),
|
| 79 |
+
geotype='unstructured_plus'
|
| 80 |
+
)
|
| 81 |
+
for _ in range(n_layers)
|
| 82 |
+
])
|
| 83 |
+
|
| 84 |
+
self.initialize_weights()
|
| 85 |
+
self.placeholder = nn.Parameter((1 / (n_hidden)) * torch.rand(n_hidden, dtype=torch.float))
|
| 86 |
+
|
| 87 |
+
def initialize_weights(self):
|
| 88 |
+
self.apply(self._init_weights)
|
| 89 |
+
|
| 90 |
+
def _init_weights(self, m):
|
| 91 |
+
if isinstance(m, nn.Linear):
|
| 92 |
+
trunc_normal_(m.weight, std=0.02)
|
| 93 |
+
if isinstance(m, nn.Linear) and m.bias is not None:
|
| 94 |
+
nn.init.constant_(m.bias, 0)
|
| 95 |
+
elif isinstance(m, (nn.LayerNorm, nn.BatchNorm1d)):
|
| 96 |
+
nn.init.constant_(m.bias, 0)
|
| 97 |
+
nn.init.constant_(m.weight, 1.0)
|
| 98 |
+
|
| 99 |
+
def get_grid(self, my_pos):
|
| 100 |
+
batchsize = my_pos.shape[0]
|
| 101 |
+
|
| 102 |
+
gridx = torch.tensor(np.linspace(-2, 4, self.ref), dtype=torch.float)
|
| 103 |
+
gridx = gridx.reshape(1, self.ref, 1, 1).repeat([batchsize, 1, self.ref, 1])
|
| 104 |
+
gridy = torch.tensor(np.linspace(-1.5, 1.5, self.ref), dtype=torch.float)
|
| 105 |
+
gridy = gridy.reshape(1, 1, self.ref, 1).repeat([batchsize, self.ref, 1, 1])
|
| 106 |
+
|
| 107 |
+
grid_ref = torch.cat((gridx, gridy), dim=-1).to(my_pos.device).reshape(batchsize, self.ref ** 2, 2)
|
| 108 |
+
|
| 109 |
+
pos = torch.sqrt(
|
| 110 |
+
torch.sum((my_pos[:, :, None, :] - grid_ref[:, None, :, :]) ** 2,
|
| 111 |
+
dim=-1)). \
|
| 112 |
+
reshape(batchsize, my_pos.shape[1], self.ref * self.ref).contiguous()
|
| 113 |
+
return pos
|
| 114 |
+
|
| 115 |
+
def forward(self, data):
|
| 116 |
+
x, fx, T = data.x, None, None
|
| 117 |
+
x = x[None, :, :]
|
| 118 |
+
|
| 119 |
+
if self.unified_pos:
|
| 120 |
+
new_pos = self.get_grid(data.pos[None, :, :])
|
| 121 |
+
x = torch.cat((x, new_pos), dim=-1)
|
| 122 |
+
|
| 123 |
+
if fx is not None:
|
| 124 |
+
fx = torch.cat((x, fx), -1)
|
| 125 |
+
fx = self.preprocess(fx)
|
| 126 |
+
else:
|
| 127 |
+
fx = self.preprocess(x)
|
| 128 |
+
fx = fx + self.placeholder[None, None, :]
|
| 129 |
+
|
| 130 |
+
for block in self.blocks:
|
| 131 |
+
fx = block(fx)
|
| 132 |
+
|
| 133 |
+
return fx[0]
|
model/__init__.py
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from .Transolver2D import Transolver2D
|
| 2 |
+
from .Transolver2D_plus import Transolver2D_plus
|
| 3 |
+
|
| 4 |
+
__all__ = ["Transolver2D", "Transolver2D_plus"]
|
scripts/__pycache__/train.cpython-311.pyc
ADDED
|
Binary file (18.5 kB). View file
|
|
|
scripts/fake_data.py
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import sys
|
| 5 |
+
from pathlib import Path
|
| 6 |
+
|
| 7 |
+
import numpy as np
|
| 8 |
+
import torch
|
| 9 |
+
from torch_geometric.data import Data
|
| 10 |
+
|
| 11 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 12 |
+
sys.path.insert(0, str(ROOT))
|
| 13 |
+
|
| 14 |
+
from onescience.utils.YParams import YParams
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def resolve_path(path: str) -> Path:
|
| 18 |
+
value = Path(path).expanduser()
|
| 19 |
+
return value if value.is_absolute() else ROOT / value
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def make_sample(num_nodes: int, input_dim: int, output_dim: int, surface_ratio: float, seed: int) -> Data:
|
| 23 |
+
generator = torch.Generator().manual_seed(seed)
|
| 24 |
+
pos = torch.rand((num_nodes, 2), generator=generator)
|
| 25 |
+
freestream = torch.ones((num_nodes, 2))
|
| 26 |
+
freestream[:, 1] = torch.linspace(-0.2, 0.2, num_nodes)
|
| 27 |
+
sdf = torch.linalg.norm(pos - 0.5, dim=1, keepdim=True) - 0.25
|
| 28 |
+
normals = torch.nn.functional.normalize(pos - 0.5, dim=1)
|
| 29 |
+
x = torch.cat([pos, freestream, sdf, normals], dim=1)
|
| 30 |
+
if x.size(1) != input_dim:
|
| 31 |
+
raise ValueError(f"fake input_dim must be 7 for this AirfRANS feature layout, got {input_dim}")
|
| 32 |
+
|
| 33 |
+
y0 = torch.sin(np.pi * pos[:, :1]) + 0.1 * freestream[:, :1]
|
| 34 |
+
y1 = torch.cos(np.pi * pos[:, 1:2]) + 0.1 * freestream[:, 1:2]
|
| 35 |
+
pressure = pos[:, :1] - pos[:, 1:2]
|
| 36 |
+
viscosity = torch.abs(sdf) + 0.01
|
| 37 |
+
y = torch.cat([y0, y1, pressure, viscosity], dim=1)
|
| 38 |
+
if y.size(1) != output_dim:
|
| 39 |
+
raise ValueError(f"fake output_dim must be 4 for this AirfRANS target layout, got {output_dim}")
|
| 40 |
+
|
| 41 |
+
surf = torch.zeros(num_nodes, dtype=torch.bool)
|
| 42 |
+
surf_count = max(1, int(num_nodes * surface_ratio))
|
| 43 |
+
surf[:surf_count] = True
|
| 44 |
+
perm = torch.randperm(num_nodes, generator=generator)
|
| 45 |
+
return Data(pos=pos[perm], x=x[perm].float(), y=y[perm].float(), surf=surf[perm])
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def main() -> int:
|
| 49 |
+
cfg = YParams(str(ROOT / "conf" / "config.yaml"), "fake_data")
|
| 50 |
+
cfg_data = YParams(str(ROOT / "conf" / "config.yaml"), "datapipe")
|
| 51 |
+
cfg_train = YParams(str(ROOT / "conf" / "config.yaml"), "training")
|
| 52 |
+
data_dir = resolve_path(cfg_data.source.data_dir)
|
| 53 |
+
stats_dir = resolve_path(cfg_data.source.stats_dir)
|
| 54 |
+
data_dir.mkdir(parents=True, exist_ok=True)
|
| 55 |
+
stats_dir.mkdir(parents=True, exist_ok=True)
|
| 56 |
+
|
| 57 |
+
split_counts = {
|
| 58 |
+
cfg_data.data.splits.train_name: int(cfg.num_train),
|
| 59 |
+
cfg_data.data.splits.val_name: int(cfg.num_val),
|
| 60 |
+
cfg_data.data.splits.test_name: int(cfg.num_test),
|
| 61 |
+
}
|
| 62 |
+
manifest: dict[str, list[str]] = {}
|
| 63 |
+
all_x, all_y = [], []
|
| 64 |
+
seed = int(cfg_train.seed)
|
| 65 |
+
|
| 66 |
+
for split, count in split_counts.items():
|
| 67 |
+
names = []
|
| 68 |
+
for idx in range(count):
|
| 69 |
+
name = f"{split}_{idx:04d}"
|
| 70 |
+
sample = make_sample(
|
| 71 |
+
num_nodes=int(cfg.num_nodes),
|
| 72 |
+
input_dim=int(cfg.input_dim),
|
| 73 |
+
output_dim=int(cfg.output_dim),
|
| 74 |
+
surface_ratio=float(cfg.surface_ratio),
|
| 75 |
+
seed=seed + len(all_x),
|
| 76 |
+
)
|
| 77 |
+
torch.save(sample, data_dir / f"{name}.pt")
|
| 78 |
+
names.append(name)
|
| 79 |
+
if split == cfg_data.data.splits.train_name:
|
| 80 |
+
all_x.append(sample.x.numpy())
|
| 81 |
+
all_y.append(sample.y.numpy())
|
| 82 |
+
manifest[split] = names
|
| 83 |
+
|
| 84 |
+
(data_dir / "manifest.json").write_text(json.dumps(manifest, indent=2), encoding="utf-8")
|
| 85 |
+
train_x = np.concatenate(all_x, axis=0)
|
| 86 |
+
train_y = np.concatenate(all_y, axis=0)
|
| 87 |
+
np.save(stats_dir / "mean_in.npy", train_x.mean(axis=0).astype(np.float32))
|
| 88 |
+
np.save(stats_dir / "std_in.npy", (train_x.std(axis=0) + 1e-6).astype(np.float32))
|
| 89 |
+
np.save(stats_dir / "mean_out.npy", train_y.mean(axis=0).astype(np.float32))
|
| 90 |
+
np.save(stats_dir / "std_out.npy", (train_y.std(axis=0) + 1e-6).astype(np.float32))
|
| 91 |
+
print(f"Fake AirfRANS graph data written to {data_dir}")
|
| 92 |
+
return 0
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
if __name__ == "__main__":
|
| 96 |
+
raise SystemExit(main())
|
scripts/inference.py
ADDED
|
@@ -0,0 +1,177 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
import logging
|
| 5 |
+
import random
|
| 6 |
+
import sys
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
import torch
|
| 11 |
+
import torch.nn as nn
|
| 12 |
+
|
| 13 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 14 |
+
sys.path.insert(0, str(ROOT))
|
| 15 |
+
|
| 16 |
+
from onescience.distributed.manager import DistributedManager
|
| 17 |
+
from onescience.utils.YParams import YParams
|
| 18 |
+
|
| 19 |
+
from train import build_datapipe, build_model, resolve_path
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def setup_logging() -> logging.Logger:
|
| 23 |
+
logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
|
| 24 |
+
return logging.getLogger("transolver-inference")
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def load_checkpoint(path: Path, device: torch.device):
|
| 28 |
+
if not path.exists():
|
| 29 |
+
raise FileNotFoundError(f"Checkpoint not found: {path}. Run scripts/train.py first or provide a compatible weight.")
|
| 30 |
+
try:
|
| 31 |
+
return torch.load(path, map_location=device, weights_only=False)
|
| 32 |
+
except TypeError:
|
| 33 |
+
return torch.load(path, map_location=device)
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def run_fake_inference(cfg_model, cfg_data, cfg_train, logger: logging.Logger, device: torch.device) -> int:
|
| 37 |
+
model_params = cfg_model.specific_params[cfg_model.name]
|
| 38 |
+
datapipe = build_datapipe(cfg_data, model_params, distributed=False)
|
| 39 |
+
test_loader = datapipe._loader(datapipe.test_dataset, False)[0]
|
| 40 |
+
|
| 41 |
+
model = build_model(cfg_model).to(device)
|
| 42 |
+
checkpoint_path = resolve_path(cfg_train.checkpoint_dir) / f"{cfg_model.name}.pth"
|
| 43 |
+
checkpoint = load_checkpoint(checkpoint_path, device)
|
| 44 |
+
model.load_state_dict(checkpoint["model_state_dict"])
|
| 45 |
+
model.eval()
|
| 46 |
+
|
| 47 |
+
result_dir = resolve_path(cfg_train.result_dir)
|
| 48 |
+
result_dir.mkdir(parents=True, exist_ok=True)
|
| 49 |
+
mean_out, std_out = datapipe.coef_norm[2], datapipe.coef_norm[3]
|
| 50 |
+
preds, targets = [], []
|
| 51 |
+
criterion = nn.MSELoss()
|
| 52 |
+
|
| 53 |
+
with torch.no_grad():
|
| 54 |
+
for data in test_loader:
|
| 55 |
+
data = data.to(device)
|
| 56 |
+
out = model(data)
|
| 57 |
+
pred = out.cpu().numpy() * (std_out + 1e-8) + mean_out
|
| 58 |
+
target = data.y.cpu().numpy() * (std_out + 1e-8) + mean_out
|
| 59 |
+
preds.append(pred)
|
| 60 |
+
targets.append(target)
|
| 61 |
+
|
| 62 |
+
pred_arr = np.concatenate(preds, axis=0)
|
| 63 |
+
target_arr = np.concatenate(targets, axis=0)
|
| 64 |
+
mse = float(criterion(torch.from_numpy(pred_arr), torch.from_numpy(target_arr)))
|
| 65 |
+
np.savez(result_dir / "predictions.npz", pred=pred_arr, target=target_arr, mse=np.array(mse, dtype=np.float32))
|
| 66 |
+
(result_dir / "score.json").write_text(json.dumps({"model_name": cfg_model.name, "mse": mse}, indent=2), encoding="utf-8")
|
| 67 |
+
logger.info("Saved fake inference outputs to %s", result_dir)
|
| 68 |
+
logger.info("MSE: %.6f", mse)
|
| 69 |
+
return 0
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def run_airfrans_inference(cfg_model, cfg_data, cfg_train, logger: logging.Logger, device: torch.device) -> int:
|
| 73 |
+
import pyvista as pv
|
| 74 |
+
import scipy.stats as sc
|
| 75 |
+
from tqdm import tqdm
|
| 76 |
+
from onescience.utils.transolver.metrics import (
|
| 77 |
+
Airfoil_test,
|
| 78 |
+
Compute_coefficients,
|
| 79 |
+
Infer_test,
|
| 80 |
+
NumpyEncoder,
|
| 81 |
+
rel_err,
|
| 82 |
+
)
|
| 83 |
+
|
| 84 |
+
model_params = cfg_model.specific_params[cfg_model.name]
|
| 85 |
+
hparams = model_params
|
| 86 |
+
hparams["subsampling"] = cfg_data.data.subsampling
|
| 87 |
+
datapipe = build_datapipe(cfg_data, model_params, distributed=False)
|
| 88 |
+
coef_norm = datapipe.coef_norm
|
| 89 |
+
test_loader = datapipe.test_dataloader()
|
| 90 |
+
test_names = datapipe.test_dataset.data_list_names
|
| 91 |
+
|
| 92 |
+
model = build_model(cfg_model).to(device)
|
| 93 |
+
checkpoint_path = resolve_path(cfg_train.checkpoint_dir) / f"{cfg_model.name}.pth"
|
| 94 |
+
checkpoint = load_checkpoint(checkpoint_path, device)
|
| 95 |
+
model.load_state_dict(checkpoint["model_state_dict"])
|
| 96 |
+
model.eval()
|
| 97 |
+
|
| 98 |
+
path_in = resolve_path(cfg_data.source.data_dir)
|
| 99 |
+
path_out = resolve_path(cfg_train.result_dir) / cfg_data.data.splits.task
|
| 100 |
+
path_out.mkdir(parents=True, exist_ok=True)
|
| 101 |
+
sample_count = min(int(cfg_train.n_test), len(test_names))
|
| 102 |
+
chosen = sorted(random.sample(range(len(test_names)), k=sample_count)) if sample_count else []
|
| 103 |
+
|
| 104 |
+
criterion = nn.MSELoss(reduction="none")
|
| 105 |
+
scores_vol, scores_surf, scores_force, scores_p, scores_wss = [], [], [], [], []
|
| 106 |
+
times, true_coefs, pred_coefs = [], [], []
|
| 107 |
+
|
| 108 |
+
for j, data in enumerate(tqdm(test_loader, desc="Testing")):
|
| 109 |
+
sim_name = test_names[j]
|
| 110 |
+
u_inf, angle = float(sim_name.split("_")[2]), float(sim_name.split("_")[3])
|
| 111 |
+
outs, elapsed = Infer_test(device, [model], [hparams], data, coef_norm=coef_norm)
|
| 112 |
+
times.append(elapsed)
|
| 113 |
+
|
| 114 |
+
intern = pv.read(path_in / sim_name / f"{sim_name}_internal.vtu")
|
| 115 |
+
aerofoil = pv.read(path_in / sim_name / f"{sim_name}_aerofoil.vtp")
|
| 116 |
+
tc, true_intern, true_airfoil = Compute_coefficients([intern], [aerofoil], data.surf.cpu(), u_inf, angle, keep_vtk=True)
|
| 117 |
+
tc, true_airfoil = tc[0], true_airfoil[0]
|
| 118 |
+
intern_pred, aerofoil_pred = Airfoil_test(intern, aerofoil, outs, coef_norm, data.surf.cpu())
|
| 119 |
+
pc, intern_pred_vtk, aerofoil_pred_vtk = Compute_coefficients(intern_pred, aerofoil_pred, data.surf.cpu(), u_inf, angle, keep_vtk=True)
|
| 120 |
+
true_coefs.append(tc)
|
| 121 |
+
pred_coefs.append(pc)
|
| 122 |
+
|
| 123 |
+
out = outs[0]
|
| 124 |
+
scores_vol.append(criterion(out[~data.surf], data.y[~data.surf]).mean(dim=0).cpu().numpy())
|
| 125 |
+
scores_surf.append(criterion(out[data.surf], data.y[data.surf]).mean(dim=0).cpu().numpy())
|
| 126 |
+
scores_force.append(rel_err(tc, pc[0]))
|
| 127 |
+
scores_wss.append(rel_err(true_airfoil.point_data["wallShearStress"], aerofoil_pred_vtk[0].point_data["wallShearStress"]).mean(axis=0))
|
| 128 |
+
scores_p.append(rel_err(true_airfoil.point_data["p"], aerofoil_pred_vtk[0].point_data["p"]).mean(axis=0))
|
| 129 |
+
if j in chosen:
|
| 130 |
+
intern_pred_vtk[0].save(path_out / f"{sim_name}_pred_internal.vtu")
|
| 131 |
+
aerofoil_pred_vtk[0].save(path_out / f"{sim_name}_pred_aerofoil.vtp")
|
| 132 |
+
|
| 133 |
+
true_coefs_arr = np.array(true_coefs)
|
| 134 |
+
pred_coefs_arr = np.array(pred_coefs)
|
| 135 |
+
spear = [
|
| 136 |
+
sc.spearmanr(true_coefs_arr[:, 0], pred_coefs_arr[:, 0, 0])[0],
|
| 137 |
+
sc.spearmanr(true_coefs_arr[:, 1], pred_coefs_arr[:, 0, 1])[0],
|
| 138 |
+
]
|
| 139 |
+
score = {
|
| 140 |
+
"model_name": cfg_model.name,
|
| 141 |
+
"mean_time": np.array(times).mean(axis=0),
|
| 142 |
+
"std_time": np.array(times).std(axis=0),
|
| 143 |
+
"mean_score_vol": np.array(scores_vol).mean(axis=0),
|
| 144 |
+
"mean_score_surf": np.array(scores_surf).mean(axis=0),
|
| 145 |
+
"mean_score_force": np.array(scores_force).mean(axis=0),
|
| 146 |
+
"mean_rel_p": np.array(scores_p).mean(axis=0),
|
| 147 |
+
"mean_rel_wss": np.array(scores_wss).mean(axis=0),
|
| 148 |
+
"spearman_coef": np.array(spear),
|
| 149 |
+
}
|
| 150 |
+
(path_out / f"score_{cfg_model.name}.json").write_text(json.dumps(score, indent=2, cls=NumpyEncoder), encoding="utf-8")
|
| 151 |
+
logger.info("Saved AirfRANS inference outputs to %s", path_out)
|
| 152 |
+
return 0
|
| 153 |
+
|
| 154 |
+
|
| 155 |
+
def main() -> int:
|
| 156 |
+
config_path = ROOT / "conf" / "config.yaml"
|
| 157 |
+
cfg_model = YParams(str(config_path), "model")
|
| 158 |
+
cfg_data = YParams(str(config_path), "datapipe")
|
| 159 |
+
cfg_train = YParams(str(config_path), "training")
|
| 160 |
+
logger = setup_logging()
|
| 161 |
+
DistributedManager.initialize()
|
| 162 |
+
device_name = getattr(cfg_train, "device", "auto")
|
| 163 |
+
if device_name == "cpu":
|
| 164 |
+
device = torch.device("cpu")
|
| 165 |
+
elif device_name.startswith("cuda"):
|
| 166 |
+
device = torch.device(device_name if torch.cuda.is_available() else "cpu")
|
| 167 |
+
else:
|
| 168 |
+
device = torch.device(f"cuda:{cfg_train.gpuid}" if torch.cuda.is_available() else "cpu")
|
| 169 |
+
if cfg_data.backend == "fake_airfrans":
|
| 170 |
+
return run_fake_inference(cfg_model, cfg_data, cfg_train, logger, device)
|
| 171 |
+
if cfg_data.backend == "airfrans":
|
| 172 |
+
return run_airfrans_inference(cfg_model, cfg_data, cfg_train, logger, device)
|
| 173 |
+
raise ValueError(f"Unsupported datapipe.backend: {cfg_data.backend}")
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
if __name__ == "__main__":
|
| 177 |
+
raise SystemExit(main())
|
scripts/result.py
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import json
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
import numpy as np
|
| 7 |
+
|
| 8 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 9 |
+
RESULT_DIR = ROOT / "result"
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def main() -> int:
|
| 13 |
+
score_path = RESULT_DIR / "score.json"
|
| 14 |
+
pred_path = RESULT_DIR / "predictions.npz"
|
| 15 |
+
if score_path.exists():
|
| 16 |
+
print(score_path.read_text(encoding="utf-8"))
|
| 17 |
+
return 0
|
| 18 |
+
if not pred_path.exists():
|
| 19 |
+
raise FileNotFoundError(f"No result found at {score_path} or {pred_path}. Run scripts/inference.py first.")
|
| 20 |
+
data = np.load(pred_path)
|
| 21 |
+
mse = float(np.mean((data["pred"] - data["target"]) ** 2))
|
| 22 |
+
summary = {"mse": mse, "num_nodes": int(data["pred"].shape[0])}
|
| 23 |
+
print(json.dumps(summary, indent=2))
|
| 24 |
+
return 0
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
if __name__ == "__main__":
|
| 28 |
+
raise SystemExit(main())
|
scripts/train.py
ADDED
|
@@ -0,0 +1,270 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from __future__ import annotations
|
| 2 |
+
|
| 3 |
+
import logging
|
| 4 |
+
import random
|
| 5 |
+
import sys
|
| 6 |
+
import time
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
import torch
|
| 11 |
+
import torch.distributed as dist
|
| 12 |
+
import torch.nn as nn
|
| 13 |
+
from torch.nn.parallel import DistributedDataParallel
|
| 14 |
+
from torch.utils.data.distributed import DistributedSampler
|
| 15 |
+
from torch_geometric.loader import DataLoader as PyGDataLoader
|
| 16 |
+
|
| 17 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 18 |
+
sys.path.insert(0, str(ROOT))
|
| 19 |
+
|
| 20 |
+
from model import Transolver2D, Transolver2D_plus
|
| 21 |
+
from onescience.distributed.manager import DistributedManager
|
| 22 |
+
from onescience.utils.YParams import YParams
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def resolve_path(path: str) -> Path:
|
| 26 |
+
value = Path(path).expanduser()
|
| 27 |
+
return value if value.is_absolute() else ROOT / value
|
| 28 |
+
|
| 29 |
+
|
| 30 |
+
def setup_logging(rank: int) -> logging.Logger:
|
| 31 |
+
logging.basicConfig(
|
| 32 |
+
level=logging.INFO if rank == 0 else logging.WARNING,
|
| 33 |
+
format="%(asctime)s - %(levelname)s - %(message)s",
|
| 34 |
+
datefmt="%Y-%m-%d %H:%M:%S",
|
| 35 |
+
)
|
| 36 |
+
return logging.getLogger("transolver-train")
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def load_graph(path: Path):
|
| 40 |
+
try:
|
| 41 |
+
return torch.load(path, weights_only=False)
|
| 42 |
+
except TypeError:
|
| 43 |
+
return torch.load(path)
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
class FakeAirfRANSDataset(torch.utils.data.Dataset):
|
| 47 |
+
def __init__(self, data_dir: Path, stats_dir: Path, split: str, coef_norm=None):
|
| 48 |
+
manifest_path = data_dir / "manifest.json"
|
| 49 |
+
if not manifest_path.exists():
|
| 50 |
+
raise FileNotFoundError(f"Fake manifest not found: {manifest_path}. Run scripts/fake_data.py first.")
|
| 51 |
+
import json
|
| 52 |
+
|
| 53 |
+
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
| 54 |
+
self.data_dir = data_dir
|
| 55 |
+
self.names = manifest[split]
|
| 56 |
+
self.coef_norm = coef_norm or (
|
| 57 |
+
np.load(stats_dir / "mean_in.npy"),
|
| 58 |
+
np.load(stats_dir / "std_in.npy"),
|
| 59 |
+
np.load(stats_dir / "mean_out.npy"),
|
| 60 |
+
np.load(stats_dir / "std_out.npy"),
|
| 61 |
+
)
|
| 62 |
+
|
| 63 |
+
def __len__(self) -> int:
|
| 64 |
+
return len(self.names)
|
| 65 |
+
|
| 66 |
+
def __getitem__(self, index: int):
|
| 67 |
+
data = load_graph(self.data_dir / f"{self.names[index]}.pt")
|
| 68 |
+
mean_in, std_in, mean_out, std_out = self.coef_norm
|
| 69 |
+
data.x = (data.x - torch.as_tensor(mean_in, dtype=data.x.dtype)) / torch.as_tensor(std_in + 1e-8, dtype=data.x.dtype)
|
| 70 |
+
data.y = (data.y - torch.as_tensor(mean_out, dtype=data.y.dtype)) / torch.as_tensor(std_out + 1e-8, dtype=data.y.dtype)
|
| 71 |
+
return data
|
| 72 |
+
|
| 73 |
+
|
| 74 |
+
class FakeAirfRANSDatapipe:
|
| 75 |
+
def __init__(self, cfg_data, distributed: bool):
|
| 76 |
+
self.cfg_data = cfg_data
|
| 77 |
+
self.distributed = distributed
|
| 78 |
+
data_dir = resolve_path(cfg_data.source.data_dir)
|
| 79 |
+
stats_dir = resolve_path(cfg_data.source.stats_dir)
|
| 80 |
+
self.train_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.train_name)
|
| 81 |
+
self.coef_norm = self.train_dataset.coef_norm
|
| 82 |
+
self.val_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.val_name, self.coef_norm)
|
| 83 |
+
self.test_dataset = FakeAirfRANSDataset(data_dir, stats_dir, cfg_data.data.splits.test_name, self.coef_norm)
|
| 84 |
+
|
| 85 |
+
def _loader(self, dataset, shuffle: bool):
|
| 86 |
+
sampler = DistributedSampler(dataset, shuffle=shuffle) if self.distributed else None
|
| 87 |
+
return (
|
| 88 |
+
PyGDataLoader(
|
| 89 |
+
dataset,
|
| 90 |
+
batch_size=self.cfg_data.dataloader.batch_size,
|
| 91 |
+
num_workers=self.cfg_data.dataloader.num_workers,
|
| 92 |
+
pin_memory=torch.cuda.is_available(),
|
| 93 |
+
shuffle=shuffle and sampler is None,
|
| 94 |
+
sampler=sampler,
|
| 95 |
+
),
|
| 96 |
+
sampler,
|
| 97 |
+
)
|
| 98 |
+
|
| 99 |
+
def train_dataloader(self):
|
| 100 |
+
return self._loader(self.train_dataset, True)
|
| 101 |
+
|
| 102 |
+
def val_dataloader(self):
|
| 103 |
+
return self._loader(self.val_dataset, False)
|
| 104 |
+
|
| 105 |
+
|
| 106 |
+
def build_model(cfg_model):
|
| 107 |
+
model_name = cfg_model.name
|
| 108 |
+
if model_name not in cfg_model.specific_params:
|
| 109 |
+
raise ValueError(f"Model '{model_name}' is missing from model.specific_params")
|
| 110 |
+
params = cfg_model.specific_params[model_name]
|
| 111 |
+
model_cls = Transolver2D_plus if model_name == "Transolver_plus" else Transolver2D
|
| 112 |
+
if model_name not in ("Transolver", "Transolver_plus"):
|
| 113 |
+
raise NotImplementedError("This refactored package localizes only Transolver and Transolver_plus.")
|
| 114 |
+
return model_cls(
|
| 115 |
+
n_hidden=params.n_hidden,
|
| 116 |
+
n_layers=params.n_layers,
|
| 117 |
+
space_dim=params.space_dim,
|
| 118 |
+
fun_dim=params.fun_dim,
|
| 119 |
+
n_head=params.n_head,
|
| 120 |
+
mlp_ratio=params.mlp_ratio,
|
| 121 |
+
out_dim=params.out_dim,
|
| 122 |
+
slice_num=params.slice_num,
|
| 123 |
+
unified_pos=bool(params.unified_pos),
|
| 124 |
+
)
|
| 125 |
+
|
| 126 |
+
|
| 127 |
+
def build_datapipe(cfg_data, model_params, distributed: bool):
|
| 128 |
+
cfg_data.model_hparams = model_params
|
| 129 |
+
if cfg_data.backend == "airfrans":
|
| 130 |
+
from onescience.datapipes.cfd import AirfRANSDatapipe
|
| 131 |
+
|
| 132 |
+
return AirfRANSDatapipe(params=cfg_data, distributed=distributed)
|
| 133 |
+
if cfg_data.backend == "fake_airfrans":
|
| 134 |
+
return FakeAirfRANSDatapipe(cfg_data, distributed=distributed)
|
| 135 |
+
raise ValueError(f"Unsupported datapipe.backend: {cfg_data.backend}")
|
| 136 |
+
|
| 137 |
+
|
| 138 |
+
def masked_mean(losses: torch.Tensor, mask: torch.Tensor) -> torch.Tensor:
|
| 139 |
+
if mask.any():
|
| 140 |
+
return losses[mask].mean()
|
| 141 |
+
return losses.mean() * 0.0
|
| 142 |
+
|
| 143 |
+
|
| 144 |
+
def save_checkpoint(model, optimizer, scheduler, epoch: int, loss: float, checkpoint_dir: Path, model_name: str) -> None:
|
| 145 |
+
checkpoint_dir.mkdir(parents=True, exist_ok=True)
|
| 146 |
+
model_to_save = model.module if hasattr(model, "module") else model
|
| 147 |
+
torch.save(
|
| 148 |
+
{
|
| 149 |
+
"model_state_dict": model_to_save.state_dict(),
|
| 150 |
+
"optimizer_state_dict": optimizer.state_dict(),
|
| 151 |
+
"scheduler_state_dict": scheduler.state_dict(),
|
| 152 |
+
"epoch": epoch,
|
| 153 |
+
"loss": loss,
|
| 154 |
+
},
|
| 155 |
+
checkpoint_dir / f"{model_name}.pth",
|
| 156 |
+
)
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def main() -> int:
|
| 160 |
+
config_path = ROOT / "conf" / "config.yaml"
|
| 161 |
+
cfg_model = YParams(str(config_path), "model")
|
| 162 |
+
cfg_data = YParams(str(config_path), "datapipe")
|
| 163 |
+
cfg_train = YParams(str(config_path), "training")
|
| 164 |
+
|
| 165 |
+
torch.manual_seed(int(cfg_train.seed))
|
| 166 |
+
np.random.seed(int(cfg_train.seed))
|
| 167 |
+
random.seed(int(cfg_train.seed))
|
| 168 |
+
|
| 169 |
+
DistributedManager.initialize()
|
| 170 |
+
manager = DistributedManager()
|
| 171 |
+
logger = setup_logging(manager.rank)
|
| 172 |
+
device_name = getattr(cfg_train, "device", "auto")
|
| 173 |
+
if device_name == "cpu":
|
| 174 |
+
device = torch.device("cpu")
|
| 175 |
+
elif device_name.startswith("cuda"):
|
| 176 |
+
device = torch.device(device_name if torch.cuda.is_available() else "cpu")
|
| 177 |
+
else:
|
| 178 |
+
device = torch.device(f"cuda:{cfg_train.gpuid}" if torch.cuda.is_available() and manager.world_size == 1 else manager.device)
|
| 179 |
+
|
| 180 |
+
model_params = cfg_model.specific_params[cfg_model.name]
|
| 181 |
+
datapipe = build_datapipe(cfg_data, model_params, distributed=(manager.world_size > 1))
|
| 182 |
+
train_loader, train_sampler = datapipe.train_dataloader()
|
| 183 |
+
val_loader, val_sampler = datapipe.val_dataloader()
|
| 184 |
+
if len(train_loader) == 0 or len(val_loader) == 0:
|
| 185 |
+
raise RuntimeError("Train and validation loaders must both be non-empty.")
|
| 186 |
+
|
| 187 |
+
model = build_model(cfg_model).to(device)
|
| 188 |
+
if manager.world_size > 1:
|
| 189 |
+
model = DistributedDataParallel(model, device_ids=[manager.local_rank], output_device=manager.local_rank)
|
| 190 |
+
|
| 191 |
+
optimizer = torch.optim.Adam(model.parameters(), lr=float(cfg_train.lr))
|
| 192 |
+
scheduler = torch.optim.lr_scheduler.OneCycleLR(
|
| 193 |
+
optimizer,
|
| 194 |
+
max_lr=float(cfg_train.lr),
|
| 195 |
+
total_steps=max(1, len(train_loader) * int(cfg_train.max_epoch)),
|
| 196 |
+
)
|
| 197 |
+
criterion = nn.L1Loss(reduction="none") if cfg_train.loss_criterion == "MAE" else nn.MSELoss(reduction="none")
|
| 198 |
+
use_weighted_loss = cfg_train.loss_criterion == "MSE_weighted"
|
| 199 |
+
checkpoint_dir = resolve_path(cfg_train.checkpoint_dir)
|
| 200 |
+
best_valid_loss = float("inf")
|
| 201 |
+
best_epoch = -1
|
| 202 |
+
|
| 203 |
+
if manager.rank == 0:
|
| 204 |
+
params = sum(p.numel() for p in model.parameters() if p.requires_grad)
|
| 205 |
+
logger.info("Training %s with %.3fM parameters on %s", cfg_model.name, params / 1e6, device)
|
| 206 |
+
|
| 207 |
+
for epoch in range(int(cfg_train.max_epoch)):
|
| 208 |
+
start = time.time()
|
| 209 |
+
if train_sampler is not None:
|
| 210 |
+
train_sampler.set_epoch(epoch)
|
| 211 |
+
if val_sampler is not None:
|
| 212 |
+
val_sampler.set_epoch(epoch)
|
| 213 |
+
|
| 214 |
+
model.train()
|
| 215 |
+
train_loss = 0.0
|
| 216 |
+
for data in train_loader:
|
| 217 |
+
data = data.to(device)
|
| 218 |
+
optimizer.zero_grad(set_to_none=True)
|
| 219 |
+
out = model(data)
|
| 220 |
+
losses = criterion(out, data.y)
|
| 221 |
+
surf_loss = masked_mean(losses, data.surf)
|
| 222 |
+
vol_loss = masked_mean(losses, ~data.surf)
|
| 223 |
+
loss = vol_loss + float(cfg_train.loss_weight) * surf_loss if use_weighted_loss else losses.mean()
|
| 224 |
+
loss.backward()
|
| 225 |
+
optimizer.step()
|
| 226 |
+
scheduler.step()
|
| 227 |
+
train_loss += float(loss.detach().cpu())
|
| 228 |
+
train_loss /= len(train_loader)
|
| 229 |
+
|
| 230 |
+
model.eval()
|
| 231 |
+
valid_loss = 0.0
|
| 232 |
+
with torch.no_grad():
|
| 233 |
+
for data in val_loader:
|
| 234 |
+
data = data.to(device)
|
| 235 |
+
out = model(data)
|
| 236 |
+
losses = criterion(out, data.y)
|
| 237 |
+
surf_loss = masked_mean(losses, data.surf)
|
| 238 |
+
vol_loss = masked_mean(losses, ~data.surf)
|
| 239 |
+
loss = vol_loss + float(cfg_train.loss_weight) * surf_loss if use_weighted_loss else losses.mean()
|
| 240 |
+
if manager.world_size > 1:
|
| 241 |
+
dist.all_reduce(loss, op=dist.ReduceOp.AVG)
|
| 242 |
+
valid_loss += float(loss.detach().cpu())
|
| 243 |
+
valid_loss /= len(val_loader)
|
| 244 |
+
|
| 245 |
+
if manager.rank == 0:
|
| 246 |
+
logger.info(
|
| 247 |
+
"Epoch %d/%d train=%.6f valid=%.6f time=%.2fs",
|
| 248 |
+
epoch + 1,
|
| 249 |
+
int(cfg_train.max_epoch),
|
| 250 |
+
train_loss,
|
| 251 |
+
valid_loss,
|
| 252 |
+
time.time() - start,
|
| 253 |
+
)
|
| 254 |
+
if valid_loss < best_valid_loss:
|
| 255 |
+
best_valid_loss = valid_loss
|
| 256 |
+
best_epoch = epoch
|
| 257 |
+
save_checkpoint(model, optimizer, scheduler, epoch, valid_loss, checkpoint_dir, cfg_model.name)
|
| 258 |
+
logger.info("Saved checkpoint to %s", checkpoint_dir / f"{cfg_model.name}.pth")
|
| 259 |
+
if epoch - best_epoch > int(cfg_train.patience):
|
| 260 |
+
logger.info("Early stopping after %d epochs without improvement.", int(cfg_train.patience))
|
| 261 |
+
break
|
| 262 |
+
|
| 263 |
+
if manager.world_size > 1 and dist.is_available() and dist.is_initialized():
|
| 264 |
+
dist.barrier()
|
| 265 |
+
dist.destroy_process_group()
|
| 266 |
+
return 0
|
| 267 |
+
|
| 268 |
+
|
| 269 |
+
if __name__ == "__main__":
|
| 270 |
+
raise SystemExit(main())
|
weight/.gitkeep
ADDED
|
File without changes
|