Sync YOLO training and evaluation utilities (part 2)

PREDICTION_THRESHOLD_FIX.md

@@ -0,0 +1,94 @@
+# 浒苔预测阈值修复说明
+
+## 问题描述
+
+当处理整张没有浒苔的256x256图片时，模型可能会把整张图片全部预测为浒苔。这是因为：
+
+1. **原始预测逻辑**：使用 `torch.argmax()` 直接选择概率最高的类别
+2. **问题原因**：当两个类别的概率接近时（比如都是0.5左右），`argmax` 会选择概率稍高的那个
+3. **类别不平衡**：如果训练数据中浒苔样本较多，模型可能倾向于预测浒苔类别
+
+## 解决方案
+
+添加了**概率阈值机制**：
+
+- **修改前**：直接使用 `argmax` 选择概率最高的类别
+- **修改后**：只有当浒苔（类别1）的概率**超过阈值**（默认0.5）时，才预测为浒苔
+
+### 代码修改
+
+在 `predict_image()` 函数中：
+
+```python
+# 修改前
+pred_class = torch.argmax(pred, dim=1).squeeze(0).cpu().numpy()
+
+# 修改后
+seaweed_prob = pred[0, 1].squeeze(0).cpu().numpy()
+pred_class = (seaweed_prob > threshold).astype(np.uint8)
+```
+
+## 使用方法
+
+### 1. 基本使用（使用默认阈值0.5）
+
+直接运行预测脚本即可，默认阈值为0.5：
+
+```bash
+python predict_seaweed_256x256_simple.py
+```
+
+### 2. 自定义阈值
+
+在脚本的 `main()` 函数中修改 `prediction_threshold` 变量：
+
+```python
+# 预测阈值：只有当浒苔概率超过此阈值时才预测为浒苔
+# 建议值：0.5-0.7，可以根据验证集调整
+prediction_threshold = 0.5  # 修改为你需要的值
+```
+
+### 3. 阈值选择建议
+
+- **0.5**：平衡阈值，适合大多数情况
+- **0.6-0.7**：更保守，减少误报（假阳性），但可能漏检一些浒苔
+- **0.4-0.5**：更宽松，减少漏检（假阴性），但可能增加误报
+
+**建议**：在验证集上测试不同阈值，选择F1分数最高的阈值。
+
+## 修改的文件
+
+1. `predict_seaweed_256x256_simple.py` - 单张256x256图片预测脚本
+2. `predict_large_image_tiles_256.py` - 大图瓦片预测脚本
+
+## 验证方法
+
+运行预测后，检查以下内容：
+
+1. **概率图**：查看 `*_probability.png` 文件，确认无浒苔图片的概率值是否低于阈值
+2. **预测结果**：查看 `*_prediction.png` 文件，确认无浒苔图片是否被正确预测为背景
+3. **统计信息**：查看 `prediction_results.json`，检查无浒苔图片的 `seaweed_ratio` 是否接近0
+
+## 进一步优化建议
+
+如果问题仍然存在，可以考虑：
+
+1. **调整阈值**：根据验证集结果调整阈值
+2. **重新训练模型**：增加无浒苔样本的训练数据
+3. **使用类别权重**：在训练时给背景类别更高的权重
+4. **后处理**：添加形态学操作（如开运算）去除小的误检区域
+
+## 技术细节
+
+### 为什么会出现这个问题？
+
+1. **Softmax特性**：softmax会将logits转换为概率分布，即使两个类别的logits很接近，softmax后的概率也会强制归一化
+2. **模型倾向性**：如果训练数据中浒苔样本较多，模型可能学习到倾向于预测浒苔的模式
+3. **边界情况**：当输入特征不明显时，模型可能给出不确定的预测
+
+### 阈值机制的工作原理
+
+- 阈值机制相当于在softmax概率上添加了一个"置信度门控"
+- 只有当模型对浒苔的预测**足够确信**（概率>阈值）时，才预测为浒苔
+- 这样可以有效减少在不确定情况下的误判
+

TRAINING_IMPROVEMENTS.md

@@ -0,0 +1,140 @@
+# 训练改进说明
+
+## 问题诊断
+
+即使有1000张负样本，模型仍然会把无浒苔图片和陆地/海岛预测为浒苔。这说明：
+
+1. **类别不平衡问题**：每张图片中背景像素通常远多于浒苔像素
+2. **损失函数问题**：原始损失函数没有给浒苔类别足够的权重
+3. **模型可能学习到"总是预测背景"的策略**
+
+## 改进方案
+
+### 1. 改进损失函数
+
+#### 1.1 Focal Loss改进
+- **之前**：`alpha=1`，所有类别权重相同
+- **现在**：`alpha=[1.0, 3.0]`，给浒苔类别3倍权重
+- **添加**：类别权重tensor，在交叉熵中应用
+
+#### 1.2 Dice Loss改进
+- **之前**：使用`mean()`，所有类别权重相同
+- **现在**：对每个类别计算dice loss，然后应用类别权重加权平均
+
+#### 1.3 类别权重
+- **背景权重**：1.0（默认）
+- **前景权重**：3.0（可调整，建议2.0-5.0）
+
+### 2. 检查训练数据
+
+运行 `check_training_data.py` 来：
+- 统计类别分布
+- 计算建议的类别权重
+- 检查数据平衡情况
+
+```bash
+python check_training_data.py
+```
+
+## 使用方法
+
+### 步骤1：检查训练数据
+
+```bash
+python check_training_data.py
+```
+
+这会输出：
+- 类别分布统计
+- 建议的类别权重
+- 推荐的训练配置
+
+### 步骤2：更新训练配置
+
+根据检查结果，更新 `train_config.json` 或使用 `train_config_improved.json`：
+
+```json
+{
+  "focal_alpha": [1.0, 3.0],
+  "background_weight": 1.0,
+  "foreground_weight": 3.0,
+  ...
+}
+```
+
+### 步骤3：重新训练
+
+```bash
+python train_seaweed_segmentation.py --config train_config_improved.json
+```
+
+## 参数说明
+
+### focal_alpha
+- **类型**：列表 `[背景权重, 前景权重]`
+- **默认**：`[1.0, 3.0]`
+- **说明**：Focal Loss的alpha参数，给前景更高的权重
+
+### background_weight / foreground_weight
+- **类型**：浮点数
+- **默认**：`1.0` / `3.0`
+- **说明**：类别权重，用于交叉熵和Dice Loss
+
+### 如何选择权重
+
+1. **运行数据检查脚本**，查看实际的类别分布
+2. **根据建议调整权重**：
+   - 如果浒苔像素 < 1%：使用 `foreground_weight >= 5.0`
+   - 如果浒苔像素 1-5%：使用 `foreground_weight = 3.0-5.0`
+   - 如果浒苔像素 > 5%：使用 `foreground_weight = 2.0-3.0`
+
+## 预期效果
+
+改进后的损失函数应该能够：
+1. **更好地学习浒苔特征**：给浒苔更高的权重，模型会更关注浒苔区域
+2. **减少误判**：模型不会简单地"总是预测背景"
+3. **提高精度**：在保持召回率的同时，减少假阳性
+
+## 进一步优化
+
+如果问题仍然存在：
+
+### 1. 增加前景权重
+```json
+"foreground_weight": 5.0,
+"focal_alpha": [1.0, 5.0]
+```
+
+### 2. 调整损失函数权重
+```json
+"dice_weight": 0.7,  // 增加Dice Loss权重
+"focal_weight": 1.0
+```
+
+### 3. 使用难例挖掘
+- 重点关注模型误判的样本
+- 增加这些样本在训练中的权重
+
+### 4. 检查数据质量
+- 确保标签正确
+- 检查是否有误标注
+- 确保负样本（无浒苔图片）的标签是全0
+
+## 验证方法
+
+训练后，检查：
+1. **训练损失**：应该逐渐下降
+2. **验证IoU**：应该逐渐上升
+3. **预测结果**：无浒苔图片应该被正确预测为背景
+4. **混淆矩阵**：假阳性（FP）应该减少
+
+## 注意事项
+
+1. **权重不要过大**：过大的权重可能导致训练不稳定
+2. **监控训练过程**：如果损失不下降，可能需要调整权重
+3. **验证集评估**：使用验证集来评估改进效果
+
+
+
+
+

improved_loss_functions.py

@@ -0,0 +1,220 @@
+"""
+改进的损失函数 - 更好地处理类别不平衡
+"""
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import numpy as np
+
+class ImprovedFocalLoss(nn.Module):
+    """
+    改进的Focal Loss，支持类别权重
+    """
+    
+    def __init__(self, alpha=None, gamma=2, ignore_index=255, reduction='mean', class_weights=None):
+        """
+        Args:
+            alpha: 类别权重列表，如果为None则使用平衡权重
+            gamma: 聚焦参数
+            ignore_index: 忽略的索引
+            reduction: 降维方式
+            class_weights: 类别权重tensor，shape为(num_classes,)
+        """
+        super().__init__()
+        if alpha is None:
+            alpha = [1.0, 1.0]  # 默认平衡权重
+        if isinstance(alpha, (list, tuple)):
+            alpha = torch.tensor(alpha, dtype=torch.float32)
+        self.alpha = alpha
+        self.gamma = gamma
+        self.ignore_index = ignore_index
+        self.reduction = reduction
+        self.class_weights = class_weights
+        
+    def forward(self, inputs, targets):
+        """
+        Args:
+            inputs: 预测值 (N, C, H, W)
+            targets: 目标值 (N, H, W)
+        """
+        # 处理多输出格式
+        if isinstance(inputs, dict):
+            inputs = inputs['out']
+        
+        # 忽略指定索引
+        if self.ignore_index is not None:
+            mask = targets != self.ignore_index
+            targets = targets[mask]
+            inputs = inputs.permute(0, 2, 3, 1)[mask]
+        else:
+            inputs = inputs.permute(0, 2, 3, 1).contiguous().view(-1, inputs.size(1))
+            targets = targets.view(-1)
+        
+        # 计算交叉熵
+        ce_loss = F.cross_entropy(inputs, targets, reduction='none', weight=self.class_weights)
+        pt = torch.exp(-ce_loss)
+        
+        # 应用alpha权重
+        if self.alpha is not None:
+            if self.alpha.device != targets.device:
+                self.alpha = self.alpha.to(targets.device)
+            alpha_t = self.alpha[targets]
+            focal_loss = alpha_t * (1 - pt) ** self.gamma * ce_loss
+        else:
+            focal_loss = (1 - pt) ** self.gamma * ce_loss
+        
+        if self.reduction == 'mean':
+            return focal_loss.mean()
+        elif self.reduction == 'sum':
+            return focal_loss.sum()
+        else:
+            return focal_loss
+
+
+class WeightedDiceLoss(nn.Module):
+    """
+    加权Dice Loss，给不同类别不同的权重
+    """
+    
+    def __init__(self, num_classes=2, smooth=1e-6, class_weights=None):
+        """
+        Args:
+            num_classes: 类别数
+            smooth: 平滑系数
+            class_weights: 类别权重tensor，shape为(num_classes,)
+        """
+        super().__init__()
+        self.num_classes = num_classes
+        self.smooth = smooth
+        self.class_weights = class_weights
+        
+    def forward(self, inputs, targets):
+        """
+        Args:
+            inputs: 预测值 (N, C, H, W)
+            targets: 目标值 (N, H, W)
+        """
+        # 处理多输出格式
+        if isinstance(inputs, dict):
+            inputs = inputs['out']
+        
+        # 将预测转换为概率
+        inputs = torch.softmax(inputs, dim=1)
+        
+        # 创建one-hot编码
+        targets_one_hot = F.one_hot(targets, num_classes=self.num_classes)
+        targets_one_hot = targets_one_hot.permute(0, 3, 1, 2).float()
+        
+        # 计算每个类别的dice系数
+        intersection = (inputs * targets_one_hot).sum(dim=(2, 3))  # (N, C)
+        union = inputs.sum(dim=(2, 3)) + targets_one_hot.sum(dim=(2, 3))  # (N, C)
+        
+        dice_score = (2. * intersection + self.smooth) / (union + self.smooth)  # (N, C)
+        
+        # 计算每个类别的dice loss
+        dice_loss_per_class = 1 - dice_score  # (N, C)
+        
+        # 应用类别权重
+        if self.class_weights is not None:
+            if self.class_weights.device != dice_loss_per_class.device:
+                self.class_weights = self.class_weights.to(dice_loss_per_class.device)
+            # 对每个类别应用权重
+            weighted_dice_loss = dice_loss_per_class * self.class_weights.unsqueeze(0)
+            dice_loss = weighted_dice_loss.mean()
+        else:
+            dice_loss = dice_loss_per_class.mean()
+        
+        return dice_loss
+
+
+class ImprovedSeaweedSegmentationLoss(nn.Module):
+    """
+    改进的浒苔分割损失函数
+    更好地处理类别不平衡，特别是背景像素远多于浒苔像素的情况
+    """
+    
+    def __init__(self, num_classes=2, focal_alpha=None, focal_gamma=2, 
+                 dice_weight=0.5, focal_weight=1.0, 
+                 background_weight=1.0, foreground_weight=2.0):
+        """
+        Args:
+            num_classes: 类别数
+            focal_alpha: Focal Loss的alpha参数（类别权重）
+            focal_gamma: Focal Loss的gamma参数
+            dice_weight: Dice Loss的权重
+            focal_weight: Focal Loss的权重
+            background_weight: 背景类别的权重（类别0）
+            foreground_weight: 前景类别（浒苔）的权重（类别1）
+        """
+        super().__init__()
+        
+        # 创建类别权重
+        class_weights = torch.tensor([background_weight, foreground_weight], dtype=torch.float32)
+        
+        # 创建Focal Loss的alpha权重
+        if focal_alpha is None:
+            # 默认给前景更高的权重
+            focal_alpha = [background_weight, foreground_weight]
+        
+        self.focal_loss = ImprovedFocalLoss(
+            alpha=focal_alpha, 
+            gamma=focal_gamma,
+            class_weights=class_weights
+        )
+        
+        self.dice_loss = WeightedDiceLoss(
+            num_classes=num_classes,
+            class_weights=class_weights
+        )
+        
+        self.dice_weight = dice_weight
+        self.focal_weight = focal_weight
+        self.num_classes = num_classes
+        
+    def forward(self, inputs, targets):
+        """计算组合损失"""
+        focal_loss = self.focal_loss(inputs, targets)
+        dice_loss = self.dice_loss(inputs, targets)
+        
+        total_loss = self.focal_weight * focal_loss + self.dice_weight * dice_loss
+        
+        return {
+            'total_loss': total_loss,
+            'focal_loss': focal_loss,
+            'dice_loss': dice_loss
+        }
+
+
+def calculate_class_weights_from_dataset(dataset, num_classes=2):
+    """
+    从数据集中计算类别权重
+    用于平衡类别不平衡问题
+    """
+    print("正在计算类别权重...")
+    total_pixels = 0
+    class_counts = torch.zeros(num_classes)
+    
+    for i in range(len(dataset)):
+        sample = dataset[i]
+        mask = sample['mask']
+        
+        # 统计每个类别的像素数
+        for c in range(num_classes):
+            class_counts[c] += (mask == c).sum().item()
+        
+        total_pixels += mask.numel()
+    
+    # 计算权重：总像素数 / (类别数 * 该类别的像素数)
+    class_weights = total_pixels / (num_classes * class_counts + 1e-6)
+    
+    # 归一化权重
+    class_weights = class_weights / class_weights.sum() * num_classes
+    
+    print(f"类别像素统计: {class_counts.tolist()}")
+    print(f"类别权重: {class_weights.tolist()}")
+    
+    return class_weights
+
+
+
+

large_image_concat.py

@@ -0,0 +1,640 @@
+import json
+import os
+import sys
+from PIL import Image
+import numpy as np
+import cv2
+from osgeo import gdal, ogr, osr
+
+# Increase PIL image size limit to handle large images
+Image.MAX_IMAGE_PIXELS = None
+
+# =======================================================
+# I. 栅格处理工具函数 (基于您的代码，进行优化和整合)
+# =======================================================
+
+def create_geotiff_from_png(png_path, ref_tiff_path, out_tiff_path):
+    """
+    将拼接好的二值PNG图转换为具有地理参考信息的GeoTIFF。
+    使用GDAL直接读取PNG，避免内存问题。
+    """
+    print(f"\n--- 🌐 步骤 II: 转换为 GeoTIFF ---")
+    
+    # 1. 打开原始 GeoTIFF 获取地理信息
+    ds_ref = gdal.Open(ref_tiff_path, gdal.GA_ReadOnly)
+    if ds_ref is None:
+        raise RuntimeError(f"致命错误: 无法打开原始参考文件 {ref_tiff_path}")
+    
+    geo_transform = ds_ref.GetGeoTransform()
+    projection = ds_ref.GetProjection()
+    
+    # 2. 使用GDAL直接打开PNG文件（避免PIL内存问题）
+    print("⏳ 正在读取PNG文件...")
+    try:
+        # 使用GDAL打开PNG
+        ds_png = gdal.Open(png_path, gdal.GA_ReadOnly)
+        if ds_png is None:
+            raise RuntimeError(f"无法使用GDAL打开PNG文件: {png_path}")
+        
+        # 获取PNG尺寸
+        png_width = ds_png.RasterXSize
+        png_height = ds_png.RasterYSize
+        
+        print(f"✅ PNG尺寸: {png_width} x {png_height}")
+        
+        # 读取PNG数据（分块读取以节省内存）
+        png_band = ds_png.GetRasterBand(1)
+        
+    except Exception as e:
+        # 如果GDAL无法打开PNG，尝试使用PIL（但分块处理）
+        print(f"⚠️  GDAL无法打开PNG，尝试使用PIL分块读取...")
+        try:
+            img_png = Image.open(png_path)
+            png_width, png_height = img_png.size
+            
+            # 分块读取（每次读取1000行）
+            chunk_size = 1000
+            data_chunks = []
+            for y_start in range(0, png_height, chunk_size):
+                y_end = min(y_start + chunk_size, png_height)
+                box = (0, y_start, png_width, y_end)
+                chunk = np.array(img_png.crop(box).convert('L'))
+                data_chunks.append(chunk)
+                if (y_start // chunk_size + 1) % 10 == 0:
+                    print(f"   已读取 {y_end}/{png_height} 行...")
+            
+            # 合并数据块
+            data = np.vstack(data_chunks)
+            img_png.close()
+            
+        except Exception as e2:
+            raise RuntimeError(f"读取PNG图像失败: {e2}")
+
+    # 3. 创建新的 GeoTIFF 文件
+    driver = gdal.GetDriverByName("GTiff")
+    if os.path.exists(out_tiff_path):
+        driver.Delete(out_tiff_path) # 确保覆盖旧文件
+        
+    print(f"⏳ 正在创建GeoTIFF文件...")
+    ds_new = driver.Create(
+        out_tiff_path,
+        png_width,  # XSize (Width)
+        png_height, # YSize (Height)
+        1,          # Band Count
+        gdal.GDT_Byte, # 确保使用 8位无符号整型存储二值数据 (0/1/255)
+        options=['COMPRESS=DEFLATE', 'NUM_THREADS=ALL_CPUS', 'TILED=YES', 'BLOCKXSIZE=256', 'BLOCKYSIZE=256']
+    )
+    
+    # 4. 设置地理信息和写入数据
+    ds_new.SetGeoTransform(geo_transform)
+    ds_new.SetProjection(projection)
+    
+    band = ds_new.GetRasterBand(1)
+    
+    # 分块写入数据（如果使用GDAL读取）
+    if 'ds_png' in locals():
+        print("⏳ 正在分块写入数据...")
+        block_size = 1000  # 每次写入1000行
+        for y_start in range(0, png_height, block_size):
+            y_end = min(y_start + block_size, png_height)
+            data_chunk = png_band.ReadAsArray(0, y_start, png_width, y_end - y_start)
+            band.WriteArray(data_chunk, 0, y_start)
+            if (y_start // block_size + 1) % 10 == 0:
+                print(f"   已写入 {y_end}/{png_height} 行...")
+        ds_png = None
+    else:
+        # 如果使用PIL读取，直接写入
+        print("⏳ 正在写入数据...")
+        band.WriteArray(data)
+    
+    band.SetNoDataValue(0) # 将背景 0 值设置为 NoData（可选）
+
+    # 5. 清理资源
+    ds_new = None
+    ds_ref = None
+    if 'data' in locals():
+        del data  # 释放内存
+    
+    print(f"✅ GeoTIFF 转换成功，保存至: {out_tiff_path}")
+    
+    # 可选：立即添加颜色表（使用您的函数）
+    add_transparent_color(out_tiff_path)
+
+    return out_tiff_path
+
+def add_transparent_color(raster_path):
+    """把 0 值设成完全透明，1 值设成任意可见色 (适用于 GDT_Byte 或 GDT_UInt16)"""
+    try:
+        ds = gdal.Open(raster_path, gdal.GA_Update)
+        if ds is None:
+            print(f"警告: 无法以更新模式打开 {raster_path} 进行颜色表设置。")
+            return
+
+        band = ds.GetRasterBand(1)
+        # 强制转换为 Byte 类型，如果不是的话，确保颜色表能生效
+        if band.DataType != gdal.GDT_Byte:
+            # 如果不是 Byte，先转换为 Byte（但这里假设 create_geotiff_from_png 已经处理了）
+            print(f"警告: 栅格类型 {gdal.GetDataTypeName(band.DataType)} 可能不支持颜色表。")
+
+        ct = gdal.ColorTable()
+        ct.SetColorEntry(0, (0, 0, 0, 0))    # A=0  完全透明
+        ct.SetColorEntry(1, (255, 0, 0, 180)) # A=180 半透明红色
+        band.SetColorTable(ct)
+        band.SetRasterColorInterpretation(gdal.GCI_PaletteIndex)
+        ds = None
+        print(f"✅ {os.path.basename(raster_path)} 已添加透明颜色表。")
+    except Exception as e:
+        print(f"❌ 颜色表设置失败: {e}")
+
+def raster2polygon(in_raster_path, out_shp_path,
+                   field_name='DN',
+                   connected_8=True,
+                   sieve_size=0,
+                   target_value=255,
+                   contour_approx_epsilon=None,
+                   smooth_gaussian=True,
+                   gaussian_kernel_size=5,
+                   use_spline=False,
+                   spline_points=100,
+                   min_area=0.0,
+                   check_topology=True):
+    """
+    使用OpenCV轮廓检测和轮廓近似进行矢量化，获得平滑边界（减少锐角）
+    
+    Args:
+        in_raster_path: 输入栅格路径
+        out_shp_path: 输出矢量路径
+        field_name: 字段名
+        connected_8: 是否使用8连通（保留参数以兼容旧代码，但新方法使用OpenCV轮廓检测）
+        sieve_size: 碎斑过滤阈值（像素数）
+        target_value: 目标像素值
+        contour_approx_epsilon: 轮廓近似精度（像素），None表示自动计算
+                                值越小越精确但顶点越多，值越大越平滑但可能丢失细节
+                                建议值：0.5-2.0像素（减小以获得更平滑的边界）
+        smooth_gaussian: 是否在矢量化前对栅格进行高斯模糊平滑（默认True）
+        gaussian_kernel_size: 高斯模糊核大小（奇数，建议3-7，默认5）
+        use_spline: 是否使用样条插值进一步平滑边界（默认False，会增加计算时间）
+        spline_points: 样条插值的点数（仅在use_spline=True时有效）
+        min_area: 最小面积阈值（平方米），小于此值的多边形将被过滤（默认0.0不过滤）
+        check_topology: 是否进行拓扑检查和修复（默认True）
+    """
+    print(f"\n--- 🗺️ 步骤 III: 栅格转矢量（轮廓平滑方法）---")
+    
+    ds = gdal.Open(in_raster_path, gdal.GA_ReadOnly)
+    if ds is None:
+        raise RuntimeError(f'无法打开输入栅格：{in_raster_path}')
+
+    band = ds.GetRasterBand(1)
+    data = band.ReadAsArray()
+    
+    # 获取地理变换参数
+    geo_transform = ds.GetGeoTransform()
+    projection = ds.GetProjection()
+    
+    # 统计
+    target_pixels = np.sum(data == target_value)
+    total_pixels = data.size
+    print(f"目标像素(值={target_value}): {target_pixels:,} / {total_pixels:,}")
+    
+    if target_pixels == 0:
+        print("⚠️ 没有找到目标像素，跳过矢量化")
+        ds = None
+        return None
+    
+    # 创建掩膜
+    mask_data = np.where(data == target_value, 255, 0).astype(np.uint8)
+    
+    # 碎斑过滤（使用OpenCV）
+    if sieve_size > 0:
+        print(f"⏳ 碎斑过滤 (阈值: {sieve_size} 像素)...")
+        # 使用形态学开运算去除小碎斑
+        kernel_size = max(3, sieve_size // 2)
+        if kernel_size % 2 == 0:
+            kernel_size += 1
+        kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (kernel_size, kernel_size))
+        mask_data = cv2.morphologyEx(mask_data, cv2.MORPH_OPEN, kernel)
+    
+    # 高斯模糊平滑（减少锐角，使边界更平滑）
+    if smooth_gaussian:
+        print(f"⏳ 高斯模糊平滑 (核大小: {gaussian_kernel_size})...")
+        # 确保核大小为奇数
+        if gaussian_kernel_size % 2 == 0:
+            gaussian_kernel_size += 1
+        mask_data = cv2.GaussianBlur(mask_data, (gaussian_kernel_size, gaussian_kernel_size), 0)
+        # 重新二值化（高斯模糊后值会变化）
+        _, mask_data = cv2.threshold(mask_data, 127, 255, cv2.THRESH_BINARY)
+    
+    # 创建输出目录
+    out_dir = os.path.dirname(out_shp_path)
+    if out_dir and not os.path.exists(out_dir):
+        os.makedirs(out_dir, exist_ok=True)
+    
+    # 删除已存在的文件
+    if os.path.exists(out_shp_path):
+        for ext in ['.shp', '.shx', '.dbf', '.prj', '.qpj']:
+            try:
+                os.remove(out_shp_path.replace('.shp', ext))
+            except:
+                pass
+    
+    # 创建矢量文件
+    driver = ogr.GetDriverByName('ESRI Shapefile')
+    out_ds = driver.CreateDataSource(out_shp_path)
+    
+    srs = osr.SpatialReference()
+    srs.ImportFromWkt(projection)
+    
+    out_layer = out_ds.CreateLayer(
+        name=os.path.splitext(os.path.basename(out_shp_path))[0],
+        geom_type=ogr.wkbPolygon,
+        srs=srs
+    )
+    
+    fd = ogr.FieldDefn(field_name, ogr.OFTInteger)
+    out_layer.CreateField(fd)
+    
+    # 使用OpenCV轮廓检测
+    print("⏳ 正在检测轮廓...")
+    # 使用CHAIN_APPROX_NONE获取所有点，然后进行平滑处理（减少锐角）
+    contours, hierarchy = cv2.findContours(
+        mask_data, 
+        cv2.RETR_CCOMP,  # 检测所有轮廓，包括孔洞
+        cv2.CHAIN_APPROX_NONE  # 获取所有点，便于后续平滑处理
+    )
+    
+    print(f"   检测到 {len(contours)} 个轮廓")
+    
+    # 计算轮廓近似精度（如果未指定）
+    if contour_approx_epsilon is None:
+        # 根据图像分辨率自动计算
+        # 为了获得更平滑的边界，使用更小的epsilon值（增加顶点数）
+        pixel_size_x = abs(geo_transform[1])
+        pixel_size_y = abs(geo_transform[5])
+        avg_pixel_size = (pixel_size_x + pixel_size_y) / 2
+        
+        # 转换为像素单位（假设地理坐标单位是度）
+        # 对于约16米分辨率，约0.00015度，对应约1-2像素
+        # 使用更小的值以获得更平滑的边界（减少锐角）
+        if avg_pixel_size < 0.001:  # 地理坐标系（度）
+            contour_approx_epsilon = 1.0  # 从2.0减小到1.0，增加顶点数
+        else:  # 投影坐标系（米）
+            contour_approx_epsilon = max(0.5, avg_pixel_size / 16.0)  # 从8.0改为16.0，更精细
+    
+    print(f"   轮廓近似精度: {contour_approx_epsilon:.2f} 像素（值越小边界越平滑）")
+    
+    # 辅助函数：样条插值平滑轮廓
+    def smooth_contour_with_spline(contour, num_points=100):
+        """
+        使用样条插值平滑轮廓，减少锐角
+        
+        Args:
+            contour: OpenCV轮廓点
+            num_points: 插值后的点数
+        
+        Returns:
+            平滑后的轮廓点
+        """
+        if len(contour) < 4:
+            return contour
+        
+        # 提取x和y坐标
+        points = contour.reshape(-1, 2)
+        x = points[:, 0].astype(np.float32)
+        y = points[:, 1].astype(np.float32)
+        
+        # 闭合轮廓（添加第一个点到末尾）
+        x = np.append(x, x[0])
+        y = np.append(y, y[0])
+        
+        # 计算累积距离作为参数
+        distances = np.zeros(len(x))
+        for i in range(1, len(x)):
+            dx = x[i] - x[i-1]
+            dy = y[i] - y[i-1]
+            distances[i] = distances[i-1] + np.sqrt(dx*dx + dy*dy)
+        
+        # 归一化参数到[0, 1]
+        if distances[-1] > 0:
+            t = distances / distances[-1]
+        else:
+            return contour
+        
+        # 生成新的参数点
+        t_new = np.linspace(0, 1, num_points)
+        
+        # 样条插值
+        try:
+            from scipy.interpolate import interp1d
+            # 使用三次样条插值
+            fx = interp1d(t, x, kind='cubic', bounds_error=False, fill_value='extrapolate')
+            fy = interp1d(t, y, kind='cubic', bounds_error=False, fill_value='extrapolate')
+            
+            x_new = fx(t_new)
+            y_new = fy(t_new)
+            
+            # 转换为OpenCV轮廓格式
+            smoothed = np.array([[int(x_new[i]), int(y_new[i])] for i in range(len(x_new))], dtype=np.int32)
+            return smoothed.reshape(-1, 1, 2)
+        except ImportError:
+            # 如果没有scipy，使用简单的线性插值
+            print("   警告: 未安装scipy，使用线性插值代替样条插值")
+            fx = np.interp(t_new, t, x)
+            fy = np.interp(t_new, t, y)
+            smoothed = np.array([[int(fx[i]), int(fy[i])] for i in range(len(fx))], dtype=np.int32)
+            return smoothed.reshape(-1, 1, 2)
+    
+    # 辅助函数：将轮廓转换为OGR环
+    def contour_to_ring(contour, geo_transform):
+        """将OpenCV轮廓转换为OGR线性环"""
+        ring = ogr.Geometry(ogr.wkbLinearRing)
+        for point in contour:
+            x_pixel = point[0][0]
+            y_pixel = point[0][1]
+            
+            # 转换为地理坐标
+            x_geo = geo_transform[0] + x_pixel * geo_transform[1] + y_pixel * geo_transform[2]
+            y_geo = geo_transform[3] + x_pixel * geo_transform[4] + y_pixel * geo_transform[5]
+            
+            ring.AddPoint(x_geo, y_geo)
+        
+        # 闭合环
+        if ring.GetPointCount() > 0:
+            first_point = ring.GetPoint(0)
+            ring.AddPoint(first_point[0], first_point[1])
+        
+        return ring
+    
+    # 处理轮廓：先处理外环，再处理对��的内环（孔洞）
+    feature_count = 0
+    processed_indices = set()
+    
+    for i, contour in enumerate(contours):
+        if i in processed_indices or len(contour) < 3:
+            continue
+        
+        # 检查是否是外环（hierarchy[i][3] == -1）
+        parent_idx = hierarchy[0][i][3]
+        if parent_idx != -1:  # 这是内环（孔洞），跳过，稍后处理
+            continue
+        
+        # 轮廓处理：先进行样条插值平滑（如果启用），再进行Douglas-Peucker近似
+        processed_contour = contour
+        
+        # 样条插值平滑（减少锐角）
+        if use_spline and len(contour) >= 4:
+            processed_contour = smooth_contour_with_spline(contour, spline_points)
+        
+        # 轮廓近似（Douglas-Peucker算法）- 使用较小的epsilon以获得更平滑的边界
+        approx = cv2.approxPolyDP(processed_contour, contour_approx_epsilon, closed=True)
+        
+        if len(approx) < 3:
+            continue
+        
+        # 创建外环
+        exterior_ring = contour_to_ring(approx, geo_transform)
+        
+        # 创建多边形
+        poly = ogr.Geometry(ogr.wkbPolygon)
+        poly.AddGeometry(exterior_ring)
+        
+        # 查找并添加内环（孔洞）
+        # 遍历所有轮廓，找到父轮廓是当前轮廓的内环
+        child_idx = hierarchy[0][i][2]  # 第一个子轮廓索引
+        while child_idx != -1:
+            if child_idx < len(contours):
+                child_contour = contours[child_idx]
+                if len(child_contour) >= 3:
+                    # 对内环也进行平滑处理
+                    processed_child = child_contour
+                    if use_spline and len(child_contour) >= 4:
+                        processed_child = smooth_contour_with_spline(child_contour, spline_points)
+                    
+                    # 近似内环
+                    child_approx = cv2.approxPolyDP(processed_child, contour_approx_epsilon, closed=True)
+                    if len(child_approx) >= 3:
+                        interior_ring = contour_to_ring(child_approx, geo_transform)
+                        poly.AddGeometry(interior_ring)
+                    processed_indices.add(child_idx)
+                # 移动到下一个兄弟轮廓
+                child_idx = hierarchy[0][child_idx][0]
+            else:
+                break
+        
+        # 拓扑检查和修复
+        if check_topology:
+            # 检查几何有效性
+            if not poly.IsValid():
+                # 尝试修复无效几何（Buffer(0)可以修复一些拓扑错误）
+                try:
+                    poly_fixed = poly.Buffer(0)
+                    if poly_fixed.IsValid():
+                        poly = poly_fixed
+                        print(f"   ✓ 修复无效几何（轮廓 {i}）")
+                    else:
+                        print(f"   ✗ 无法修复无效几何（轮廓 {i}），跳过")
+                        processed_indices.add(i)
+                        continue
+                except Exception as e:
+                    print(f"   ✗ 修复几何失败（轮廓 {i}）: {e}，跳过")
+                    processed_indices.add(i)
+                    continue
+            
+            # 检查面积（过滤太小的多边形）
+            area = poly.GetArea()
+            if area <= 0:
+                print(f"   ✗ 跳过零面积几何（轮廓 {i}）")
+                processed_indices.add(i)
+                continue
+            
+            # 如果设置了最小面积阈值，过滤太小的多边形
+            if min_area > 0 and area < min_area:
+                print(f"   ✗ 跳过面积过小的几何（轮廓 {i}，面积: {area:.2f} 平方米）")
+                processed_indices.add(i)
+                continue
+        
+        # 创建要素
+        feature = ogr.Feature(out_layer.GetLayerDefn())
+        feature.SetGeometry(poly)
+        feature.SetField(field_name, target_value)
+        out_layer.CreateFeature(feature)
+        feature = None
+        feature_count += 1
+        processed_indices.add(i)
+    
+    # 清理
+    out_ds = None
+    ds = None
+    
+    print(f"✅ 矢量化完成，共 {feature_count} 个面要素")
+    print(f"   输出: {out_shp_path}")
+    
+    return out_shp_path
+
+# =======================================================
+# II. 拼接主函数 (基于上一次修正)
+# =======================================================
+
+def stitch_binary_predictions_patch_final(output_dir, json_filename, output_filename="stitched_prediction.png"):
+    """
+    拼接切片为 PNG 文件，确保与 'patch00000_prediction.png' 命名规则匹配。
+    （代码与上一个回答的最终版本一致，这里仅作为整合）
+    """
+    json_filepath = os.path.join(output_dir, json_filename)
+    print(f"--- 🚀 步骤 I: 拼接二值图 ---")
+    
+    try:
+        with open(json_filepath, 'r') as f:
+            data = json.load(f)
+    except Exception as e:
+        raise RuntimeError(f"❌ 错误: 读取或解析JSON文件出错: {e}")
+
+    tile_results = data.get('tile_results', [])
+    if not tile_results:
+        print("⚠️ 警告: JSON文件中没有找到 'tile_results' 数据。")
+        return None
+
+    def get_tile_filename(tile_id):
+        return f"patch{tile_id:05d}_prediction.png"
+
+    first_tile_id = tile_results[0]['tile_id']
+    first_tile_path = os.path.join(output_dir, get_tile_filename(first_tile_id))
+    
+    try:
+        with Image.open(first_tile_path) as img:
+            tile_width, tile_height = img.size
+    except Exception as e:
+        raise RuntimeError(f"❌ 致命错误: 无法读取第一个切片 {first_tile_path}。请确认文件名。")
+
+    max_x = max(item['x'] for item in tile_results)
+    max_y = max(item['y'] for item in tile_results)
+    stitched_width = max_x + tile_width
+    stitched_height = max_y + tile_height
+    
+    print(f"✅ 切片尺寸 (W x H): {tile_width} x {tile_height} | 预计大图尺寸: {stitched_width} x {stitched_height}")
+    
+    # 检查内存需求（估算）
+    estimated_memory_mb = (stitched_width * stitched_height * 1) / (1024 * 1024)  # 1字节每像素
+    print(f"📊 预计内存需求: {estimated_memory_mb:.2f} MB")
+    
+    if estimated_memory_mb > 10000:  # 超过10GB
+        print("⚠️  警告: 图像很大，可能需要大量内存")
+    
+    try:
+        stitched_image = Image.new('L', (stitched_width, stitched_height))
+    except MemoryError:
+        error_msg = (f"❌ 内存不足: 无法创建 {stitched_width}x{stitched_height} 的图像。\n"
+                    f"   预计需要约 {estimated_memory_mb:.2f} MB 内存。\n"
+                    f"   建议: 1) 关闭其他程序 2) 使用更大的内存 3) 分块处理")
+        raise RuntimeError(error_msg)
+    except Exception as e:
+        raise RuntimeError(f"❌ 创建拼接图像失败: {e}")
+    
+    print(f"⏳ 开始拼接 {len(tile_results)} 个切片...")
+    processed_count = 0
+    
+    for tile_info in tile_results:
+        tile_id = tile_info['tile_id']
+        x_offset = tile_info['x']
+        y_offset = tile_info['y']
+        tile_path = os.path.join(output_dir, get_tile_filename(tile_id))
+
+        try:
+            with Image.open(tile_path) as tile_img:
+                if tile_img.mode != 'L':
+                    tile_img = tile_img.convert('L')
+                stitched_image.paste(tile_img, (x_offset, y_offset))
+                processed_count += 1
+                if processed_count % 1000 == 0:
+                    print(f"   已处理 {processed_count}/{len(tile_results)} 个切片...")
+        except FileNotFoundError:
+            print(f"⚠️ 警告: 找不到切片图像文件，跳过: {tile_path}")
+        except Exception as e:
+            print(f"❌ 处理切片 {tile_id} 时出错，跳过: {e}")
+    
+    print(f"✅ 完成拼接，共处理 {processed_count} 个切片")
+
+    final_output_path = os.path.join(output_dir, output_filename)
+    
+    # 直接保存，避免转换为numpy数组（节省内存）
+    print(f"⏳ 正在保存拼接结果到 {output_filename}...")
+    try:
+        # 直接保存PIL Image，不需要转换为numpy
+        stitched_image.save(final_output_path, format='PNG', compress_level=1)
+        print(f"✅ PNG 拼接完成，保存至: {final_output_path}")
+    except MemoryError:
+        # 如果直接保存也失败，尝试使用更低的压缩级别或分块保存
+        print("⚠️  直接保存失败，尝试使用分块保存...")
+        try:
+            # 尝试使用更低的压缩级别
+            stitched_image.save(final_output_path, format='PNG', compress_level=9, optimize=True)
+            print(f"✅ PNG 拼接完成（使用压缩），保存至: {final_output_path}")
+        except Exception as e:
+            raise RuntimeError(f"❌ 保存失败: {e}\n"
+                             f"   图像太大 ({stitched_width}x{stitched_height})，内存不足。\n"
+                             f"   建议: 1) 关闭其他程序 2) 增加虚拟内存 3) 使用分块处理")
+    except Exception as e:
+        raise RuntimeError(f"❌ 保存PNG文件失败: {e}")
+    
+    # 清理内存
+    stitched_image = None
+    
+    return final_output_path
+
+# =======================================================
+# III. 主流程调用
+# =======================================================
+if __name__ == '__main__':
+    # ------------------- 配置文件路径 -------------------
+    # 存放切片和 JSON 的目录
+    PREDICTION_OUTPUT_DIR = "outputs/test_large" 
+    # JSON 文件名
+    JSON_FILENAME = "GF6_PMS_E121.1_N33.6_20250604_L1A1420584616-MUX_fuse_tile_results.json" 
+    
+    # 原始影像路径 (用于获取地理参考信息，您在 JSON 中提到过文件名)
+    ORIGINAL_TIFF_PATH = r"datu/test/GF6_PMS_E121.1_N33.6_20250604_L1A1420584616-MUX_fuse.tif" 
+    
+    # 输出文件名定义
+    STITCHED_PNG_FILENAME = "stitched_prediction.png"
+    OUTPUT_GEOTIFF_FILENAME = "final_binary_prediction.tif"
+    OUTPUT_SHP_FILENAME = "output_vector_test/sargassum_polygon.shp"
+
+    # 碎斑过滤设置
+    SIEVE_PIXELS = 0
+    # ----------------------------------------------------
+    
+    try:
+        # 1. 拼接切片 (生成 PNG)
+        stitched_png_path = stitch_binary_predictions_patch_final(
+            PREDICTION_OUTPUT_DIR, 
+            JSON_FILENAME, 
+            STITCHED_PNG_FILENAME
+        )
+        if not stitched_png_path:
+            sys.exit(1)
+
+        # 2. 转换为 GeoTIFF (添加地理参考)
+        stitched_tiff_path = create_geotiff_from_png(
+            stitched_png_path,
+            ORIGINAL_TIFF_PATH,
+            os.path.join(PREDICTION_OUTPUT_DIR, OUTPUT_GEOTIFF_FILENAME)
+        )
+        
+        # 3. 栅格矢量化
+        raster2polygon(
+            in_raster_path=stitched_tiff_path,
+            out_shp_path=os.path.join(PREDICTION_OUTPUT_DIR, OUTPUT_SHP_FILENAME),
+            sieve_size=SIEVE_PIXELS,
+            connected_8=True
+        )
+
+    except Exception as e:
+        import traceback
+        print(f"\n❌ 致命错误发生: {e}")
+        print("\n详细错误信息:")
+        traceback.print_exc()
+        print("\n可能的原因:")
+        print("1. 文件路径不正确或文件不存在")
+        print("2. 内存不足（图像太大）")
+        print("3. 文件权限问题")
+        print("4. GDAL库配置问题")
+        sys.exit(1)

marine_features.py

@@ -0,0 +1,59 @@
+"""Seed labels and metadata helpers for registry-driven marine feature tasks."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True)
+class SeedLabel:
+    id: int
+    name: str
+    description: str
+    role: str
+
+
+SEED_CONTEXT_LABELS = [
+    SeedLabel(0, "other", "valid background or unlabeled non-target area", "context"),
+    SeedLabel(1, "invalid", "black border, no-data, saturated, missing, or unusable pixels", "validity"),
+    SeedLabel(2, "water", "valid water background", "context"),
+    SeedLabel(3, "land", "land or non-water hard negative", "context"),
+    SeedLabel(4, "cloud_shadow", "cloud, haze, cloud shadow, or atmospheric interference", "context"),
+]
+
+SEED_ELEMENT_CARDS = {
+    "green_tide": "Enteromorpha / green tide / seaweed",
+    "red_tide": "red tide / harmful algal bloom",
+    "golden_tide": "Sargassum / golden tide",
+    "aquaculture": "aquaculture areas, rafts, cages, ponds, or facilities",
+    "ship": "ship or vessel target",
+    "oil_spill": "oil film or suspected oil-spill area",
+    "sea_ice": "sea ice or ice-water boundary",
+}
+
+TASK_TYPES = {
+    "semantic_segmentation",
+    "instance_segmentation",
+    "detection",
+    "polygon_extraction",
+    "change_detection",
+    "anomaly_detection",
+}
+
+DEFAULT_BANDS_4CH = ["blue", "green", "red", "nir"]
+FUSION_STATES = {"none", "fused_product", "runtime_fusion", "unknown"}
+
+LABEL_NAME_TO_ID = {item.name: item.id for item in SEED_CONTEXT_LABELS}
+LABEL_ID_TO_NAME = {item.id: item.name for item in SEED_CONTEXT_LABELS}
+
+
+def is_seed_element(name: str) -> bool:
+    return name in SEED_ELEMENT_CARDS
+
+
+def is_fusion_state(value: str) -> bool:
+    return value in FUSION_STATES
+
+
+def context_label_id(name: str) -> int:
+    return LABEL_NAME_TO_ID[name]

predict_large_image_tiles_256.py

@@ -0,0 +1,602 @@
+"""
+大图浒苔分割预测脚本 - 256x256瓦片版本
+读取大图遥感图像，切割为256x256小patch，对每个patch进行预测
+预测结果格式与predict_seaweed_256x256_simple.py保持一致
+"""
+import os
+import torch
+import numpy as np
+from pathlib import Path
+import json
+import time
+from datetime import datetime
+import cv2
+from tqdm import tqdm
+import rasterio
+from rasterio.windows import Window
+import warnings
+warnings.filterwarnings('ignore')
+
+# 导入自定义模块
+from dinov3_deeplabv3plus import DinoV3DeepLabV3Plus
+
+def load_model(model_path, config, device):
+    """加载训练好的模型 - 与predict_seaweed_256x256_simple.py保持一致"""
+    print(f"正在加载模型: {model_path}")
+    
+    # 创建模型
+    model = DinoV3DeepLabV3Plus(
+        num_classes=config['num_classes'],
+        backbone_name=config['backbone_name'],
+        pretrained=False,  # 不加载预训练权重，使用我们自己的权重
+        weights=config['backbone_weights'],
+        use_4channel=config['use_4channel']
+    ).to(device)
+    
+    # 加载权重
+    checkpoint = torch.load(model_path, map_location=device)
+    
+    # 处理不同的checkpoint格式
+    if 'model_state_dict' in checkpoint:
+        model.load_state_dict(checkpoint['model_state_dict'])
+        print(f"加载模型权重 (epoch {checkpoint.get('epoch', 'unknown')})")
+    elif 'state_dict' in checkpoint:
+        model.load_state_dict(checkpoint['state_dict'])
+        print("加载模型权重")
+    else:
+        # 直接加载state dict
+        model.load_state_dict(checkpoint)
+        print("加载模型权重")
+    
+    model.eval()
+    print("模型加载完成，进入评估模式")
+    return model
+
+def process_image_for_model(image_path, target_size=256, use_4channel=True):
+    """处理图像用于模型预测 - 完全复用predict_seaweed_256x256_simple.py的逻辑"""
+    # 使用OpenCV读取16位TIF图像
+    img_16bit = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
+    
+    if img_16bit is None:
+        raise ValueError(f"无法读取图像文件: {image_path}")
+    
+    # 使用rasterio方式读取多波段数据
+    try:
+        import rasterio
+        with rasterio.open(image_path) as src:
+            # 读取所有波段
+            image = src.read()
+            # 转换为HWC格式
+            image = np.transpose(image, (1, 2, 0))
+    except Exception as e:
+        print(f"rasterio读取失败，使用OpenCV: {e}")
+        # 回退到OpenCV
+        image = img_16bit
+    
+    # 处理通道数
+    if use_4channel and image.shape[2] >= 4:
+        # 使用4通道
+        processed_image = image[:, :, :4]
+    else:
+        # 使用3通道假彩色（432波段）
+        if image.shape[2] >= 4:
+            # 选择第4,3,2波段（索引3,2,1）
+            processed_image = image[:, :, [3, 2, 1]]
+        else:
+            # 如果通道数不足，使用可用通道
+            processed_image = image[:, :, :3]
+            if image.shape[2] < 3:
+                # 如果还是不足3通道，重复填充
+                while processed_image.shape[2] < 3:
+                    processed_image = np.concatenate([processed_image, processed_image[:, :, -1:]], axis=2)
+    
+    # 调整尺寸
+    if processed_image.shape[0] != target_size or processed_image.shape[1] != target_size:
+        processed_image = cv2.resize(processed_image, (target_size, target_size), interpolation=cv2.INTER_LINEAR)
+    
+    # 转换为float32并标准化到0-1范围
+    processed_image = processed_image.astype(np.float32)
+    if processed_image.max() > 1.0:
+        processed_image = processed_image / 65535.0  # 16位最大值
+    
+    # 标准化
+    if use_4channel:
+        # 4通道标准化
+        mean = np.array([0.430, 0.411, 0.296, 0.350])
+        std = np.array([0.213, 0.156, 0.143, 0.180])
+    else:
+        # 3通道标准化
+        mean = np.array([0.430, 0.411, 0.296])
+        std = np.array([0.213, 0.156, 0.143])
+    
+    # 应用标准化
+    for i in range(processed_image.shape[2]):
+        processed_image[:, :, i] = (processed_image[:, :, i] - mean[i]) / std[i]
+    
+    # 转换为tensor并添加batch维度
+    img_tensor = torch.from_numpy(processed_image).permute(2, 0, 1).float()
+    
+    return img_tensor.unsqueeze(0)  # 添加batch维度
+
+def predict_image(model, image_tensor, device, config, threshold=0.5):
+    """
+    预测单张图像 - 与predict_seaweed_256x256_simple.py保持一致
+    
+    Args:
+        model: 训练好的模型
+        image_tensor: 图像tensor
+        device: 设备
+        config: 配置字典
+        threshold: 浒苔预测阈值，只有当浒苔概率超过此阈值时才预测为浒苔（默认0.5）
+    """
+    # 预测
+    with torch.no_grad():
+        start_time = time.time()
+        output = model(image_tensor)
+        inference_time = time.time() - start_time
+        
+        # 处理输出格式
+        if isinstance(output, dict):
+            output = output['out']
+        
+        # 获取预测结果
+        pred = torch.softmax(output, dim=1)
+        
+        # 计算浒苔概率
+        seaweed_prob = pred[0, 1].squeeze(0).cpu().numpy()  # 类别1是浒苔
+        
+        # 使用阈值进行预测：只有当浒苔概率超过阈值时才预测为浒苔
+        # 这样可以避免在无浒苔图片中，由于两个类别概率接近而误判
+        pred_class = (seaweed_prob > threshold).astype(np.uint8)
+        
+        return {
+            'prediction': pred_class,
+            'seaweed_probability': seaweed_prob,
+            'inference_time': inference_time
+        }
+
+def save_prediction_results(prediction, seaweed_prob, output_path, base_name):
+    """保存预测结果到文件 - 与predict_seaweed_256x256_simple.py保持一致"""
+    # 保存预测掩码 (0-1)
+    pred_mask = prediction.astype(np.uint8)
+    np.save(str(output_path / f"{base_name}_prediction.npy"), pred_mask)
+    
+    # 保存概率图 (0-1)
+    np.save(str(output_path / f"{base_name}_probability.npy"), seaweed_prob)
+    
+    # 保存为PNG图像
+    pred_png = (prediction * 255).astype(np.uint8)
+    prob_png = (seaweed_prob * 255).astype(np.uint8)
+    
+    cv2.imwrite(str(output_path / f"{base_name}_prediction.png"), pred_png)
+    cv2.imwrite(str(output_path / f"{base_name}_probability.png"), prob_png)
+
+def calculate_statistics(prediction, seaweed_prob):
+    """计算统计信息 - 与predict_seaweed_256x256_simple.py保持一致"""
+    total_pixels = prediction.size
+    seaweed_pixels = np.sum(prediction == 1)
+    seaweed_ratio = seaweed_pixels / total_pixels
+    
+    # 计算平均概率
+    avg_probability = np.mean(seaweed_prob)
+    max_probability = np.max(seaweed_prob)
+    
+    return {
+        'total_pixels': int(total_pixels),
+        'seaweed_pixels': int(seaweed_pixels),
+        'seaweed_ratio': float(seaweed_ratio),
+        'avg_probability': float(avg_probability),
+        'max_probability': float(max_probability)
+    }
+
+def get_tile_info_from_large_image(image_path, tile_size=256, overlap=0):
+    """
+    获取大图的瓦片信息，但不实际读取数据（内存友好）
+    
+    Args:
+        image_path: 大图路径
+        tile_size: 瓦片大小
+        overlap: 重叠大小
+    
+    Returns:
+        list: 瓦片信息列表，每个元素包含(x, y, window, tile_id)
+    """
+    tile_info_list = []
+    
+    with rasterio.open(image_path) as src:
+        height, width = src.height, src.width
+        bands = src.count
+        
+        print(f"大图信息: {width}x{height}, {bands}波段")
+        
+        # 计算步长
+        stride = tile_size - overlap
+        
+        # 计算瓦片数量
+        tiles_y = (height - tile_size) // stride + 1
+        tiles_x = (width - tile_size) // stride + 1
+        
+        # 处理边界情况
+        if (height - tile_size) % stride != 0:
+            tiles_y += 1
+        if (width - tile_size) % stride != 0:
+            tiles_x += 1
+        
+        total_tiles = tiles_x * tiles_y
+        print(f"将提取 {total_tiles} 个瓦片 ({tiles_x} x {tiles_y})")
+        
+        # 生成瓦片信息
+        tile_count = 0
+        for y in range(0, height - tile_size + 1, stride):
+            for x in range(0, width - tile_size + 1, stride):
+                
+                # 处理边界情况
+                actual_x = min(x, width - tile_size)
+                actual_y = min(y, height - tile_size)
+                
+                # 创建窗口信息
+                window = Window(actual_x, actual_y, tile_size, tile_size)
+                
+                tile_info_list.append({
+                    'x': actual_x,
+                    'y': actual_y,
+                    'window': window,
+                    'tile_id': tile_count
+                })
+                
+                tile_count += 1
+    
+    return tile_info_list
+
+def read_tile_data(image_path, window, bands):
+    """读取单个瓦片的数据"""
+    with rasterio.open(image_path) as src:
+        # 读取多波段数据
+        if bands >= 4:
+            # 使用4通道
+            tile_data = src.read([1, 2, 3, 4], window=window)
+        else:
+            # 使用可用通道
+            tile_data = src.read(list(range(1, min(bands, 4) + 1)), window=window)
+        
+        # 转换为HWC格式
+        tile_data = np.transpose(tile_data, (1, 2, 0))
+        
+        return tile_data
+
+def save_tile_temporarily(tile_data, temp_dir, tile_id):
+    """临时保存瓦片为文件，用于process_image_for_model函数"""
+    temp_path = os.path.join(temp_dir, f"temp_tile_{tile_id}.tif")
+    
+    # 确保临时目录存在
+    os.makedirs(temp_dir, exist_ok=True)
+    
+    # 保存为临时TIFF文件
+    with rasterio.open(
+        temp_path, 'w',
+        driver='GTiff',
+        height=tile_data.shape[0],
+        width=tile_data.shape[1],
+        count=tile_data.shape[2],
+        dtype=tile_data.dtype,
+        compress='lzw'
+    ) as dst:
+        for i in range(tile_data.shape[2]):
+            dst.write(tile_data[:, :, i], i + 1)
+    
+    return temp_path
+
+def predict_tiles(tiles, model, device, config, output_dir, threshold=0.5):
+    """
+    对所有瓦片进行预测
+    
+    Args:
+        tiles: 瓦片列表
+        model: 模型
+        device: 设备
+        config: 配置
+        output_dir: 输出目录
+        threshold: 预测阈值
+    """
+    results = []
+    total_inference_time = 0
+    
+    # 创建临时目录
+    temp_dir = os.path.join(output_dir, "temp_tiles")
+    
+    print(f"\n开始预测 {len(tiles)} 个瓦片...")
+    
+    # 使用tqdm显示进度条
+    for tile_info in tqdm(tiles, desc="预测瓦片"):
+        try:
+            tile_id = tile_info['tile_id']
+            tile_data = tile_info['data']
+            
+            # 临时保存瓦片为文件
+            temp_tile_path = save_tile_temporarily(tile_data, temp_dir, tile_id)
+            
+            # 处理图像用于模型
+            image_tensor = process_image_for_model(
+                temp_tile_path,
+                target_size=config['image_size'],
+                use_4channel=config['use_4channel']
+            ).to(device)
+            
+            # 预测（使用阈值）
+            result = predict_image(model, image_tensor, device, config, threshold=threshold)
+            
+            # 计算统计信息
+            stats = calculate_statistics(result['prediction'], result['seaweed_probability'])
+            
+            # 保存预测结果
+            base_name = f"patch{tile_id:05d}"
+            save_prediction_results(
+                result['prediction'], 
+                result['seaweed_probability'], 
+                Path(output_dir), 
+                base_name
+            )
+            
+            # 保存结果信息
+            result_info = {
+                'tile_id': tile_id,
+                'x': tile_info['x'],
+                'y': tile_info['y'],
+                'inference_time': float(result['inference_time']),
+                'statistics': stats
+            }
+            results.append(result_info)
+            
+            total_inference_time += result['inference_time']
+            
+            # 清理临时文件
+            if os.path.exists(temp_tile_path):
+                os.remove(temp_tile_path)
+                
+        except Exception as e:
+            print(f"  - 瓦片 {tile_id} 预测失败: {str(e)}")
+            continue
+    
+    # 清理临时目录
+    if os.path.exists(temp_dir):
+        try:
+            os.rmdir(temp_dir)
+        except:
+            pass
+    
+    return results, total_inference_time
+
+def predict_tiles_memory_friendly(tile_info_list, image_path, model, device, config, output_dir, threshold=0.5):
+    """
+    内存友好的瓦片预测 - 逐块处理
+    
+    Args:
+        tile_info_list: 瓦片信息列表
+        image_path: 图像路径
+        model: 模型
+        device: 设备
+        config: 配置
+        output_dir: 输出目录
+        threshold: 预测阈值
+    """
+    results = []
+    total_inference_time = 0
+    
+    # 创建临时目录
+    temp_dir = os.path.join(output_dir, "temp_tiles")
+    os.makedirs(temp_dir, exist_ok=True)
+    
+    # 获取图像信息
+    with rasterio.open(image_path) as src:
+        bands = src.count
+    
+    print(f"\n开始预测 {len(tile_info_list)} 个瓦片...")
+    
+    # 使用tqdm显示进度条
+    for tile_info in tqdm(tile_info_list, desc="预测瓦片"):
+        try:
+            tile_id = tile_info['tile_id']
+            
+            # 读取瓦片数据
+            tile_data = read_tile_data(image_path, tile_info['window'], bands)
+            
+            # 临时保存瓦片为文件
+            temp_tile_path = save_tile_temporarily(tile_data, temp_dir, tile_id)
+            
+            # 处理图像用于模型
+            image_tensor = process_image_for_model(
+                temp_tile_path,
+                target_size=config['image_size'],
+                use_4channel=config['use_4channel']
+            ).to(device)
+            
+            # 预测（使用阈值）
+            result = predict_image(model, image_tensor, device, config, threshold=threshold)
+            
+            # 计算统计信息
+            stats = calculate_statistics(result['prediction'], result['seaweed_probability'])
+            
+            # 保存预测结果
+            base_name = f"patch{tile_id:05d}"
+            save_prediction_results(
+                result['prediction'], 
+                result['seaweed_probability'], 
+                Path(output_dir), 
+                base_name
+            )
+            
+            # 保存结果信息
+            result_info = {
+                'tile_id': tile_id,
+                'x': tile_info['x'],
+                'y': tile_info['y'],
+                'inference_time': float(result['inference_time']),
+                'statistics': stats
+            }
+            results.append(result_info)
+            
+            total_inference_time += result['inference_time']
+            
+            # 清理临时文件
+            if os.path.exists(temp_tile_path):
+                os.remove(temp_tile_path)
+                
+        except Exception as e:
+            print(f"  - 瓦片 {tile_id} 预测失败: {str(e)}")
+            continue
+    
+    # 清理临时目录
+    try:
+        os.rmdir(temp_dir)
+    except:
+        pass
+    
+    return results, total_inference_time
+
+def main():
+    """主函数"""
+    print("=" * 60)
+    print("大图瓦片浒苔分割预测系统 - 256x256版本")
+    print("=" * 60)
+    
+    # 配置
+    model_path = "seaweed_segmentation_improved_epoch500/best_checkpoint.pth"
+    config_path = "seaweed_segmentation_improved_epoch500/config.json"
+    large_image_dir = "E:\GF6\山科影像\浒苔\GF6_PMS_E121.1_N33.6_20250604_L1A1420584616-MUX_fuse.tif"
+    output_dir = "outputs/test_large"
+    
+    # 瓦片配置
+    tile_size = 256
+    overlap = 0  # 无重叠
+    
+    # 预测阈值：只有当浒苔概率超过此阈值时才预测为浒苔
+    # 建议值：0.5-0.7，可以根据验证集调整
+    prediction_threshold = 0.5
+    
+    # 创建设备
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(f"使用设备: {device}")
+    
+    # 加载配置
+    with open(config_path, 'r', encoding='utf-8') as f:
+        config = json.load(f)
+    
+    print(f"模型配置:")
+    print(f"  - 输入通道: {4 if config['use_4channel'] else 3}")
+    print(f"  - 图像尺寸: {config['image_size']}x{config['image_size']}")
+    print(f"  - 类别数: {config['num_classes']}")
+    print(f"  - 瓦片大小: {tile_size}x{tile_size}")
+    print(f"  - 重叠: {overlap}")
+    print(f"  - 预测阈值: {prediction_threshold}")
+    
+    # 加载模型
+    model = load_model(model_path, config, device)
+    
+    # 创建输出目录
+    output_path = Path(output_dir)
+    output_path.mkdir(parents=True, exist_ok=True)
+    
+    # 获取大图文件
+    image_extensions = ['.tif', '.tiff']
+    large_images = []
+    for ext in image_extensions:
+        large_images.extend(Path(large_image_dir).glob(f"*{ext}"))
+    
+    print(f"\n找到 {len(large_images)} 个大图文件")
+    
+    # 处理每个大图
+    all_results = []
+    
+    for image_file in large_images:
+        print(f"\n处理大图: {image_file.name}")
+        
+        try:
+            # 获取瓦片信息（不读取实际数据）
+            tile_info_list = get_tile_info_from_large_image(
+                str(image_file), 
+                tile_size=tile_size, 
+                overlap=overlap
+            )
+            
+            if not tile_info_list:
+                print(f"  - 未能生成瓦片信息，跳过")
+                continue
+            
+            # 内存友好的瓦片预测
+            results, total_inference_time = predict_tiles_memory_friendly(
+                tile_info_list, str(image_file), model, device, config, output_path, threshold=prediction_threshold
+            )
+            
+            if results:
+                # 计算平均推理时间
+                avg_inference_time = total_inference_time / len(results)
+                
+                # 计算平均统计
+                all_ratios = [r['statistics']['seaweed_ratio'] for r in results]
+                avg_seaweed_ratio = np.mean(all_ratios)
+                
+                print(f"\n瓦片预测完成:")
+                print(f"  - 总瓦片数: {len(results)}")
+                print(f"  - 平均推理时间: {avg_inference_time:.3f}s")
+                print(f"  - 平均浒苔比例: {avg_seaweed_ratio:.2%}")
+                
+                # 保存大图处理结果
+                image_results = {
+                    'image_file': str(image_file),
+                    'total_tiles': len(results),
+                    'avg_inference_time': avg_inference_time,
+                    'avg_seaweed_ratio': float(avg_seaweed_ratio),
+                    'tile_results': results
+                }
+                all_results.append(image_results)
+                
+                # 保存结果文件
+                result_file = output_path / f"{image_file.stem}_tile_results.json"
+                with open(result_file, 'w', encoding='utf-8') as f:
+                    json.dump(image_results, f, indent=2, ensure_ascii=False, default=str)
+                
+                print(f"  - 结果保存: {result_file}")
+            
+        except Exception as e:
+            print(f"  - 处理失败: {str(e)}")
+            import traceback
+            traceback.print_exc()
+            continue
+    
+    # 保存总体结果
+    if all_results:
+        summary = {
+            'timestamp': datetime.now().isoformat(),
+            'total_images': len(all_results),
+            'total_tiles': sum([r['total_tiles'] for r in all_results]),
+            'results': all_results
+        }
+        
+        summary_file = output_path / "large_image_tile_summary.json"
+        with open(summary_file, 'w', encoding='utf-8') as f:
+            json.dump(summary, f, indent=2, ensure_ascii=False, default=str)
+        
+        print(f"\n" + "=" * 60)
+        print("大图瓦片预测完成总结")
+        print("=" * 60)
+        print(f"总图像数: {len(all_results)}")
+        print(f"总瓦片数: {summary['total_tiles']}")
+        print(f"结果保存目录: {output_path}")
+        print(f"总结文件: {summary_file}")
+        
+        # 显示浒苔分布统计
+        all_ratios = []
+        for result in all_results:
+            all_ratios.extend([r['statistics']['seaweed_ratio'] for r in result['tile_results']])
+        
+        if all_ratios:
+            print(f"\n浒苔分布统计:")
+            print(f"  - 最小浒苔比例: {min(all_ratios):.2%}")
+            print(f"  - 最大浒苔比例: {max(all_ratios):.2%}")
+            print(f"  - 浒苔比例标准差: {np.std(all_ratios):.2%}")
+    
+    print("\n🎉 大图瓦片浒苔预测完成！")
+    print(f"预测结果保存在: {output_dir}")
+
+if __name__ == "__main__":
+    main()

predict_seaweed_256x256_simple.py

@@ -0,0 +1,512 @@
+"""
+浒苔预测脚本 - 256x256图像尺寸
+使用训练好的模型进行浒苔分割预测 - 简化版本
+"""
+import os
+import torch
+import numpy as np
+from pathlib import Path
+import json
+import time
+from datetime import datetime
+import cv2
+from tqdm import tqdm
+
+# 导入自定义模块
+from dinov3_deeplabv3plus import DinoV3DeepLabV3Plus
+
+def load_model(model_path, config, device):
+    """加载训练好的模型"""
+    print(f"正在加载模型: {model_path}")
+    
+    # 创建模型
+    model = DinoV3DeepLabV3Plus(
+        num_classes=config['num_classes'],
+        backbone_name=config['backbone_name'],
+        pretrained=False,  # 不加载预训练权重，使用我们自己的权重
+        weights=config['backbone_weights'],
+        use_4channel=config['use_4channel']
+    ).to(device)
+    
+    # 加载权重
+    checkpoint = torch.load(model_path, map_location=device)
+    
+    # 处理不同的checkpoint格式
+    if 'model_state_dict' in checkpoint:
+        model.load_state_dict(checkpoint['model_state_dict'])
+        print(f"加载模型权重 (epoch {checkpoint.get('epoch', 'unknown')})")
+    elif 'state_dict' in checkpoint:
+        model.load_state_dict(checkpoint['state_dict'])
+        print("加载模型权重")
+    else:
+        # 直接加载state dict
+        model.load_state_dict(checkpoint)
+        print("加载模型权重")
+    
+    model.eval()
+    print("模型加载完成，进入评估模式")
+    return model
+
+def process_image_for_model(image_path, target_size=256, use_4channel=True):
+    """处理图像用于模型预测 - 使用与训练相同的方式"""
+    # 使用OpenCV读取16位TIF图像
+    img_16bit = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
+    
+    if img_16bit is None:
+        raise ValueError(f"无法读取图像文件: {image_path}")
+    
+    # 使用rasterio方式读取多波段数据
+    try:
+        import rasterio
+        with rasterio.open(image_path) as src:
+            # 读取所有波段
+            image = src.read()
+            # 转换为HWC格式
+            image = np.transpose(image, (1, 2, 0))
+    except Exception as e:
+        print(f"rasterio读取失败，使用OpenCV: {e}")
+        # 回退到OpenCV
+        image = img_16bit
+    
+    # 处理通道数
+    if use_4channel and image.shape[2] >= 4:
+        # 使用4通道
+        processed_image = image[:, :, :4]
+    else:
+        # 使用3通道假彩色（432波段）
+        if image.shape[2] >= 4:
+            # 选择第4,3,2波段（索引3,2,1）
+            processed_image = image[:, :, [3, 2, 1]]
+        else:
+            # 如果通道数不足，使用可用通道
+            processed_image = image[:, :, :3]
+            if image.shape[2] < 3:
+                # 如果还是不足3通道，重复填充
+                while processed_image.shape[2] < 3:
+                    processed_image = np.concatenate([processed_image, processed_image[:, :, -1:]], axis=2)
+    
+    # 调整尺寸
+    if processed_image.shape[0] != target_size or processed_image.shape[1] != target_size:
+        processed_image = cv2.resize(processed_image, (target_size, target_size), interpolation=cv2.INTER_LINEAR)
+    
+    # 转换为float32并标准化到0-1范围
+    processed_image = processed_image.astype(np.float32)
+    if processed_image.max() > 1.0:
+        processed_image = processed_image / 65535.0  # 16位最大值
+    
+    # 标准化
+    if use_4channel:
+        # 4通道标准化
+        mean = np.array([0.430, 0.411, 0.296, 0.350])
+        std = np.array([0.213, 0.156, 0.143, 0.180])
+    else:
+        # 3通道标准化
+        mean = np.array([0.430, 0.411, 0.296])
+        std = np.array([0.213, 0.156, 0.143])
+    
+    # 应用标准化
+    for i in range(processed_image.shape[2]):
+        processed_image[:, :, i] = (processed_image[:, :, i] - mean[i]) / std[i]
+    
+    # 转换为tensor并添加batch维度
+    img_tensor = torch.from_numpy(processed_image).permute(2, 0, 1).float()
+    
+    return img_tensor.unsqueeze(0)  # 添加batch维度
+
+def predict_image(model, image_path, device, config, threshold=0.5):
+    """
+    预测单张图像
+    
+    Args:
+        model: 训练好的模型
+        image_path: 图像路径
+        device: 设备
+        config: 配置字典
+        threshold: 浒苔预测阈值，只有当浒苔概率超过此阈值时才预测为浒苔（默认0.5）
+    """
+    # 处理图像
+    image_tensor = process_image_for_model(
+        image_path, 
+        target_size=config['image_size'], 
+        use_4channel=config['use_4channel']
+    ).to(device)
+    
+    # 预测
+    with torch.no_grad():
+        start_time = time.time()
+        output = model(image_tensor)
+        inference_time = time.time() - start_time
+        
+        # 处理输出格式
+        if isinstance(output, dict):
+            output = output['out']
+        
+        # 获取预测结果
+        pred = torch.softmax(output, dim=1)
+        
+        # 计算浒苔概率
+        seaweed_prob = pred[0, 1].squeeze(0).cpu().numpy()  # 类别1是浒苔
+        
+        # 使用阈值进行预测：只有当浒苔概率超过阈值时才预测为浒苔
+        # 这样可以避免在无浒苔图片中，由于两个类别概率接近而误判
+        pred_class = (seaweed_prob > threshold).astype(np.uint8)
+        
+        return {
+            'prediction': pred_class,
+            'seaweed_probability': seaweed_prob,
+            'inference_time': inference_time
+        }
+
+def save_prediction_results(prediction, seaweed_prob, output_path, base_name):
+    """保存预测结果到文件"""
+    # 保存预测掩码 (0-1)
+    pred_mask = prediction.astype(np.uint8)
+    np.save(str(output_path / f"{base_name}_prediction.npy"), pred_mask)
+    
+    # 保存概率图 (0-1)
+    np.save(str(output_path / f"{base_name}_probability.npy"), seaweed_prob)
+    
+    # 保存为PNG图像
+    pred_png = (prediction * 255).astype(np.uint8)
+    prob_png = (seaweed_prob * 255).astype(np.uint8)
+    
+    cv2.imwrite(str(output_path / f"{base_name}_prediction.png"), pred_png)
+    cv2.imwrite(str(output_path / f"{base_name}_probability.png"), prob_png)
+
+def calculate_metrics(prediction, target, num_classes=2):
+    """计算评估指标 - 精确率、召回率、IoU、F1"""
+    # 确保输入是numpy数组
+    if isinstance(prediction, torch.Tensor):
+        prediction = prediction.cpu().numpy()
+    if isinstance(target, torch.Tensor):
+        target = target.cpu().numpy()
+    
+    # 展平数组
+    pred_flat = prediction.flatten()
+    target_flat = target.flatten()
+    
+    # 计算混淆矩阵
+    confusion_matrix = np.zeros((num_classes, num_classes))
+    for i in range(num_classes):
+        for j in range(num_classes):
+            confusion_matrix[i, j] = np.sum((pred_flat == i) & (target_flat == j))
+    
+    # 计算每个类别的指标
+    metrics_per_class = []
+    for i in range(num_classes):
+        tp = confusion_matrix[i, i]  # 真正例
+        fp = confusion_matrix[i, :].sum() - tp  # 假正例
+        fn = confusion_matrix[:, i].sum() - tp  # 假负例
+        tn = confusion_matrix.sum() - tp - fp - fn  # 真负例
+        
+        # 计算指标
+        precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
+        recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
+        iou = tp / (tp + fp + fn) if (tp + fp + fn) > 0 else 0.0
+        f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0.0
+        
+        metrics_per_class.append({
+            'precision': precision,
+            'recall': recall,
+            'iou': iou,
+            'f1': f1,
+            'tp': int(tp),
+            'fp': int(fp),
+            'fn': int(fn),
+            'tn': int(tn)
+        })
+    
+    # 只计算前景（浒苔，类别1）的指标
+    foreground_metrics = metrics_per_class[1] if len(metrics_per_class) > 1 else metrics_per_class[0]
+    
+    # 计算整体准确率
+    accuracy = np.sum(pred_flat == target_flat) / len(pred_flat)
+    
+    return {
+        'accuracy': float(accuracy),
+        'foreground_precision': foreground_metrics['precision'],
+        'foreground_recall': foreground_metrics['recall'],
+        'foreground_iou': foreground_metrics['iou'],
+        'foreground_f1': foreground_metrics['f1'],
+        'foreground_tp': foreground_metrics['tp'],
+        'foreground_fp': foreground_metrics['fp'],
+        'foreground_fn': foreground_metrics['fn'],
+        'foreground_tn': foreground_metrics['tn'],
+        'metrics_per_class': metrics_per_class
+    }
+
+def calculate_statistics(prediction, seaweed_prob):
+    """计算统计信息"""
+    total_pixels = prediction.size
+    seaweed_pixels = np.sum(prediction == 1)
+    seaweed_ratio = seaweed_pixels / total_pixels
+    
+    # 计算平均概率
+    avg_probability = np.mean(seaweed_prob)
+    max_probability = np.max(seaweed_prob)
+    
+    return {
+        'total_pixels': int(total_pixels),
+        'seaweed_pixels': int(seaweed_pixels),
+        'seaweed_ratio': float(seaweed_ratio),
+        'avg_probability': float(avg_probability),
+        'max_probability': float(max_probability)
+    }
+
+def load_label_mask(mask_path, target_size=256):
+    """加载标签掩码"""
+    try:
+        # 尝试使用OpenCV读取
+        mask = cv2.imread(mask_path, cv2.IMREAD_UNCHANGED)
+        if mask is None:
+            # 如果OpenCV失败，尝试使用rasterio
+            import rasterio
+            with rasterio.open(mask_path) as src:
+                mask = src.read(1)  # 读取第一个波段
+        else:
+            # 如果OpenCV成功但有多通道，只取第一个通道
+            if len(mask.shape) > 2:
+                mask = mask[:, :, 0]
+        
+        # 调整尺寸
+        if mask.shape[0] != target_size or mask.shape[1] != target_size:
+            mask = cv2.resize(mask, (target_size, target_size), interpolation=cv2.INTER_NEAREST)
+        
+        # 确保是二值掩码 (0, 1)
+        mask = (mask > 0).astype(np.uint8)
+        
+        return mask
+        
+    except Exception as e:
+        print(f"加载标签失败 {mask_path}: {str(e)}")
+        return None
+
+def main():
+    """主预测函数"""
+    print("=" * 60)
+    print("浒苔预测系统 - 256x256图像 (带评估指标)")
+    print("=" * 60)
+    
+    # 配置
+    model_path = "outputs/seaweed_segmentation_improved_epoch500/best_checkpoint.pth"
+    config_path = "outputs/seaweed_segmentation_improved_epoch500/config.json"
+    test_image_dir = "data/test/images"
+    test_mask_dir = "data/test/masks"  # 添加标签目录
+    output_dir = "outputs/predictions_256x256_results"
+    
+    # 预测阈值：只有当浒苔概率超过此阈值时才预测为浒苔
+    # 建议值：0.5-0.7，可以根据验证集调整
+    prediction_threshold = 0.5
+    
+    # 创建设备
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(f"使用设备: {device}")
+    
+    # 加载配置
+    with open(config_path, 'r', encoding='utf-8') as f:
+        config = json.load(f)
+    
+    print(f"模型配置:")
+    print(f"  - 输入通道: {4 if config['use_4channel'] else 3}")
+    print(f"  - 图像尺寸: {config['image_size']}x{config['image_size']}")
+    print(f"  - 类别数: {config['num_classes']}")
+    print(f"  - 预测阈值: {prediction_threshold}")
+    
+    # 加载模型
+    model = load_model(model_path, config, device)
+    
+    # 创建输出目录
+    output_path = Path(output_dir)
+    output_path.mkdir(parents=True, exist_ok=True)
+    
+    # 获取测试图像
+    test_images = [f for f in os.listdir(test_image_dir) if f.endswith('.TIF')]
+    print(f"\n找到 {len(test_images)} 张测试图像")
+    
+    # 检查是否有标签数据
+    has_labels = os.path.exists(test_mask_dir)
+    if has_labels:
+        print(f"找到标签目录，将进行评估指标计算")
+    else:
+        print(f"未找到标签目录，仅进行预测")
+    
+    # 预测结果统计
+    results = []
+    evaluation_results = []
+    total_inference_time = 0
+    
+    print("\n开始预测...")
+    
+    # 使用tqdm显示进度条
+    for i, image_file in enumerate(tqdm(test_images, desc="预测进度"), 1):
+        image_path = os.path.join(test_image_dir, image_file)
+        
+        try:
+            # 预测（使用阈值）
+            result = predict_image(model, image_path, device, config, threshold=prediction_threshold)
+            
+            # 计算统计信息
+            stats = calculate_statistics(result['prediction'], result['seaweed_probability'])
+            
+            # 如果有标签，计算评估指标
+            if has_labels:
+                # 构建对应的标签文件名
+                mask_file = image_file.replace('.TIF', '.png')  # 假设标签是PNG格式
+                mask_path = os.path.join(test_mask_dir, mask_file)
+                
+                if os.path.exists(mask_path):
+                    # 加载真实标签
+                    true_mask = load_label_mask(mask_path, config['image_size'])
+                    
+                    if true_mask is not None:
+                        # 计算评估指标
+                        metrics = calculate_metrics(result['prediction'], true_mask)
+                        
+                        # 保存评估结果
+                        eval_result = {
+                            'image_file': image_file,
+                            'accuracy': metrics['accuracy'],
+                            'precision': metrics['foreground_precision'],
+                            'recall': metrics['foreground_recall'],
+                            'iou': metrics['foreground_iou'],
+                            'f1': metrics['foreground_f1'],
+                            'tp': metrics['foreground_tp'],
+                            'fp': metrics['foreground_fp'],
+                            'fn': metrics['foreground_fn'],
+                            'tn': metrics['foreground_tn']
+                        }
+                        evaluation_results.append(eval_result)
+                        
+                        # 在统计信息中添加评估指标
+                        stats.update({
+                            'accuracy': metrics['accuracy'],
+                            'precision': metrics['foreground_precision'],
+                            'recall': metrics['foreground_recall'],
+                            'iou': metrics['foreground_iou'],
+                            'f1': metrics['foreground_f1']
+                        })
+            
+            # 保存预测结果
+            base_name = Path(image_file).stem
+            save_prediction_results(result['prediction'], result['seaweed_probability'], output_path, base_name)
+            
+            # 保存结果
+            result_info = {
+                'image_file': image_file,
+                'inference_time': float(result['inference_time']),
+                'statistics': stats
+            }
+            results.append(result_info)
+            
+            total_inference_time += result['inference_time']
+            
+        except Exception as e:
+            print(f"  - 预测失败: {str(e)}")
+            continue
+    
+    # 保存预测结果
+    results_file = output_path / "prediction_results.json"
+    with open(results_file, 'w', encoding='utf-8') as f:
+        json.dump(results, f, indent=2, ensure_ascii=False, default=str)
+    
+    # 保存评估结果
+    if evaluation_results:
+        eval_file = output_path / "evaluation_results.json"
+        with open(eval_file, 'w', encoding='utf-8') as f:
+            json.dump(evaluation_results, f, indent=2, ensure_ascii=False, default=str)
+    
+    # 生成总结报告
+    if results:
+        avg_inference_time = total_inference_time / len(results)
+        
+        # 计算平均统计
+        all_ratios = [r['statistics']['seaweed_ratio'] for r in results]
+        avg_seaweed_ratio = np.mean(all_ratios)
+        
+        print("\n" + "=" * 60)
+        print("预测完成总结")
+        print("=" * 60)
+        print(f"总图像数: {len(results)}")
+        print(f"平均推理时间: {avg_inference_time:.3f}s")
+        print(f"平均浒苔比例: {avg_seaweed_ratio:.2%}")
+        
+        # 如果有评估结果，显示评估指标
+        if evaluation_results:
+            avg_accuracy = np.mean([r['accuracy'] for r in evaluation_results])
+            avg_precision = np.mean([r['precision'] for r in evaluation_results])
+            avg_recall = np.mean([r['recall'] for r in evaluation_results])
+            avg_iou = np.mean([r['iou'] for r in evaluation_results])
+            avg_f1 = np.mean([r['f1'] for r in evaluation_results])
+            
+            print(f"\n评估指标:")
+            print(f"  - 准确率 (Accuracy): {avg_accuracy:.4f}")
+            print(f"  - 精确率 (Precision): {avg_precision:.4f}")
+            print(f"  - 召回率 (Recall): {avg_recall:.4f}")
+            print(f"  - IoU: {avg_iou:.4f}")
+            print(f"  - F1分数: {avg_f1:.4f}")
+            
+            # 计算总体混淆矩阵
+            total_tp = sum([r['tp'] for r in evaluation_results])
+            total_fp = sum([r['fp'] for r in evaluation_results])
+            total_fn = sum([r['fn'] for r in evaluation_results])
+            total_tn = sum([r['tn'] for r in evaluation_results])
+            
+            print(f"\n总体混淆矩阵:")
+            print(f"  - 真正例 (TP): {total_tp:,}")
+            print(f"  - 假正例 (FP): {total_fp:,}")
+            print(f"  - 假负例 (FN): {total_fn:,}")
+            print(f"  - 真负例 (TN): {total_tn:,}")
+        
+        print(f"\n结果保存目录: {output_path}")
+        print(f"详细结果文件: {results_file}")
+        
+        # 保存总结报告
+        summary = {
+            'timestamp': datetime.now().isoformat(),
+            'total_images': len(results),
+            'avg_inference_time': avg_inference_time,
+            'avg_seaweed_ratio': float(avg_seaweed_ratio),
+            'results': results
+        }
+        
+        # 添加评估指标到总结报告
+        if evaluation_results:
+            summary.update({
+                'avg_accuracy': float(avg_accuracy),
+                'avg_precision': float(avg_precision),
+                'avg_recall': float(avg_recall),
+                'avg_iou': float(avg_iou),
+                'avg_f1': float(avg_f1),
+                'total_confusion_matrix': {
+                    'tp': total_tp,
+                    'fp': total_fp,
+                    'fn': total_fn,
+                    'tn': total_tn
+                },
+                'evaluation_results': evaluation_results
+            })
+        
+        summary_file = output_path / "prediction_summary.json"
+        with open(summary_file, 'w', encoding='utf-8') as f:
+            json.dump(summary, f, indent=2, ensure_ascii=False, default=str)
+        
+        print(f"总结报告: {summary_file}")
+        
+        # 显示一些统计信息
+        ratios = [r['statistics']['seaweed_ratio'] for r in results]
+        print(f"\n浒苔分布统计:")
+        print(f"  - 最小浒苔比例: {min(ratios):.2%}")
+        print(f"  - 最大浒苔比例: {max(ratios):.2%}")
+        print(f"  - 浒苔比例标准差: {np.std(ratios):.2%}")
+        
+        # 列出保存的文件类型
+        print(f"\n保存的文件类型:")
+        print(f"  - .npy文件: NumPy数组格式的预测结果")
+        print(f"  - .png文件: 可视化的图像格式")
+        print(f"  - .json文件: 预测结果统计信息和评估指标")
+    
+    print("\n🎉 浒苔预测和评估完成！")
+    print(f"预测结果保存在: {output_dir}")
+
+if __name__ == "__main__":
+    main()

prepare_labels.py

@@ -0,0 +1,156 @@
+"""
+标签数据预处理脚本
+将TIF格式的标签转换为PNG格式的二值mask
+"""
+import os
+import numpy as np
+from PIL import Image
+import rasterio
+from pathlib import Path
+import shutil
+
+def convert_tif_label_to_png_mask(tif_path, output_path, threshold=1):
+    """将TIF标签转换为PNG二值mask"""
+    try:
+        with rasterio.open(tif_path) as src:
+            # 读取第一个波段
+            label_data = src.read(1)
+            
+            # 创建二值mask (0: 背景, 255: 前景)
+            mask = np.zeros_like(label_data, dtype=np.uint8)
+            mask[label_data >= threshold] = 255
+            
+            # 保存为PNG
+            mask_img = Image.fromarray(mask, mode='L')
+            mask_img.save(output_path)
+            
+            print(f"转换: {tif_path} -> {output_path}")
+            print(f"  原始数据范围: {label_data.min()} - {label_data.max()}")
+            print(f"  Mask唯一值: {np.unique(mask)}")
+            return True
+            
+    except Exception as e:
+        print(f"转换失败 {tif_path}: {str(e)}")
+        return False
+
+def prepare_dataset_labels():
+    """准备数据集标签"""
+    
+    # 创建mask目录
+    train_mask_dir = Path("data/train/masks")
+    val_mask_dir = Path("data/val/masks")
+    test_mask_dir = Path("data/test/masks")
+    
+    train_mask_dir.mkdir(parents=True, exist_ok=True)
+    val_mask_dir.mkdir(parents=True, exist_ok=True)
+    test_mask_dir.mkdir(parents=True, exist_ok=True)
+    
+    # 处理训练集标签
+    train_label_dir = Path("data/train/labels")
+    if train_label_dir.exists():
+        print("处理训练集标签...")
+        label_files = [f for f in train_label_dir.glob("*.TIF")]
+        
+        for label_file in label_files:
+            # 获取对应的图像文件名
+            # 假设标签文件名是 labelXXXXX.TIF，图像是 patchXXXXX.TIF
+            label_name = label_file.stem  # label10012
+            if label_name.startswith('label'):
+                # 提取数字部分
+                number = label_name[5:]  # 10012
+                image_name = f"patch{number}.TIF"
+                
+                # 检查对应的图像是否存在
+                image_path = Path("data/train/images") / image_name
+                if image_path.exists():
+                    # 转换标签为mask
+                    mask_name = f"patch{number}.png"
+                    mask_path = train_mask_dir / mask_name
+                    
+                    if convert_tif_label_to_png_mask(str(label_file), str(mask_path)):
+                        print(f"  成功: {label_name} -> {mask_name}")
+                else:
+                    print(f"  警告: 找不到对应的图像 {image_name}")
+    
+    # 处理验证集标签
+    val_label_dir = Path("data/val/labels")
+    if val_label_dir.exists():
+        print("\n处理验证集标签...")
+        label_files = [f for f in val_label_dir.glob("*.TIF")]
+        
+        for label_file in label_files:
+            label_name = label_file.stem
+            if label_name.startswith('label'):
+                number = label_name[5:]
+                image_name = f"patch{number}.TIF"
+                
+                image_path = Path("data/val/images") / image_name
+                if image_path.exists():
+                    mask_name = f"patch{number}.png"
+                    mask_path = val_mask_dir / mask_name
+                    
+                    if convert_tif_label_to_png_mask(str(label_file), str(mask_path)):
+                        print(f"  成功: {label_name} -> {mask_name}")
+    
+    # 处理测试集标签
+    test_label_dir = Path("data/test/labels")
+    if test_label_dir.exists():
+        print("\n处理测试集标签...")
+        label_files = [f for f in test_label_dir.glob("*.TIF")]
+        
+        for label_file in label_files:
+            label_name = label_file.stem
+            if label_name.startswith('label'):
+                number = label_name[5:]
+                image_name = f"patch{number}.TIF"
+                
+                image_path = Path("data/test/images") / image_name
+                if image_path.exists():
+                    mask_name = f"patch{number}.png"
+                    mask_path = test_mask_dir / mask_name
+                    
+                    if convert_tif_label_to_png_mask(str(label_file), str(mask_path)):
+                        print(f"  成功: {label_name} -> {mask_name}")
+    
+    print("\n标签数据准备完成！")
+
+def check_data_integrity():
+    """检查数据完整性"""
+    print("\n检查数据完整性...")
+    
+    datasets = [
+        ("训练集", "data/train/images", "data/train/masks"),
+        ("验证集", "data/val/images", "data/val/masks"),
+        ("测试集", "data/test/images", "data/test/masks")
+    ]
+    
+    for name, image_dir, mask_dir in datasets:
+        image_path = Path(image_dir)
+        mask_path = Path(mask_dir)
+        
+        if image_path.exists() and mask_path.exists():
+            image_files = list(image_path.glob("*.TIF"))
+            mask_files = list(mask_path.glob("*.png"))
+            
+            print(f"{name}:")
+            print(f"  图像文件: {len(image_files)} 个")
+            print(f"  Mask文件: {len(mask_files)} 个")
+            
+            # 检查匹配情况
+            matched = 0
+            for image_file in image_files:
+                expected_mask = mask_path / (image_file.stem + ".png")
+                if expected_mask.exists():
+                    matched += 1
+            
+            print(f"  匹配文件: {matched} 个")
+            if matched < len(image_files):
+                print(f"  ⚠️  警告: 有 {len(image_files) - matched} 个图像缺少对应的mask")
+        else:
+            print(f"{name}: 目录不存在")
+
+if __name__ == "__main__":
+    print("开始准备标签数据...")
+    prepare_dataset_labels()
+    check_data_integrity()
+    print("\n数据准备完成！")

requirements.txt

@@ -0,0 +1,13 @@
+torch
+torchvision
+numpy
+pillow
+rasterio
+albumentations
+opencv-python
+matplotlib
+tqdm
+scipy
+einops
+omegaconf
+deepspeed; platform_system != "Windows"

scripts/build_marine_feature_dataset.py

@@ -0,0 +1,327 @@
+"""Build a normalized manifest for marine ecological feature datasets.
+
+The script scans one or more roots, classifies likely feature types from path
+keywords, pairs images and masks when possible, and writes a manifest without
+copying large raster files.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import re
+from dataclasses import asdict, dataclass
+from datetime import datetime
+from pathlib import Path
+from typing import Iterable
+
+
+IMAGE_EXTS = {".tif", ".tiff", ".png", ".jpg", ".jpeg"}
+MASK_HINTS = ("mask", "masks", "label", "labels", "gt", "annotation", "annotations", "seg")
+IMAGE_HINTS = ("image", "images", "img", "imgs", "tif", "tile", "tiles")
+
+ELEMENT_KEYWORDS = {
+    "green_tide": ("浒苔", "绿潮", "green_tide", "greentide", "entgreentide", "enteromorpha", "seaweed"),
+    "red_tide": ("赤潮", "red_tide", "redtide", "harmful_algal", "hab"),
+    "golden_tide": ("马尾藻", "金潮", "sarg", "sargassum", "golden_tide", "goldentide"),
+    "aquaculture": ("养殖", "aquaculture", "raft", "cage", "pond"),
+}
+
+SATELLITE_PATTERN = re.compile(r"\b(GF\d+|HY\d+|Sentinel-?2|Landsat-?\d*)\b", re.IGNORECASE)
+DATE_PATTERN = re.compile(r"(20\d{6}|19\d{6})")
+PATCH_SIZE_PATTERN = re.compile(r"(?:^|[_\\/\-])(?:size)?(128|256|512|1024)(?:[_\\/\-]|$)")
+
+
+@dataclass
+class AssetRecord:
+    asset_id: str
+    path: str
+    filename: str
+    suffix: str
+    role: str
+    element: str
+    satellite: str | None
+    sensor: str | None
+    acquired_at: str | None
+    patch_size: int | None
+    source_project: str
+    source_dataset: str
+    size_bytes: int
+    modified_at: str
+    quality_flags: list[str]
+
+
+@dataclass
+class SampleRecord:
+    sample_id: str
+    element: str
+    task_type: str
+    image_path: str
+    mask_path: str | None
+    label_encoding: dict[str, str] | None
+    satellite: str | None
+    sensor: str | None
+    resolution_m: float | None
+    patch_size: int | None
+    bands: list[str] | None
+    band_count: int | None
+    dtype: str | None
+    fusion: dict
+    acquired_at: str | None
+    source_project: str
+    source_dataset: str
+    split: str | None
+    quality_flags: list[str]
+    notes: str
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--roots", nargs="+", required=True, help="Dataset roots to scan.")
+    parser.add_argument("--output-root", required=True, help="Output normalized dataset root.")
+    parser.add_argument("--max-files", type=int, default=0, help="Optional scan limit for debugging.")
+    return parser.parse_args()
+
+
+def norm_text(path: Path) -> str:
+    return str(path).replace("\\", "/").lower()
+
+
+def infer_element(path: Path) -> str:
+    text = norm_text(path)
+    for element, keywords in ELEMENT_KEYWORDS.items():
+        if any(keyword.lower() in text for keyword in keywords):
+            return element
+    return "unknown"
+
+
+def infer_role(path: Path) -> str:
+    parts = [part.lower() for part in path.parts]
+    stem = path.stem.lower()
+    if any(hint in parts or hint in stem for hint in MASK_HINTS):
+        return "mask"
+    if any(hint in parts for hint in IMAGE_HINTS):
+        return "image"
+    if path.suffix.lower() in {".png", ".jpg", ".jpeg"} and any(hint in stem for hint in MASK_HINTS):
+        return "mask"
+    return "image"
+
+
+def infer_satellite(path: Path) -> str | None:
+    match = SATELLITE_PATTERN.search(str(path))
+    return match.group(1).upper().replace("-", "") if match else None
+
+
+def infer_sensor(path: Path) -> str | None:
+    upper = path.name.upper()
+    for sensor in ("PMS", "MUX", "MSS", "PAN", "WFV"):
+        if sensor in upper:
+            return sensor
+    return None
+
+
+def infer_date(path: Path) -> str | None:
+    match = DATE_PATTERN.search(path.name)
+    if not match:
+        return None
+    raw = match.group(1)
+    try:
+        return datetime.strptime(raw, "%Y%m%d").date().isoformat()
+    except ValueError:
+        return None
+
+
+def infer_patch_size(path: Path) -> int | None:
+    match = PATCH_SIZE_PATTERN.search(str(path))
+    return int(match.group(1)) if match else None
+
+
+def infer_source_project(path: Path, roots: list[Path]) -> str:
+    for root in roots:
+        try:
+            rel = path.relative_to(root)
+        except ValueError:
+            continue
+        return rel.parts[0] if len(rel.parts) > 1 else root.name
+    return path.parent.name
+
+
+def infer_split(path: Path) -> str | None:
+    parts = {part.lower() for part in path.parts}
+    for split in ("train", "val", "test"):
+        if split in parts:
+            return split
+    return None
+
+
+def source_dataset(path: Path) -> str:
+    for part in reversed(path.parts):
+        lower = part.lower()
+        if any(token in lower for token in ("gf", "sentinel", "landsat", "浒苔", "赤潮", "马尾藻", "养殖")):
+            return part
+    return path.parent.name
+
+
+def is_fused(path: Path) -> bool | None:
+    lower = path.name.lower()
+    if "fuse" in lower or "fusion" in lower or "pan" not in lower and "mux" in lower:
+        return True
+    if "pan" in lower or "mss" in lower:
+        return False
+    return None
+
+
+def infer_fusion(path: Path) -> dict:
+    fused = is_fused(path)
+    lower = path.name.lower()
+    if fused is True:
+        state = "fused_product"
+        method = "unknown_vendor_product"
+        persisted = True
+    elif fused is False and ("pan" in lower or "mss" in lower):
+        state = "none"
+        method = "none"
+        persisted = False
+    else:
+        state = "unknown"
+        method = "unknown"
+        persisted = False
+    return {
+        "state": state,
+        "method": method,
+        "sources": [{"role": "source", "path": str(path), "resolution_m": None}],
+        "target_resolution_m": None,
+        "native_multispectral_resolution_m": None,
+        "persisted": persisted,
+        "reproducible": state != "unknown",
+        "spectral_preservation": "unknown",
+        "notes": "Auto-inferred from local filename; verify before training.",
+    }
+
+
+def asset_id(path: Path) -> str:
+    safe = re.sub(r"[^A-Za-z0-9]+", "_", str(path.stem)).strip("_").lower()
+    return safe[:180]
+
+
+def iter_files(roots: Iterable[Path], max_files: int) -> Iterable[Path]:
+    count = 0
+    for root in roots:
+        if not root.exists():
+            continue
+        for path in root.rglob("*"):
+            if not path.is_file() or path.suffix.lower() not in IMAGE_EXTS:
+                continue
+            yield path
+            count += 1
+            if max_files and count >= max_files:
+                return
+
+
+def write_jsonl(path: Path, rows: Iterable[dict]) -> None:
+    with path.open("w", encoding="utf-8") as f:
+        for row in rows:
+            f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def main() -> None:
+    args = parse_args()
+    roots = [Path(root) for root in args.roots]
+    output_root = Path(args.output_root)
+    manifest_dir = output_root / "manifests"
+    report_dir = output_root / "reports"
+    manifest_dir.mkdir(parents=True, exist_ok=True)
+    report_dir.mkdir(parents=True, exist_ok=True)
+
+    assets: list[AssetRecord] = []
+    for path in iter_files(roots, args.max_files):
+        stat = path.stat()
+        role = infer_role(path)
+        flags = []
+        if infer_element(path) == "unknown":
+            flags.append("unknown_element")
+        if role == "image" and "black" in norm_text(path):
+            flags.append("possibly_invalid")
+        assets.append(
+            AssetRecord(
+                asset_id=asset_id(path),
+                path=str(path),
+                filename=path.name,
+                suffix=path.suffix.lower(),
+                role=role,
+                element=infer_element(path),
+                satellite=infer_satellite(path),
+                sensor=infer_sensor(path),
+                acquired_at=infer_date(path),
+                patch_size=infer_patch_size(path),
+                source_project=infer_source_project(path, roots),
+                source_dataset=source_dataset(path),
+                size_bytes=stat.st_size,
+                modified_at=datetime.fromtimestamp(stat.st_mtime).isoformat(timespec="seconds"),
+                quality_flags=flags,
+            )
+        )
+
+    masks_by_stem = {Path(asset.path).stem.lower(): asset for asset in assets if asset.role == "mask"}
+    samples: list[SampleRecord] = []
+    for asset in assets:
+        if asset.role != "image":
+            continue
+        path = Path(asset.path)
+        mask_asset = masks_by_stem.get(path.stem.lower())
+        element = asset.element if asset.element != "unknown" else (mask_asset.element if mask_asset else "unknown")
+        flags = list(asset.quality_flags)
+        if mask_asset is None:
+            flags.append("unpaired_image")
+        sample_id = f"{element}_{asset.asset_id}"
+        samples.append(
+            SampleRecord(
+                sample_id=sample_id,
+                element=element,
+                task_type="semantic_segmentation",
+                image_path=asset.path,
+                mask_path=mask_asset.path if mask_asset else None,
+                label_encoding={"0": "background", "1": element} if mask_asset else None,
+                satellite=asset.satellite,
+                sensor=asset.sensor,
+                resolution_m=None,
+                patch_size=asset.patch_size,
+                bands=None,
+                band_count=None,
+                dtype=None,
+                fusion=infer_fusion(path),
+                acquired_at=asset.acquired_at,
+                source_project=asset.source_project,
+                source_dataset=asset.source_dataset,
+                split=infer_split(path),
+                quality_flags=flags,
+                notes="auto-generated; verify ambiguous labels before training",
+            )
+        )
+
+    write_jsonl(manifest_dir / "assets_raw.jsonl", (asdict(asset) for asset in assets))
+    write_jsonl(manifest_dir / "samples.jsonl", (asdict(sample) for sample in samples))
+
+    with (report_dir / "asset_inventory.csv").open("w", newline="", encoding="utf-8-sig") as f:
+        writer = csv.DictWriter(f, fieldnames=list(asdict(assets[0]).keys()) if assets else ["asset_id"])
+        writer.writeheader()
+        for asset in assets:
+            row = asdict(asset)
+            row["quality_flags"] = ";".join(row["quality_flags"])
+            writer.writerow(row)
+
+    summary = {
+        "roots": [str(root) for root in roots],
+        "assets": len(assets),
+        "samples": len(samples),
+        "by_element": {},
+        "output_root": str(output_root),
+    }
+    for sample in samples:
+        summary["by_element"][sample.element] = summary["by_element"].get(sample.element, 0) + 1
+    print(json.dumps(summary, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/build_pseudo_dataset.py

@@ -0,0 +1,130 @@
+"""Build a train/val dataset from large images and prediction rasters.
+
+This is a bootstrap utility. Masks created from previous predictions are
+pseudo-labels, not human-verified ground truth.
+"""
+
+from __future__ import annotations
+
+import argparse
+import random
+from pathlib import Path
+
+import numpy as np
+import rasterio
+from rasterio.windows import Window
+
+
+def find_pairs(source_dir: Path):
+    images = [p for p in source_dir.glob("*.tif") if "_prediction" not in p.stem.lower()]
+    pairs = []
+    for image in images:
+        pred = None
+        for candidate in source_dir.glob(f"{image.stem}_*/{image.stem}_prediction.tif"):
+            pred = candidate
+            break
+        if pred:
+            pairs.append((image, pred))
+    return pairs
+
+
+def ensure_layout(output_dir: Path):
+    for split in ("train", "val"):
+        (output_dir / split / "images").mkdir(parents=True, exist_ok=True)
+        (output_dir / split / "masks").mkdir(parents=True, exist_ok=True)
+
+
+def write_tile(src, mask_src, window: Window, image_path: Path, mask_path: Path, foreground_threshold: int):
+    image = src.read(window=window)
+    mask = mask_src.read(1, window=window)
+    if image.shape[1] != window.height or image.shape[2] != window.width:
+        return False
+    if mask.shape[0] != window.height or mask.shape[1] != window.width:
+        return False
+
+    image_meta = src.meta.copy()
+    image_meta.update(
+        {
+            "height": int(window.height),
+            "width": int(window.width),
+            "transform": src.window_transform(window),
+            "compress": "lzw",
+        }
+    )
+    mask_meta = mask_src.meta.copy()
+    mask_meta.update(
+        {
+            "count": 1,
+            "dtype": "uint8",
+            "height": int(window.height),
+            "width": int(window.width),
+            "transform": mask_src.window_transform(window),
+            "compress": "lzw",
+        }
+    )
+
+    binary_mask = (mask >= foreground_threshold).astype(np.uint8) * 255
+    with rasterio.open(image_path, "w", **image_meta) as dst:
+        dst.write(image)
+    with rasterio.open(mask_path, "w", **mask_meta) as dst:
+        dst.write(binary_mask, 1)
+    return True
+
+
+def build_dataset(source_dir: Path, output_dir: Path, tile_size: int, stride: int, val_ratio: float, foreground_threshold: int):
+    pairs = find_pairs(source_dir)
+    if not pairs:
+        raise RuntimeError(f"No image/prediction pairs found under {source_dir}")
+    ensure_layout(output_dir)
+
+    rng = random.Random(42)
+    written = {"train": 0, "val": 0}
+
+    for pair_index, (image_path, pred_path) in enumerate(pairs):
+        with rasterio.open(image_path) as src, rasterio.open(pred_path) as mask_src:
+            windows = []
+            for y in range(0, src.height - tile_size + 1, stride):
+                for x in range(0, src.width - tile_size + 1, stride):
+                    windows.append(Window(x, y, tile_size, tile_size))
+            rng.shuffle(windows)
+
+            for tile_index, window in enumerate(windows):
+                split = "val" if rng.random() < val_ratio else "train"
+                name = f"pair{pair_index:02d}_{tile_index:06d}.tif"
+                ok = write_tile(
+                    src,
+                    mask_src,
+                    window,
+                    output_dir / split / "images" / name,
+                    output_dir / split / "masks" / name,
+                    foreground_threshold,
+                )
+                if ok:
+                    written[split] += 1
+
+    print(f"Wrote pseudo dataset to {output_dir}")
+    print(f"train tiles: {written['train']}")
+    print(f"val tiles: {written['val']}")
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--source-dir", default="data")
+    parser.add_argument("--output-dir", default="data_pseudo")
+    parser.add_argument("--tile-size", type=int, default=256)
+    parser.add_argument("--stride", type=int, default=256)
+    parser.add_argument("--val-ratio", type=float, default=0.15)
+    parser.add_argument("--foreground-threshold", type=int, default=1)
+    args = parser.parse_args()
+    build_dataset(
+        Path(args.source_dir),
+        Path(args.output_dir),
+        args.tile_size,
+        args.stride,
+        args.val_ratio,
+        args.foreground_threshold,
+    )
+
+
+if __name__ == "__main__":
+    main()

scripts/compose_task_profile.py

@@ -0,0 +1,176 @@
+"""Compose task profiles from Markdown capability cards."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+from typing import Any
+
+
+REGISTRY_ROOT = Path(__file__).resolve().parents[1] / "docs" / "registry"
+
+
+def parse_scalar(value: str) -> Any:
+    value = value.strip()
+    if value in {"true", "True"}:
+        return True
+    if value in {"false", "False"}:
+        return False
+    if value in {"null", "None"}:
+        return None
+    if value.startswith("[") and value.endswith("]"):
+        inner = value[1:-1].strip()
+        if not inner:
+            return []
+        return [parse_scalar(part.strip()) for part in inner.split(",")]
+    try:
+        if "." in value:
+            return float(value)
+        return int(value)
+    except ValueError:
+        return value.strip("\"'")
+
+
+def parse_front_matter(text: str) -> tuple[dict[str, Any], str]:
+    lines = text.splitlines()
+    if not lines or lines[0].strip() != "---":
+        return {}, text
+    meta: dict[str, Any] = {}
+    end = None
+    for idx, line in enumerate(lines[1:], start=1):
+        if line.strip() == "---":
+            end = idx
+            break
+        if not line.strip() or line.lstrip().startswith("#"):
+            continue
+        if ":" not in line:
+            continue
+        key, value = line.split(":", 1)
+        meta[key.strip()] = parse_scalar(value)
+    if end is None:
+        return meta, text
+    return meta, "\n".join(lines[end + 1 :]).strip()
+
+
+def load_card(kind: str, card_id: str, registry_root: Path) -> dict[str, Any]:
+    path = registry_root / kind / f"{card_id}.md"
+    if not path.exists():
+        available = sorted(p.stem for p in (registry_root / kind).glob("*.md"))
+        raise FileNotFoundError(f"Card not found: {path}. Available {kind}: {available}")
+    text = path.read_text(encoding="utf-8")
+    meta, body = parse_front_matter(text)
+    return {"kind": kind, "id": card_id, "path": str(path), "meta": meta, "body": body}
+
+
+def list_cards(registry_root: Path) -> dict[str, list[str]]:
+    result: dict[str, list[str]] = {}
+    for kind in ("elements", "satellites", "sensors", "resolutions"):
+        folder = registry_root / kind
+        result[kind] = sorted(p.stem for p in folder.glob("*.md")) if folder.exists() else []
+    return result
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--element")
+    parser.add_argument("--satellite")
+    parser.add_argument("--sensor")
+    parser.add_argument("--resolution")
+    parser.add_argument(
+        "--fusion",
+        help="Optional sensor/product fusion card, for example FUSED_OPTICAL or STREAM_FUSION.",
+    )
+    parser.add_argument("--registry-root", default=str(REGISTRY_ROOT))
+    parser.add_argument("--output", required=False)
+    parser.add_argument("--list", action="store_true", help="List available cards and exit.")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    registry_root = Path(args.registry_root)
+
+    if args.list:
+        print(json.dumps(list_cards(registry_root), indent=2, ensure_ascii=False))
+        return
+
+    required = {
+        "elements": args.element,
+        "satellites": args.satellite,
+        "sensors": args.sensor,
+        "resolutions": args.resolution,
+    }
+    missing = [key for key, value in required.items() if not value]
+    if missing:
+        raise SystemExit(f"Missing required cards: {missing}. Use --list to inspect available cards.")
+
+    cards = {kind: load_card(kind, card_id, registry_root) for kind, card_id in required.items() if card_id}
+    if args.fusion:
+        cards["fusion"] = load_card("sensors", args.fusion, registry_root)
+
+    element_meta = cards["elements"]["meta"]
+    sensor_meta = cards["sensors"]["meta"]
+    resolution_meta = cards["resolutions"]["meta"]
+    fusion_meta = cards.get("fusion", {}).get("meta", {})
+    fusion_state = fusion_meta.get("fusion_state", sensor_meta.get("fusion_state", "none"))
+
+    profile = {
+        "profile_id": "_".join(
+            part for part in [args.element, args.satellite, args.sensor, args.fusion, args.resolution] if part
+        ),
+        "cards": cards,
+        "task": {
+            "element": args.element,
+            "task_types": element_meta.get("task_types", []),
+            "preferred_heads": element_meta.get("preferred_heads", []),
+            "label_formats": element_meta.get("label_formats", []),
+            "negative_policy": element_meta.get("negative_policy", "unlabeled_is_ignore"),
+        },
+        "input": {
+            "satellite": args.satellite,
+            "sensor": args.sensor,
+            "resolution_m": resolution_meta.get("resolution_m"),
+            "modalities": sensor_meta.get("modalities", []),
+            "common_bands": sensor_meta.get("common_bands", []),
+            "recommended_patch_sizes": resolution_meta.get("recommended_patch_sizes", []),
+        },
+        "fusion": {
+            "card": args.fusion,
+            "state": fusion_state,
+            "modalities": fusion_meta.get("modalities", sensor_meta.get("modalities", [])),
+            "supports_streaming_fusion": fusion_meta.get(
+                "supports_streaming_fusion", sensor_meta.get("supports_streaming_fusion", False)
+            ),
+            "requires_fusion_metadata": fusion_meta.get(
+                "requires_fusion_metadata", sensor_meta.get("requires_fusion_metadata", False)
+            ),
+            "required_manifest_fields": [
+                "state",
+                "method",
+                "sources",
+                "target_resolution_m",
+                "native_multispectral_resolution_m",
+                "persisted",
+                "reproducible",
+                "spectral_preservation",
+            ],
+        },
+        "constraints": {
+            "do_not_assume_external_coastline_or_land_mask": True,
+            "unlabeled_elements_are_ignore_not_negative": True,
+        },
+    }
+
+    output = json.dumps(profile, indent=2, ensure_ascii=False)
+    if args.output:
+        output_path = Path(args.output)
+        output_path.parent.mkdir(parents=True, exist_ok=True)
+        output_path.write_text(output + "\n", encoding="utf-8")
+        print(output_path)
+    else:
+        print(output)
+
+
+if __name__ == "__main__":
+    main()

scripts/evaluate_yolo_detection.py

@@ -0,0 +1,198 @@
+"""Evaluate a YOLO detector on a dataset split with fixed-threshold metrics."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+import yaml
+from PIL import Image
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--weights", required=True)
+    parser.add_argument("--data", required=True)
+    parser.add_argument("--split", default="test", choices=["train", "val", "test"])
+    parser.add_argument("--imgsz", type=int, default=640)
+    parser.add_argument("--conf", type=float, default=0.25)
+    parser.add_argument("--iou", type=float, default=0.5)
+    parser.add_argument("--device", default="0")
+    parser.add_argument("--output", required=True)
+    return parser.parse_args()
+
+
+def xywhn_to_xyxy(row: list[float], width: int, height: int) -> list[float]:
+    _, x, y, w, h = row[:5]
+    cx = x * width
+    cy = y * height
+    bw = w * width
+    bh = h * height
+    return [cx - bw / 2, cy - bh / 2, cx + bw / 2, cy + bh / 2]
+
+
+def box_iou(a: list[float], b: list[float]) -> float:
+    x1 = max(a[0], b[0])
+    y1 = max(a[1], b[1])
+    x2 = min(a[2], b[2])
+    y2 = min(a[3], b[3])
+    inter = max(0.0, x2 - x1) * max(0.0, y2 - y1)
+    area_a = max(0.0, a[2] - a[0]) * max(0.0, a[3] - a[1])
+    area_b = max(0.0, b[2] - b[0]) * max(0.0, b[3] - b[1])
+    denom = area_a + area_b - inter
+    return inter / denom if denom else 0.0
+
+
+def read_label(label_path: Path, width: int, height: int) -> list[list[float]]:
+    if not label_path.exists():
+        return []
+    boxes: list[list[float]] = []
+    for line in label_path.read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        values = [float(part) for part in line.split()]
+        boxes.append(xywhn_to_xyxy(values, width, height))
+    return boxes
+
+
+def resolve_split_images(data_yaml: Path, split: str) -> list[Path]:
+    data = yaml.safe_load(data_yaml.read_text(encoding="utf-8"))
+    root = Path(data.get("path", data_yaml.parent))
+    split_value = Path(data[split])
+    image_dir = split_value if split_value.is_absolute() else root / split_value
+    images = []
+    for suffix in ("*.jpg", "*.jpeg", "*.png", "*.tif", "*.tiff"):
+        images.extend(image_dir.glob(suffix))
+    return sorted(images)
+
+
+def label_for_image(image_path: Path) -> Path:
+    parts = list(image_path.parts)
+    for idx, part in enumerate(parts):
+        if part == "images":
+            parts[idx] = "labels"
+            return Path(*parts).with_suffix(".txt")
+    return image_path.with_suffix(".txt")
+
+
+def main() -> None:
+    args = parse_args()
+    from ultralytics import YOLO
+
+    image_paths = resolve_split_images(Path(args.data), args.split)
+    model = YOLO(args.weights)
+
+    image_tp = image_fn = image_fp = image_tn = 0
+    matched = false_positive_instances = false_negative_instances = 0
+    gt_instances = pred_instances = 0
+    matched_ious: list[float] = []
+    error_images: list[dict[str, object]] = []
+
+    for image_path in image_paths:
+        with Image.open(image_path) as image:
+            width, height = image.size
+        gt_boxes = read_label(label_for_image(image_path), width, height)
+        result = model.predict(
+            source=str(image_path),
+            imgsz=args.imgsz,
+            conf=args.conf,
+            device=args.device,
+            verbose=False,
+        )[0]
+        pred_boxes = result.boxes.xyxy.cpu().tolist() if result.boxes is not None else []
+        pred_scores = result.boxes.conf.cpu().tolist() if result.boxes is not None else []
+        pred_order = sorted(range(len(pred_boxes)), key=lambda i: pred_scores[i], reverse=True)
+
+        gt_instances += len(gt_boxes)
+        pred_instances += len(pred_boxes)
+        used_gt: set[int] = set()
+        image_matched = 0
+        image_false_positives = 0
+
+        for pred_idx in pred_order:
+            best_gt = -1
+            best_iou = 0.0
+            for gt_idx, gt_box in enumerate(gt_boxes):
+                if gt_idx in used_gt:
+                    continue
+                iou = box_iou(pred_boxes[pred_idx], gt_box)
+                if iou > best_iou:
+                    best_iou = iou
+                    best_gt = gt_idx
+            if best_gt >= 0 and best_iou >= args.iou:
+                used_gt.add(best_gt)
+                matched += 1
+                image_matched += 1
+                matched_ious.append(best_iou)
+            else:
+                false_positive_instances += 1
+                image_false_positives += 1
+
+        missed = len(gt_boxes) - image_matched
+        false_negative_instances += missed
+        if missed or image_false_positives:
+            error_images.append(
+                {
+                    "image": str(image_path),
+                    "gt_instances": len(gt_boxes),
+                    "pred_instances": len(pred_boxes),
+                    "matched_instances": image_matched,
+                    "false_positives": image_false_positives,
+                    "false_negatives": missed,
+                }
+            )
+
+        if gt_boxes:
+            if image_matched:
+                image_tp += 1
+            else:
+                image_fn += 1
+        elif image_false_positives:
+            image_fp += 1
+        else:
+            image_tn += 1
+
+    tpr = image_tp / (image_tp + image_fn) if image_tp + image_fn else None
+    fpr = image_fp / (image_fp + image_tn) if image_fp + image_tn else None
+    accuracy = (image_tp + image_tn) / len(image_paths) if image_paths else None
+    precision = matched / (matched + false_positive_instances) if matched + false_positive_instances else None
+    recall = matched / gt_instances if gt_instances else None
+    mean_iou = sum(matched_ious) / len(matched_ious) if matched_ious else None
+
+    metrics = {
+        "weights": args.weights,
+        "data": args.data,
+        "split": args.split,
+        "images": len(image_paths),
+        "conf_threshold": args.conf,
+        "iou_threshold": args.iou,
+        "classification": {
+            "TPR": tpr,
+            "FPR": fpr,
+            "Accuracy": accuracy,
+            "TP": image_tp,
+            "FN": image_fn,
+            "FP": image_fp,
+            "TN": image_tn,
+        },
+        "detection": {
+            "gt_instances": gt_instances,
+            "pred_instances": pred_instances,
+            "matched_instances": matched,
+            "false_positives": false_positive_instances,
+            "false_negatives": false_negative_instances,
+            "precision": precision,
+            "recall": recall,
+            "mean_matched_iou": mean_iou,
+        },
+        "error_images": error_images,
+    }
+    output = Path(args.output)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(metrics, indent=2, ensure_ascii=False), encoding="utf-8")
+    print(json.dumps(metrics, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/find_max_batch.ps1

@@ -0,0 +1,16 @@
+param(
+  [string]$Config = "configs/train_2gpu_4090d.json",
+  [int]$Start = 8,
+  [int]$Stop = 32,
+  [int]$Step = 2
+)
+
+$ErrorActionPreference = "Continue"
+for ($batch = $Start; $batch -le $Stop; $batch += $Step) {
+  Write-Host "Trying per-GPU batch=$batch"
+  powershell -ExecutionPolicy Bypass -File scripts/train_2gpu.ps1 -Config $Config -BatchSize $batch -Epochs 1
+  if ($LASTEXITCODE -ne 0) {
+    Write-Host "Failed at per-GPU batch=$batch. Use the previous successful value."
+    exit 0
+  }
+}

scripts/import_quality_gated_hf_dataset.py

@@ -0,0 +1,309 @@
+"""Import only quality-gated Hugging Face samples into project manifests.
+
+This script consumes `hf_assets_raw.jsonl` produced by
+`search_hf_marine_datasets.py`. It accepts only datasets with an explicit adapter
+and verifiable labels. Everything else is rejected or left for review.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import os
+from dataclasses import asdict, dataclass
+from pathlib import Path
+from typing import Iterable
+
+from huggingface_hub import snapshot_download
+
+
+ACCEPTED_REGLAB_REPO = "reglab/aquaculture_detection"
+MIN_IMAGE_BYTES = 10_000
+
+
+@dataclass
+class ImportedSample:
+    sample_id: str
+    element: str
+    task_type: str
+    image_path: str
+    mask_path: str | None
+    annotation_path: str | None
+    annotation_format: str | None
+    label_encoding: dict[str, str]
+    satellite: str | None
+    sensor: str | None
+    resolution_m: float | None
+    patch_size: int | None
+    bands: list[str] | None
+    band_count: int | None
+    dtype: str | None
+    fusion: dict
+    acquired_at: str | None
+    source_project: str
+    source_dataset: str
+    split: str | None
+    license: str | None
+    quality_flags: list[str]
+    quality_score: float
+    notes: str
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--discovery-root", required=True)
+    parser.add_argument("--output-root", required=True)
+    parser.add_argument("--hf-cache-dir", default=None)
+    parser.add_argument(
+        "--discard-unaccepted",
+        action="store_true",
+        help="Write only accepted samples and aggregate discard counts; do not keep rejected/review sample manifests.",
+    )
+    return parser.parse_args()
+
+
+def read_jsonl(path: Path) -> Iterable[dict]:
+    with path.open(encoding="utf-8") as f:
+        for line in f:
+            if line.strip():
+                yield json.loads(line)
+
+
+def write_jsonl(path: Path, rows: Iterable[dict]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    with path.open("w", encoding="utf-8") as f:
+        for row in rows:
+            f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def image_label_stem(path: str) -> str:
+    return Path(path).stem
+
+
+def is_valid_yolo_label(local_path: Path) -> tuple[bool, int, str]:
+    rows = 0
+    try:
+        text = local_path.read_text(encoding="utf-8").strip()
+    except UnicodeDecodeError:
+        text = local_path.read_text(encoding="latin-1").strip()
+    if not text:
+        return False, 0, "empty_label"
+    for line in text.splitlines():
+        parts = line.split()
+        if len(parts) != 5:
+            return False, rows, "invalid_yolo_column_count"
+        try:
+            cls = int(float(parts[0]))
+            x, y, w, h = [float(value) for value in parts[1:]]
+        except ValueError:
+            return False, rows, "non_numeric_yolo_value"
+        if cls < 0:
+            return False, rows, "negative_class_id"
+        if not all(0.0 <= value <= 1.0 for value in (x, y, w, h)):
+            return False, rows, "bbox_value_out_of_range"
+        if w <= 0.0 or h <= 0.0:
+            return False, rows, "non_positive_bbox"
+        rows += 1
+    return rows > 0, rows, "ok"
+
+
+def reject(asset: dict, reason: str) -> dict:
+    return {
+        "repo_id": asset.get("repo_id"),
+        "path": asset.get("path"),
+        "role": asset.get("role"),
+        "element": asset.get("element"),
+        "reason": reason,
+    }
+
+
+def review(asset: dict, reason: str) -> dict:
+    row = reject(asset, reason)
+    row["hf_path"] = asset.get("hf_path")
+    row["license"] = asset.get("license")
+    return row
+
+
+def import_reglab_aquaculture(assets: list[dict], cache_dir: str | None) -> tuple[list[ImportedSample], list[dict], list[dict]]:
+    repo_assets = [asset for asset in assets if asset["repo_id"] == ACCEPTED_REGLAB_REPO]
+    images = {image_label_stem(asset["path"]): asset for asset in repo_assets if asset["role"] == "image"}
+    labels = {image_label_stem(asset["path"]): asset for asset in repo_assets if asset["role"] == "annotation_table"}
+
+    accepted: list[ImportedSample] = []
+    rejected: list[dict] = []
+    review_rows: list[dict] = []
+
+    try:
+        repo_snapshot = Path(
+            snapshot_download(
+                repo_id=ACCEPTED_REGLAB_REPO,
+                repo_type="dataset",
+                allow_patterns=["labels/*.txt"],
+                token=os.environ.get("HF_TOKEN"),
+                cache_dir=cache_dir,
+                max_workers=8,
+            )
+        )
+    except Exception as exc:  # noqa: BLE001 - keep the whole import from blocking on HF.
+        for asset in repo_assets:
+            review_rows.append(review(asset, f"adapter_label_download_failed:{exc}"))
+        return accepted, rejected, review_rows
+
+    for stem, image in sorted(images.items()):
+        label = labels.get(stem)
+        if not label:
+            review_rows.append(review(image, "missing_matching_yolo_label"))
+            continue
+        if (image.get("size_bytes") or 0) < MIN_IMAGE_BYTES:
+            rejected.append(reject(image, "image_file_too_small"))
+            continue
+        if image.get("element") != "aquaculture":
+            rejected.append(reject(image, "unexpected_element"))
+            continue
+        if image.get("license") != "cc-by-nc-2.0":
+            rejected.append(reject(image, "license_not_recorded_as_expected"))
+            continue
+
+        local_label = repo_snapshot / label["path"]
+        if not local_label.exists():
+            review_rows.append(review(label, "label_not_available_in_snapshot"))
+            continue
+        ok, bbox_count, label_reason = is_valid_yolo_label(local_label)
+        if not ok:
+            rejected.append(reject(label, label_reason))
+            continue
+
+        accepted.append(
+            ImportedSample(
+                sample_id=f"hf_reglab_aquaculture_{stem.lower()}",
+                element="aquaculture",
+                task_type="object_detection",
+                image_path=image["hf_path"],
+                mask_path=None,
+                annotation_path=label["hf_path"],
+                annotation_format="yolo_bbox_txt",
+                label_encoding={"0": "aquaculture"},
+                satellite=None,
+                sensor=None,
+                resolution_m=None,
+                patch_size=1024,
+                bands=["red", "green", "blue"],
+                band_count=3,
+                dtype=None,
+                fusion={
+                    "state": "none",
+                    "method": "none",
+                    "sources": [{"role": "orthophoto_tile", "path": image["hf_path"], "resolution_m": None}],
+                    "target_resolution_m": None,
+                    "native_multispectral_resolution_m": None,
+                    "persisted": False,
+                    "reproducible": True,
+                    "spectral_preservation": "not_applicable_rgb_orthophoto",
+                    "notes": "RGB orthophoto tile from HF aquaculture detection dataset.",
+                },
+                acquired_at=None,
+                source_project="aquaculture_detection",
+                source_dataset=ACCEPTED_REGLAB_REPO,
+                split=None,
+                license=image.get("license"),
+                quality_flags=[
+                    "accepted",
+                    "explicit_adapter:reglab_aquaculture_yolo",
+                    "paired_image_label",
+                    "valid_yolo_bbox",
+                    f"bbox_count:{bbox_count}",
+                    "manual_training_gate_required_before_download",
+                ],
+                quality_score=0.92,
+                notes="Accepted by strict adapter: paired image and YOLO label, valid bbox rows, known aquaculture semantics.",
+            )
+        )
+
+    for stem, label in sorted(labels.items()):
+        if stem not in images:
+            review_rows.append(review(label, "label_without_matching_image"))
+
+    return accepted, rejected, review_rows
+
+
+def main() -> None:
+    args = parse_args()
+    discovery_root = Path(args.discovery_root)
+    output_root = Path(args.output_root)
+    manifest_dir = output_root / "manifests"
+    report_dir = output_root / "reports"
+    manifest_dir.mkdir(parents=True, exist_ok=True)
+    report_dir.mkdir(parents=True, exist_ok=True)
+
+    assets = list(read_jsonl(discovery_root / "manifests" / "hf_assets_raw.jsonl"))
+    accepted, rejected, review_rows = import_reglab_aquaculture(assets, args.hf_cache_dir)
+
+    accepted_repo_paths = {(sample.source_dataset, sample.image_path) for sample in accepted}
+    explicitly_seen = {ACCEPTED_REGLAB_REPO}
+    for asset in assets:
+        if asset["repo_id"] in explicitly_seen:
+            continue
+        if asset["role"] in {"image", "mask", "annotation_table", "archive"}:
+            review_rows.append(review(asset, "no_quality_adapter_or_unverified_semantics"))
+
+    write_jsonl(manifest_dir / "accepted_samples.jsonl", (asdict(sample) for sample in accepted))
+    rejected_path = manifest_dir / "rejected_samples.jsonl"
+    review_path = manifest_dir / "needs_manual_review.jsonl"
+    if args.discard_unaccepted:
+        rejected_path.unlink(missing_ok=True)
+        review_path.unlink(missing_ok=True)
+    else:
+        write_jsonl(rejected_path, rejected)
+        write_jsonl(review_path, review_rows)
+
+    with (report_dir / "quality_report.csv").open("w", newline="", encoding="utf-8-sig") as f:
+        writer = csv.DictWriter(
+            f,
+            fieldnames=[
+                "sample_id",
+                "element",
+                "task_type",
+                "source_dataset",
+                "image_path",
+                "annotation_path",
+                "license",
+                "quality_score",
+                "quality_flags",
+            ],
+        )
+        writer.writeheader()
+        for sample in accepted:
+            row = asdict(sample)
+            row["quality_flags"] = ";".join(sample.quality_flags)
+            writer.writerow({key: row.get(key) for key in writer.fieldnames})
+
+    summary = {
+        "accepted_samples": len(accepted),
+        "rejected_assets": len(rejected),
+        "needs_manual_review": len(review_rows),
+        "unaccepted_sample_details_retained": not args.discard_unaccepted,
+        "accepted_by_element": {},
+        "accepted_by_task_type": {},
+        "quality_policy": "Only explicit adapters with verified labels can enter accepted_samples.jsonl.",
+    }
+    for sample in accepted:
+        summary["accepted_by_element"][sample.element] = summary["accepted_by_element"].get(sample.element, 0) + 1
+        summary["accepted_by_task_type"][sample.task_type] = summary["accepted_by_task_type"].get(sample.task_type, 0) + 1
+    (report_dir / "quality_summary.json").write_text(json.dumps(summary, indent=2, ensure_ascii=False), encoding="utf-8")
+    if args.discard_unaccepted:
+        discard_summary = {
+            "discarded_rejected_assets": len(rejected),
+            "discarded_review_assets": len(review_rows),
+            "reason": "User requested all non-accepted samples be discarded.",
+            "retained_manifest": "manifests/accepted_samples.jsonl",
+        }
+        (report_dir / "discard_summary.json").write_text(
+            json.dumps(discard_summary, indent=2, ensure_ascii=False), encoding="utf-8"
+        )
+
+    print(json.dumps(summary, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/infer_whole_scene.py

@@ -0,0 +1,356 @@
+"""Sliding-window inference for full-scene seaweed segmentation rasters."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+from pathlib import Path
+
+import numpy as np
+import rasterio
+import torch
+from rasterio.windows import Window
+from torchvision import transforms
+from tqdm import tqdm
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
+
+from dinov3_deeplabv3plus import DinoV3DeepLabV3Plus
+
+
+NORMALIZE_3CH = transforms.Normalize(mean=(0.430, 0.411, 0.296), std=(0.213, 0.156, 0.143))
+NORMALIZE_4CH = transforms.Normalize(mean=(0.430, 0.411, 0.296, 0.350), std=(0.213, 0.156, 0.143, 0.180))
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--image", required=True, help="Input multispectral whole-scene raster path.")
+    parser.add_argument("--pan", default=None, help="Optional panchromatic raster path for on-the-fly pan sharpening.")
+    parser.add_argument("--checkpoint", required=True, help="Model checkpoint .pth path.")
+    parser.add_argument("--output-dir", default="outputs/scene_inference", help="Directory for prediction rasters.")
+    parser.add_argument("--tile-size", type=int, default=256, help="Inference tile size.")
+    parser.add_argument("--overlap", type=int, default=32, help="Tile overlap in output pixels for weighted blending.")
+    parser.add_argument("--stripe-height", type=int, default=1024, help="Rows to blend/write at a time.")
+    parser.add_argument("--batch-size", type=int, default=4, help="Number of tiles per forward pass.")
+    parser.add_argument("--threshold", type=float, default=0.5, help="Foreground probability threshold.")
+    parser.add_argument("--max-tiles", type=int, default=0, help="Optional smoke-test tile limit; 0 means full scene.")
+    parser.add_argument("--max-stripes", type=int, default=0, help="Optional smoke-test stripe limit; 0 means all stripes.")
+    parser.add_argument("--device", default="cuda", choices=["cuda", "cpu"], help="Inference device.")
+    parser.add_argument("--write-probability", action="store_true", help="Write the foreground probability raster.")
+    parser.add_argument("--no-probability", action="store_true", help="Deprecated; probability output is disabled by default.")
+    return parser.parse_args()
+
+
+def get_checkpoint_state(path: Path, device: torch.device) -> dict:
+    checkpoint = torch.load(path, map_location=device, weights_only=False)
+    if not isinstance(checkpoint, dict):
+        raise ValueError(f"Unsupported checkpoint format: {path}")
+    return checkpoint
+
+
+def build_model(checkpoint: dict, device: torch.device) -> DinoV3DeepLabV3Plus:
+    config = checkpoint.get("config") or {}
+    model = DinoV3DeepLabV3Plus(
+        num_classes=int(config.get("num_classes", 2)),
+        backbone_name=config.get("backbone_name", "dinov3_vitl16"),
+        pretrained=False,
+        weights=config.get("backbone_weights", "SAT493M"),
+        use_4channel=bool(config.get("use_4channel", True)),
+        freeze_backbone=False,
+    ).to(device)
+
+    state_dict = checkpoint.get("model_state_dict") or checkpoint.get("state_dict")
+    if state_dict is None:
+        raise KeyError("Checkpoint does not contain model_state_dict/state_dict.")
+    cleaned = {k.removeprefix("module."): v for k, v in state_dict.items()}
+    model.load_state_dict(cleaned, strict=True)
+    model.eval()
+    return model
+
+
+def axis_starts(length: int, tile_size: int, overlap: int) -> list[int]:
+    if length <= tile_size:
+        return [0]
+    stride = tile_size - overlap
+    if stride <= 0:
+        raise ValueError("--overlap must be smaller than --tile-size.")
+    starts = list(range(0, length - tile_size + 1, stride))
+    last = length - tile_size
+    if starts[-1] != last:
+        starts.append(last)
+    return starts
+
+
+def tile_grid(width: int, height: int, tile_size: int, overlap: int) -> list[tuple[int, int, int, int]]:
+    tiles: list[tuple[int, int, int, int]] = []
+    for y in axis_starts(height, tile_size, overlap):
+        for x in axis_starts(width, tile_size, overlap):
+            w = min(tile_size, width - x)
+            h = min(tile_size, height - y)
+            tiles.append((x, y, w, h))
+    return tiles
+
+
+def blend_weight(tile_size: int, overlap: int) -> np.ndarray:
+    if overlap <= 0:
+        return np.ones((tile_size, tile_size), dtype=np.float32)
+    ramp = np.minimum(np.arange(tile_size, dtype=np.float32) + 1, tile_size - np.arange(tile_size, dtype=np.float32))
+    ramp = np.clip(ramp / float(overlap), 1.0 / float(overlap), 1.0)
+    return np.minimum(ramp[:, None], ramp[None, :]).astype(np.float32, copy=False)
+
+
+def pad_chw(tile: np.ndarray, tile_size: int) -> np.ndarray:
+    bands, height, width = tile.shape
+    padded = np.zeros((bands, tile_size, tile_size), dtype=np.float32)
+    padded[:, :height, :width] = tile.astype(np.float32, copy=False)
+    return padded
+
+
+def match_pan_to_intensity(pan: np.ndarray, intensity: np.ndarray) -> np.ndarray:
+    pan = pan.astype(np.float32, copy=False)
+    intensity = intensity.astype(np.float32, copy=False)
+    pan_std = float(np.std(pan))
+    intensity_std = float(np.std(intensity))
+    if pan_std < 1e-6 or intensity_std < 1e-6:
+        return pan
+    return (pan - float(np.mean(pan))) * (intensity_std / pan_std) + float(np.mean(intensity))
+
+
+def pan_sharpen_tile(ms_tile: np.ndarray, pan_tile: np.ndarray, tile_size: int) -> np.ndarray:
+    """Lightweight additive component substitution for one tile.
+
+    The result keeps the multispectral band count and PAN spatial resolution.
+    It is intended for streaming inference, not radiometric product generation.
+    """
+    ms_padded = pad_chw(ms_tile, tile_size)
+    pan_padded = pad_chw(pan_tile[:1], tile_size)[0]
+    bands = ms_padded[:4] if ms_padded.shape[0] >= 4 else ms_padded
+    intensity = np.mean(bands, axis=0)
+    matched_pan = match_pan_to_intensity(pan_padded, intensity)
+    fused = bands + (matched_pan - intensity)[None, :, :]
+    return np.clip(fused, 0, 65535).astype(np.float32, copy=False)
+
+
+def to_model_tensor(tile: np.ndarray, tile_size: int, use_4channel: bool) -> torch.Tensor:
+    # rasterio returns C,H,W. Pad edge tiles to the training tile size.
+    padded = pad_chw(tile, tile_size)
+
+    if use_4channel:
+        if padded.shape[0] < 4:
+            padded = np.pad(padded, ((0, 4 - padded.shape[0]), (0, 0), (0, 0)), mode="edge")
+        data = padded[:4]
+        normalizer = NORMALIZE_4CH
+    else:
+        if padded.shape[0] >= 4:
+            data = padded[[3, 2, 1]]
+        else:
+            data = padded[: min(3, padded.shape[0])]
+            while data.shape[0] < 3:
+                data = np.concatenate([data, data[-1:]], axis=0)
+        normalizer = NORMALIZE_3CH
+
+    tensor = torch.from_numpy(data)
+    if float(tensor.max()) > 1.0:
+        tensor = tensor / 65535.0
+    return normalizer(tensor)
+
+
+def read_fused_tile(
+    ms_src: rasterio.DatasetReader,
+    pan_src: rasterio.DatasetReader,
+    x: int,
+    y: int,
+    w: int,
+    h: int,
+    tile_size: int,
+) -> np.ndarray:
+    scale_x = pan_src.width / ms_src.width
+    scale_y = pan_src.height / ms_src.height
+    ms_window = Window(x / scale_x, y / scale_y, w / scale_x, h / scale_y)
+    ms_tile = ms_src.read(
+        window=ms_window,
+        out_shape=(ms_src.count, h, w),
+        resampling=rasterio.enums.Resampling.bilinear,
+        boundless=True,
+        fill_value=0,
+    )
+    pan_tile = pan_src.read(1, window=Window(x, y, w, h), boundless=True, fill_value=0)[None, :, :]
+    return pan_sharpen_tile(ms_tile, pan_tile, tile_size)
+
+
+def run_batch(model: torch.nn.Module, batch: list[torch.Tensor], device: torch.device) -> np.ndarray:
+    inputs = torch.stack(batch, dim=0).to(device, non_blocking=True)
+    with torch.inference_mode():
+        with torch.autocast(device_type="cuda", enabled=device.type == "cuda"):
+            output = model(inputs)
+            logits = output["out"] if isinstance(output, dict) else output
+            probs = torch.softmax(logits.float(), dim=1)[:, 1]
+    return probs.detach().cpu().numpy()
+
+
+def add_probs_to_stripe(
+    probs: np.ndarray,
+    windows: list[tuple[int, int, int, int]],
+    stripe_y: int,
+    stripe_prob_sum: np.ndarray,
+    stripe_weight_sum: np.ndarray,
+    weight: np.ndarray,
+) -> None:
+    for prob, (wx, wy, ww, wh) in zip(probs, windows):
+        out_y0 = max(wy, stripe_y)
+        out_y1 = min(wy + wh, stripe_y + stripe_prob_sum.shape[0])
+        if out_y0 >= out_y1:
+            continue
+        prob_y0 = out_y0 - wy
+        prob_y1 = out_y1 - wy
+        stripe_local_y0 = out_y0 - stripe_y
+        stripe_local_y1 = out_y1 - stripe_y
+
+        cropped = prob[prob_y0:prob_y1, :ww]
+        cropped_weight = weight[prob_y0:prob_y1, :ww]
+        stripe_prob_sum[stripe_local_y0:stripe_local_y1, wx : wx + ww] += (
+            cropped.astype(np.float32, copy=False) * cropped_weight
+        )
+        stripe_weight_sum[stripe_local_y0:stripe_local_y1, wx : wx + ww] += cropped_weight
+
+
+def write_stripe(
+    mask_dst: rasterio.DatasetWriter,
+    prob_dst: rasterio.DatasetWriter | None,
+    stripe_y: int,
+    stripe_prob_sum: np.ndarray,
+    stripe_weight_sum: np.ndarray,
+    threshold: float,
+) -> None:
+    probs = np.divide(
+        stripe_prob_sum,
+        stripe_weight_sum,
+        out=np.zeros_like(stripe_prob_sum, dtype=np.float32),
+        where=stripe_weight_sum > 0,
+    )
+    mask = (probs >= threshold).astype(np.uint8) * 255
+    window = Window(0, stripe_y, probs.shape[1], probs.shape[0])
+    mask_dst.write(mask, 1, window=window)
+    if prob_dst is not None:
+        prob_dst.write(probs.astype(np.float32, copy=False), 1, window=window)
+
+
+def main() -> None:
+    args = parse_args()
+    write_probability = bool(args.write_probability) and not bool(args.no_probability)
+    image_path = Path(args.image)
+    pan_path = Path(args.pan) if args.pan else None
+    checkpoint_path = Path(args.checkpoint)
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    if args.device == "cuda" and not torch.cuda.is_available():
+        print("CUDA is not available; falling back to CPU.")
+        device = torch.device("cpu")
+    else:
+        device = torch.device(args.device)
+
+    checkpoint = get_checkpoint_state(checkpoint_path, device)
+    config = checkpoint.get("config") or {}
+    use_4channel = bool(config.get("use_4channel", True))
+    model = build_model(checkpoint, device)
+
+    start = time.time()
+    with rasterio.open(image_path) as ms_src:
+        if use_4channel and ms_src.count < 4:
+            raise ValueError(f"Model expects 4 channels, but image has {ms_src.count}: {image_path}")
+
+        pan_src = rasterio.open(pan_path) if pan_path else None
+        ref_src = pan_src or ms_src
+        all_tiles = tile_grid(ref_src.width, ref_src.height, args.tile_size, args.overlap)
+        tiles = all_tiles[: args.max_tiles] if args.max_tiles > 0 else all_tiles
+        stem = image_path.stem
+        if pan_src is not None:
+            stem = f"{stem}_pansharpened"
+        suffix = "smoke" if args.max_tiles > 0 else "full"
+        mask_path = output_dir / f"{stem}_{suffix}_mask.tif"
+        prob_path = output_dir / f"{stem}_{suffix}_prob.tif"
+
+        profile = ref_src.profile.copy()
+        mask_profile = profile.copy()
+        mask_profile.update(count=1, dtype="uint8", compress="lzw", nodata=0)
+        prob_profile = profile.copy()
+        prob_profile.update(count=1, dtype="float32", compress="lzw", nodata=0.0)
+        weight = blend_weight(args.tile_size, args.overlap)
+
+        try:
+            mode = "Pan-sharpen inference" if pan_src is not None else "Inference"
+            prob_dst = None
+            with rasterio.open(mask_path, "w", **mask_profile) as mask_dst:
+                if write_probability:
+                    prob_dst = rasterio.open(prob_path, "w", **prob_profile)
+                try:
+                    stripe_starts = list(range(0, ref_src.height, args.stripe_height))
+                    if args.max_stripes > 0:
+                        stripe_starts = stripe_starts[: args.max_stripes]
+                    for stripe_y in tqdm(
+                        stripe_starts,
+                        desc=f"{mode} stripes {image_path.name}",
+                        unit="stripe",
+                    ):
+                        stripe_h = min(args.stripe_height, ref_src.height - stripe_y)
+                        stripe_prob_sum = np.zeros((stripe_h, ref_src.width), dtype=np.float32)
+                        stripe_weight_sum = np.zeros((stripe_h, ref_src.width), dtype=np.float32)
+                        stripe_tiles = [
+                            tile
+                            for tile in tiles
+                            if tile[1] < stripe_y + stripe_h and tile[1] + tile[3] > stripe_y
+                        ]
+
+                        batch: list[torch.Tensor] = []
+                        windows: list[tuple[int, int, int, int]] = []
+                        for x, y, w, h in stripe_tiles:
+                            if pan_src is None:
+                                tile = ms_src.read(window=Window(x, y, w, h))
+                            else:
+                                tile = read_fused_tile(ms_src, pan_src, x, y, w, h, args.tile_size)
+                            batch.append(to_model_tensor(tile, args.tile_size, use_4channel))
+                            windows.append((x, y, w, h))
+
+                            if len(batch) == args.batch_size:
+                                probs = run_batch(model, batch, device)
+                                add_probs_to_stripe(probs, windows, stripe_y, stripe_prob_sum, stripe_weight_sum, weight)
+                                batch.clear()
+                                windows.clear()
+
+                        if batch:
+                            probs = run_batch(model, batch, device)
+                            add_probs_to_stripe(probs, windows, stripe_y, stripe_prob_sum, stripe_weight_sum, weight)
+
+                        write_stripe(mask_dst, prob_dst, stripe_y, stripe_prob_sum, stripe_weight_sum, args.threshold)
+                finally:
+                    if prob_dst is not None:
+                        prob_dst.close()
+        finally:
+            if pan_src is not None:
+                pan_src.close()
+
+    summary = {
+        "image": str(image_path),
+        "pan": str(pan_path) if pan_path else None,
+        "checkpoint": str(checkpoint_path),
+        "device": str(device),
+        "tile_size": args.tile_size,
+        "overlap": args.overlap,
+        "batch_size": args.batch_size,
+        "tiles_processed": len(tiles),
+        "tiles_total": len(all_tiles),
+        "max_stripes": args.max_stripes,
+        "mask": str(mask_path),
+        "probability": str(prob_path) if write_probability else None,
+        "seconds": round(time.time() - start, 2),
+        "checkpoint_epoch": checkpoint.get("epoch"),
+        "checkpoint_best_val_iou": checkpoint.get("best_val_iou"),
+    }
+    print(json.dumps(summary, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/launch_ddp_no_libuv.py

@@ -0,0 +1,48 @@
+"""Windows-friendly two-process DDP launcher for PyTorch builds without libuv."""
+
+from __future__ import annotations
+
+import argparse
+import os
+import subprocess
+import sys
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--nproc_per_node", type=int, default=2)
+    parser.add_argument("--master_addr", default="127.0.0.1")
+    parser.add_argument("--master_port", default="29500")
+    parser.add_argument("training_script")
+    parser.add_argument("training_args", nargs=argparse.REMAINDER)
+    args = parser.parse_args()
+
+    processes = []
+    for local_rank in range(args.nproc_per_node):
+        env = os.environ.copy()
+        env.update(
+            {
+                "MASTER_ADDR": args.master_addr,
+                "MASTER_PORT": args.master_port,
+                "WORLD_SIZE": str(args.nproc_per_node),
+                "RANK": str(local_rank),
+                "LOCAL_RANK": str(local_rank),
+                "USE_LIBUV": "0",
+            }
+        )
+        cmd = [sys.executable, args.training_script, *args.training_args]
+        processes.append(subprocess.Popen(cmd, env=env))
+
+    exit_code = 0
+    for process in processes:
+        exit_code = max(exit_code, process.wait())
+
+    if exit_code:
+        for process in processes:
+            if process.poll() is None:
+                process.terminate()
+    raise SystemExit(exit_code)
+
+
+if __name__ == "__main__":
+    main()

scripts/materialize_yolo_from_accepted.py

@@ -0,0 +1,158 @@
+"""Materialize accepted HF object-detection samples as a YOLO dataset."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import random
+import shutil
+from pathlib import Path
+
+from huggingface_hub import snapshot_download
+from PIL import Image
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--accepted-manifest", required=True)
+    parser.add_argument("--output-root", required=True)
+    parser.add_argument("--seed", type=int, default=20260704)
+    parser.add_argument("--train-ratio", type=float, default=0.8)
+    parser.add_argument("--val-ratio", type=float, default=0.1)
+    return parser.parse_args()
+
+
+def read_jsonl(path: Path) -> list[dict]:
+    rows: list[dict] = []
+    with path.open(encoding="utf-8") as f:
+        for line in f:
+            if line.strip():
+                rows.append(json.loads(line))
+    return rows
+
+
+def parse_hf_path(hf_path: str) -> tuple[str, str]:
+    prefix = "hf://datasets/"
+    if not hf_path.startswith(prefix):
+        raise ValueError(f"Unsupported path: {hf_path}")
+    rest = hf_path[len(prefix) :]
+    owner, name, rel = rest.split("/", 2)
+    return f"{owner}/{name}", rel
+
+
+def split_rows(rows: list[dict], seed: int, train_ratio: float, val_ratio: float) -> dict[str, list[dict]]:
+    rng = random.Random(seed)
+    shuffled = list(rows)
+    rng.shuffle(shuffled)
+    train_end = int(len(shuffled) * train_ratio)
+    val_end = train_end + int(len(shuffled) * val_ratio)
+    return {
+        "train": shuffled[:train_end],
+        "val": shuffled[train_end:val_end],
+        "test": shuffled[val_end:],
+    }
+
+
+def copy_sample(row: dict, split: str, output_root: Path, snapshots: dict[str, Path]) -> tuple[int, int]:
+    image_repo, image_rel = parse_hf_path(row["image_path"])
+    label_repo, label_rel = parse_hf_path(row["annotation_path"])
+    if image_repo != label_repo:
+        raise ValueError(f"Image and label repos differ: {image_repo} vs {label_repo}")
+
+    snapshot = snapshots[image_repo]
+    image_src = snapshot / image_rel
+    label_src = snapshot / label_rel
+    if not image_src.exists() or not label_src.exists():
+        raise FileNotFoundError(f"Missing snapshot files: {image_rel}, {label_rel}")
+
+    image_dst = output_root / "images" / split / f"{row['sample_id']}{image_src.suffix.lower()}"
+    label_dst = output_root / "labels" / split / f"{row['sample_id']}.txt"
+    image_dst.parent.mkdir(parents=True, exist_ok=True)
+    label_dst.parent.mkdir(parents=True, exist_ok=True)
+
+    shutil.copy2(image_src, image_dst)
+    remap_yolo_label_to_single_class(label_src, label_dst)
+
+    with Image.open(image_dst) as img:
+        width, height = img.size
+        if width <= 0 or height <= 0:
+            raise ValueError(f"Invalid image dimensions: {image_dst}")
+    text = label_dst.read_text(encoding="utf-8").strip()
+    if not text:
+        raise ValueError(f"Empty label: {label_dst}")
+    return width, height
+
+
+def remap_yolo_label_to_single_class(src: Path, dst: Path) -> None:
+    """Collapse source dataset subclasses into class 0 for the element card."""
+    lines: list[str] = []
+    for raw in src.read_text(encoding="utf-8").splitlines():
+        if not raw.strip():
+            continue
+        parts = raw.split()
+        if len(parts) != 5:
+            raise ValueError(f"Invalid YOLO row in {src}: {raw}")
+        parts[0] = "0"
+        lines.append(" ".join(parts))
+    if not lines:
+        raise ValueError(f"Empty YOLO label after remap: {src}")
+    dst.write_text("\n".join(lines) + "\n", encoding="utf-8")
+
+
+def main() -> None:
+    args = parse_args()
+    accepted_manifest = Path(args.accepted_manifest)
+    output_root = Path(args.output_root)
+    if output_root.exists():
+        shutil.rmtree(output_root)
+    output_root.mkdir(parents=True)
+
+    rows = read_jsonl(accepted_manifest)
+    repos = sorted({parse_hf_path(row["image_path"])[0] for row in rows})
+    snapshots = {
+        repo: Path(
+            snapshot_download(
+                repo_id=repo,
+                repo_type="dataset",
+                allow_patterns=["images/*", "labels/*"],
+                max_workers=8,
+            )
+        )
+        for repo in repos
+    }
+    splits = split_rows(rows, args.seed, args.train_ratio, args.val_ratio)
+    stats = {
+        "source_manifest": str(accepted_manifest),
+        "total": len(rows),
+        "splits": {split: len(items) for split, items in splits.items()},
+        "classes": ["aquaculture"],
+        "image_sizes": {},
+    }
+
+    for split, items in splits.items():
+        for row in items:
+            width, height = copy_sample(row, split, output_root, snapshots)
+            key = f"{width}x{height}"
+            stats["image_sizes"][key] = stats["image_sizes"].get(key, 0) + 1
+
+    data_yaml = output_root / "data.yaml"
+    data_yaml.write_text(
+        "\n".join(
+            [
+                f"path: {output_root.as_posix()}",
+                "train: images/train",
+                "val: images/val",
+                "test: images/test",
+                "names:",
+                "  0: aquaculture",
+                "",
+            ]
+        ),
+        encoding="utf-8",
+    )
+    (output_root / "dataset_summary.json").write_text(json.dumps(stats, indent=2, ensure_ascii=False), encoding="utf-8")
+    print(json.dumps(stats, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/postprocess_mask.py

@@ -0,0 +1,105 @@
+"""Post-process seaweed masks with no-data and conservative water constraints."""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+import numpy as np
+import rasterio
+from rasterio.windows import Window
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--image", required=True, help="Source 4-band fused/multispectral raster.")
+    parser.add_argument("--mask", required=True, help="Predicted binary mask raster.")
+    parser.add_argument("--output", required=True, help="Filtered output mask raster.")
+    parser.add_argument("--stripe-height", type=int, default=1024)
+    parser.add_argument("--black-threshold", type=float, default=32.0, help="Pixels with all bands <= this are no-data.")
+    parser.add_argument(
+        "--land-ndwi-threshold",
+        type=float,
+        default=-0.35,
+        help="Very conservative NDWI cutoff for obvious dry land. Lower is safer for floating algae.",
+    )
+    parser.add_argument(
+        "--land-nir-ratio",
+        type=float,
+        default=1.6,
+        help="Only suppress NDWI-low pixels when NIR is this many times brighter than green.",
+    )
+    parser.add_argument("--no-land-filter", action="store_true", help="Only remove black/no-data regions.")
+    return parser.parse_args()
+
+
+def obvious_land_mask(tile: np.ndarray, ndwi_threshold: float, nir_ratio: float) -> np.ndarray:
+    """Return a conservative land mask from B,G,R,NIR-like 4-band data.
+
+    This is not a substitute for an official coastline/water mask. It only removes
+    strongly land-like pixels to avoid deleting real floating algae.
+    """
+    if tile.shape[0] < 4:
+        return np.zeros(tile.shape[1:], dtype=bool)
+    green = tile[1].astype(np.float32, copy=False)
+    nir = tile[3].astype(np.float32, copy=False)
+    ndwi = (green - nir) / (green + nir + 1e-6)
+    return (ndwi < ndwi_threshold) & (nir > green * nir_ratio)
+
+
+def main() -> None:
+    args = parse_args()
+    image_path = Path(args.image)
+    mask_path = Path(args.mask)
+    output_path = Path(args.output)
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    with rasterio.open(image_path) as image_src, rasterio.open(mask_path) as mask_src:
+        if (image_src.width, image_src.height) != (mask_src.width, mask_src.height):
+            raise ValueError(
+                f"Image and mask sizes differ: image={image_src.width}x{image_src.height}, "
+                f"mask={mask_src.width}x{mask_src.height}"
+            )
+        profile = mask_src.profile.copy()
+        profile.update(count=1, dtype="uint8", compress="lzw", nodata=0)
+
+        total_pixels = image_src.width * image_src.height
+        input_fg = 0
+        output_fg = 0
+        invalid_pixels = 0
+        land_pixels = 0
+
+        with rasterio.open(output_path, "w", **profile) as dst:
+            for y in range(0, image_src.height, args.stripe_height):
+                height = min(args.stripe_height, image_src.height - y)
+                window = Window(0, y, image_src.width, height)
+                image = image_src.read(window=window)
+                mask = mask_src.read(1, window=window)
+
+                predicted = mask > 0
+                valid = np.max(image, axis=0) > args.black_threshold
+                land = np.zeros(valid.shape, dtype=bool)
+                if not args.no_land_filter:
+                    land = obvious_land_mask(image, args.land_ndwi_threshold, args.land_nir_ratio)
+
+                filtered = predicted & valid & ~land
+                dst.write((filtered.astype(np.uint8) * 255), 1, window=window)
+
+                input_fg += int(predicted.sum())
+                output_fg += int(filtered.sum())
+                invalid_pixels += int((~valid).sum())
+                land_pixels += int(land.sum())
+
+    print(f"image={image_path}")
+    print(f"mask={mask_path}")
+    print(f"output={output_path}")
+    print(f"total_pixels={total_pixels}")
+    print(f"input_foreground={input_fg} ratio={input_fg / total_pixels:.6f}")
+    print(f"output_foreground={output_fg} ratio={output_fg / total_pixels:.6f}")
+    print(f"removed_foreground={input_fg - output_fg} ratio={(input_fg - output_fg) / total_pixels:.6f}")
+    print(f"invalid_or_black_pixels={invalid_pixels} ratio={invalid_pixels / total_pixels:.6f}")
+    print(f"conservative_land_pixels={land_pixels} ratio={land_pixels / total_pixels:.6f}")
+
+
+if __name__ == "__main__":
+    main()

scripts/run_train_2gpu.bat

@@ -0,0 +1,8 @@
+@echo off
+setlocal
+cd /d D:\hutai-2
+set CUDA_VISIBLE_DEVICES=0,1
+set PYTHONUTF8=1
+set USE_LIBUV=0
+set NO_ALBUMENTATIONS_UPDATE=1
+call D:\anaconda3\condabin\conda.bat run -n dinov3 python scripts\launch_ddp_no_libuv.py --nproc_per_node=2 --master_addr=127.0.0.1 --master_port=29502 train_seaweed_segmentation.py --config configs\train_2gpu_4090d.json --batch-size 2 --epochs 500

scripts/search_hf_marine_datasets.py

@@ -0,0 +1,499 @@
+"""Search Hugging Face for marine feature datasets and write normalized manifests.
+
+The script does not download full datasets. It inspects dataset repository
+metadata and file lists, then writes project-compatible manifests using hf://
+paths so large public datasets can be reviewed before any costly download.
+"""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import os
+import re
+from dataclasses import asdict, dataclass
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Iterable
+
+from huggingface_hub import HfApi
+
+
+IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".tif", ".tiff", ".jp2", ".bmp", ".webp"}
+METADATA_EXTS = {".csv", ".json", ".jsonl", ".parquet", ".txt"}
+ARCHIVE_EXTS = {".zip", ".tar", ".gz", ".tgz", ".7z"}
+TRACKED_EXTS = IMAGE_EXTS | METADATA_EXTS | ARCHIVE_EXTS
+MASK_HINTS = ("mask", "label", "labels", "gt", "annotation", "annotations", "seg", "target")
+TRAIN_SPLITS = ("train", "training", "val", "valid", "validation", "test")
+
+QUERIES = [
+    "seaweed",
+    "green tide",
+    "red tide",
+    "sargassum",
+    "aquaculture",
+    "ship",
+    "oil spill",
+    "sea ice",
+    "marine",
+    "ocean",
+    "remote sensing",
+    "sar ship",
+    "satellite imagery",
+]
+
+ELEMENT_KEYWORDS = {
+    "green_tide": ("green_tide", "greentide", "green tide", "enteromorpha", "seaweed"),
+    "red_tide": ("red_tide", "redtide", "red tide", "harmful algal", "hab"),
+    "golden_tide": ("golden_tide", "goldentide", "sargassum", "sarg"),
+    "aquaculture": ("aquaculture", "oyster", "raft", "fish cage", "fish-cage"),
+    "ship": ("ship", "sar ship"),
+    "oil_spill": ("oilspill", "oil_spill", "oil spill", "oil-spill"),
+    "sea_ice": ("seaice", "sea_ice", "sea ice", "arctic ice"),
+}
+
+SHIP_CONTEXT_KEYWORDS = (
+    "ship",
+    "sar",
+    "satellite",
+    "remote sensing",
+    "marine",
+    "ocean",
+    "sentinel",
+    "ais",
+)
+
+EXCLUDE_KEYWORDS = (
+    "medical",
+    "retinal",
+    "retina",
+    "miccai",
+    "flare",
+    "drive digital retinal",
+    "godot",
+    "shipping law",
+    "shipping orders",
+    "textual inversion",
+    "kantaicollection",
+    "waifu",
+    "anime",
+    "cancer",
+    "community health",
+    "plankton",
+    "mammal",
+    "legal",
+)
+
+SATELLITE_PATTERN = re.compile(r"\b(GF\d+|HY\d+|Sentinel-?1|Sentinel-?2|Landsat-?\d*|SAR)\b", re.IGNORECASE)
+PATCH_SIZE_PATTERN = re.compile(r"(?:^|[_/\-])(?:size)?(128|256|512|1024)(?:[_/\-]|$)")
+
+
+@dataclass
+class HfAssetRecord:
+    asset_id: str
+    repo_id: str
+    path: str
+    hf_path: str
+    filename: str
+    suffix: str
+    role: str
+    element: str
+    satellite: str | None
+    sensor: str | None
+    patch_size: int | None
+    source_project: str
+    source_dataset: str
+    size_bytes: int | None
+    downloads: int | None
+    likes: int | None
+    license: str | None
+    tags: list[str]
+    discovered_by: list[str]
+    quality_flags: list[str]
+
+
+@dataclass
+class HfSampleRecord:
+    sample_id: str
+    element: str
+    task_type: str
+    image_path: str
+    mask_path: str | None
+    label_encoding: dict[str, str] | None
+    satellite: str | None
+    sensor: str | None
+    resolution_m: float | None
+    patch_size: int | None
+    bands: list[str] | None
+    band_count: int | None
+    dtype: str | None
+    fusion: dict
+    acquired_at: str | None
+    source_project: str
+    source_dataset: str
+    split: str | None
+    quality_flags: list[str]
+    notes: str
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--output-root", required=True)
+    parser.add_argument("--author", default="cuibinge", help="HF namespace to always include.")
+    parser.add_argument("--limit-per-query", type=int, default=20)
+    parser.add_argument("--max-repos", type=int, default=80)
+    parser.add_argument("--max-files-per-repo", type=int, default=5000)
+    parser.add_argument("--include-low-confidence", action="store_true")
+    return parser.parse_args()
+
+
+def normalize_text(value: str) -> str:
+    return value.replace("_", " ").replace("-", " ").replace("/", " ").lower()
+
+
+def infer_element(*parts: str) -> str:
+    text = normalize_text(" ".join(part for part in parts if part))
+    if "vessel" in text and all(keyword not in text for keyword in SHIP_CONTEXT_KEYWORDS):
+        return "unknown"
+    for element, keywords in ELEMENT_KEYWORDS.items():
+        if any(keyword in text for keyword in keywords):
+            return element
+    if "vessel" in text and any(keyword in text for keyword in SHIP_CONTEXT_KEYWORDS):
+        return "ship"
+    return "unknown"
+
+
+def infer_role(path: str) -> str:
+    lower = path.lower()
+    stem = Path(path).stem.lower()
+    suffix = Path(path).suffix.lower()
+    if suffix in ARCHIVE_EXTS:
+        return "archive"
+    if suffix in METADATA_EXTS:
+        if any(hint in lower or hint in stem for hint in MASK_HINTS) or stem in {"train", "val", "test"}:
+            return "annotation_table"
+        return "metadata"
+    if any(hint in lower or hint in stem for hint in MASK_HINTS):
+        return "mask"
+    return "image"
+
+
+def infer_satellite(*parts: str) -> str | None:
+    match = SATELLITE_PATTERN.search(" ".join(parts))
+    return match.group(1).upper().replace("-", "") if match else None
+
+
+def infer_sensor(*parts: str) -> str | None:
+    upper = " ".join(parts).upper()
+    for sensor in ("PMS", "MUX", "MSS", "PAN", "WFV", "SAR", "MSI", "OLI"):
+        if sensor in upper:
+            return sensor
+    return None
+
+
+def infer_patch_size(path: str) -> int | None:
+    match = PATCH_SIZE_PATTERN.search(path)
+    return int(match.group(1)) if match else None
+
+
+def infer_split(path: str) -> str | None:
+    parts = {part.lower() for part in Path(path).parts}
+    for split in TRAIN_SPLITS:
+        if split in parts:
+            return "val" if split in {"valid", "validation"} else "train" if split == "training" else split
+    return None
+
+
+def infer_license(tags: list[str]) -> str | None:
+    for tag in tags:
+        if tag.startswith("license:"):
+            return tag.split(":", 1)[1]
+    return None
+
+
+def infer_fusion(repo_id: str, path: str, sensor: str | None) -> dict:
+    lower = f"{repo_id}/{path}".lower()
+    if "fuse" in lower or "fusion" in lower:
+        state = "fused_product"
+        method = "unknown_vendor_product"
+        persisted = True
+    elif sensor in {"SAR", "MSI", "OLI"}:
+        state = "none"
+        method = "none"
+        persisted = False
+    else:
+        state = "unknown"
+        method = "unknown"
+        persisted = False
+    return {
+        "state": state,
+        "method": method,
+        "sources": [{"role": "hf_dataset_file", "path": f"hf://datasets/{repo_id}/{path}", "resolution_m": None}],
+        "target_resolution_m": None,
+        "native_multispectral_resolution_m": None,
+        "persisted": persisted,
+        "reproducible": False,
+        "spectral_preservation": "unknown",
+        "notes": "Inferred from Hugging Face repository metadata; verify before training.",
+    }
+
+
+def safe_id(value: str) -> str:
+    return re.sub(r"[^A-Za-z0-9]+", "_", value).strip("_").lower()[:180]
+
+
+def dataset_search(api: HfApi, author: str, limit_per_query: int, max_repos: int) -> dict[str, dict]:
+    repos: dict[str, dict] = {}
+
+    def add_repo(dataset, reason: str) -> None:
+        entry = repos.setdefault(dataset.id, {"dataset": dataset, "reasons": []})
+        if reason not in entry["reasons"]:
+            entry["reasons"].append(reason)
+
+    for dataset in api.list_datasets(author=author, full=True):
+        add_repo(dataset, f"author:{author}")
+
+    for query in QUERIES:
+        for dataset in api.list_datasets(search=query, limit=limit_per_query, full=True):
+            add_repo(dataset, f"query:{query}")
+            if len(repos) >= max_repos:
+                return repos
+    return repos
+
+
+def relevant_repo(repo_id: str, tags: list[str], reasons: list[str]) -> bool:
+    text = normalize_text(" ".join([repo_id, " ".join(tags), " ".join(reasons)]))
+    if any(keyword in text for keyword in EXCLUDE_KEYWORDS):
+        return False
+    return any(keyword in text for keywords in ELEMENT_KEYWORDS.values() for keyword in keywords) or any(
+        token in text for token in ("marine", "ocean", "remote sensing", "satellite", "sar", "coast", "sea land")
+    )
+
+
+def iter_siblings(api: HfApi, repo_id: str, max_files: int) -> Iterable:
+    info = api.dataset_info(repo_id=repo_id, files_metadata=True)
+    for idx, sibling in enumerate(info.siblings or []):
+        if idx >= max_files:
+            break
+        yield sibling
+
+
+def write_jsonl(path: Path, rows: Iterable[dict]) -> None:
+    with path.open("w", encoding="utf-8") as f:
+        for row in rows:
+            f.write(json.dumps(row, ensure_ascii=False) + "\n")
+
+
+def main() -> None:
+    args = parse_args()
+    output_root = Path(args.output_root)
+    manifest_dir = output_root / "manifests"
+    report_dir = output_root / "reports"
+    manifest_dir.mkdir(parents=True, exist_ok=True)
+    report_dir.mkdir(parents=True, exist_ok=True)
+
+    api = HfApi(token=os.environ.get("HF_TOKEN"))
+    repos = dataset_search(api, args.author, args.limit_per_query, args.max_repos)
+
+    assets: list[HfAssetRecord] = []
+    repo_rows: list[dict] = []
+
+    for repo_id, entry in sorted(repos.items()):
+        dataset = entry["dataset"]
+        tags = list(getattr(dataset, "tags", []) or [])
+        reasons = entry["reasons"]
+        if not args.include_low_confidence and not relevant_repo(repo_id, tags, reasons):
+            continue
+
+        try:
+            siblings = list(iter_siblings(api, repo_id, args.max_files_per_repo))
+        except Exception as exc:  # noqa: BLE001 - keep discovery resilient.
+            repo_rows.append({"repo_id": repo_id, "status": "error", "error": str(exc), "reasons": ";".join(reasons)})
+            continue
+
+        image_count = 0
+        asset_count = 0
+        for sibling in siblings:
+            rel_path = sibling.rfilename
+            suffix = Path(rel_path).suffix.lower()
+            if suffix not in TRACKED_EXTS:
+                continue
+            role = infer_role(rel_path)
+            if suffix in IMAGE_EXTS:
+                image_count += 1
+            element = infer_element(repo_id, rel_path, " ".join(tags))
+            sensor = infer_sensor(repo_id, rel_path, " ".join(tags))
+            flags: list[str] = []
+            if element == "unknown":
+                flags.append("unknown_element")
+            if role in {"mask", "annotation_table"}:
+                flags.append("mask_asset")
+            assets.append(
+                HfAssetRecord(
+                    asset_id=safe_id(f"{repo_id}_{rel_path}"),
+                    repo_id=repo_id,
+                    path=rel_path,
+                    hf_path=f"hf://datasets/{repo_id}/{rel_path}",
+                    filename=Path(rel_path).name,
+                    suffix=suffix,
+                    role=role,
+                    element=element,
+                    satellite=infer_satellite(repo_id, rel_path, " ".join(tags)),
+                    sensor=sensor,
+                    patch_size=infer_patch_size(rel_path),
+                    source_project=repo_id.split("/", 1)[-1],
+                    source_dataset=repo_id,
+                    size_bytes=getattr(sibling, "size", None),
+                    downloads=getattr(dataset, "downloads", None),
+                    likes=getattr(dataset, "likes", None),
+                    license=infer_license(tags),
+                    tags=tags,
+                    discovered_by=reasons,
+                    quality_flags=flags,
+                )
+            )
+            asset_count += 1
+
+        repo_rows.append(
+            {
+                "repo_id": repo_id,
+                "status": "ok",
+                "reasons": ";".join(reasons),
+                "downloads": getattr(dataset, "downloads", None),
+                "likes": getattr(dataset, "likes", None),
+                "license": infer_license(tags),
+                "tags": ";".join(tags[:20]),
+                "files_scanned": len(siblings),
+                "image_assets": image_count,
+                "normalized_assets": asset_count,
+            }
+        )
+
+    masks_by_key: dict[tuple[str, str], HfAssetRecord] = {}
+    for asset in assets:
+        if asset.role == "mask":
+            masks_by_key[(asset.repo_id, Path(asset.path).stem.lower())] = asset
+
+    samples: list[HfSampleRecord] = []
+    for asset in assets:
+        if asset.role != "image":
+            continue
+        mask = masks_by_key.get((asset.repo_id, Path(asset.path).stem.lower()))
+        flags = list(asset.quality_flags)
+        if mask is None:
+            flags.append("unpaired_hf_image")
+        if asset.element == "unknown":
+            flags.append("needs_element_review")
+        samples.append(
+            HfSampleRecord(
+                sample_id=f"hf_{safe_id(asset.repo_id)}_{asset.asset_id}",
+                element=asset.element,
+                task_type="semantic_segmentation" if mask else "image_asset",
+                image_path=asset.hf_path,
+                mask_path=mask.hf_path if mask else None,
+                label_encoding={"0": "background", "1": asset.element} if mask and asset.element != "unknown" else None,
+                satellite=asset.satellite,
+                sensor=asset.sensor,
+                resolution_m=None,
+                patch_size=asset.patch_size,
+                bands=None,
+                band_count=None,
+                dtype=None,
+                fusion=infer_fusion(asset.repo_id, asset.path, asset.sensor),
+                acquired_at=None,
+                source_project=asset.source_project,
+                source_dataset=asset.source_dataset,
+                split=infer_split(asset.path),
+                quality_flags=flags,
+                notes="HF-discovered asset; inspect license, labels, georeferencing, and split before training.",
+            )
+        )
+
+    ready_samples = [
+        sample
+        for sample in samples
+        if sample.element != "unknown" and sample.task_type == "semantic_segmentation" and sample.mask_path
+    ]
+    review_samples = [sample for sample in samples if sample not in ready_samples]
+
+    write_jsonl(manifest_dir / "hf_assets_raw.jsonl", (asdict(asset) for asset in assets))
+    write_jsonl(manifest_dir / "samples.jsonl", (asdict(sample) for sample in samples))
+    write_jsonl(manifest_dir / "samples_ready.jsonl", (asdict(sample) for sample in ready_samples))
+    write_jsonl(manifest_dir / "samples_review.jsonl", (asdict(sample) for sample in review_samples))
+
+    with (report_dir / "hf_dataset_inventory.csv").open("w", newline="", encoding="utf-8-sig") as f:
+        fieldnames = [
+            "repo_id",
+            "status",
+            "reasons",
+            "downloads",
+            "likes",
+            "license",
+            "tags",
+            "files_scanned",
+            "image_assets",
+            "normalized_assets",
+            "error",
+        ]
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        for row in repo_rows:
+            writer.writerow({name: row.get(name, "") for name in fieldnames})
+
+    summary = {
+        "created_at": datetime.now(timezone.utc).isoformat(),
+        "repo_count": len(repo_rows),
+        "asset_count": len(assets),
+        "sample_count": len(samples),
+        "ready_sample_count": len(ready_samples),
+        "review_sample_count": len(review_samples),
+        "by_element": {},
+        "ready_by_element": {},
+        "output_root": str(output_root),
+        "notes": [
+            "hf:// paths are references; full dataset download is intentionally deferred.",
+            "Unknown elements and unpaired images require manual review before training.",
+            "No external coastline or land-mask vector is assumed.",
+        ],
+    }
+    for sample in samples:
+        summary["by_element"][sample.element] = summary["by_element"].get(sample.element, 0) + 1
+    for sample in ready_samples:
+        summary["ready_by_element"][sample.element] = summary["ready_by_element"].get(sample.element, 0) + 1
+    (report_dir / "hf_dataset_summary.json").write_text(json.dumps(summary, indent=2, ensure_ascii=False), encoding="utf-8")
+
+    md_lines = [
+        "# Hugging Face Marine Dataset Discovery",
+        "",
+        f"- Created: {summary['created_at']}",
+        f"- Repositories reviewed: {summary['repo_count']}",
+        f"- Image assets indexed: {summary['asset_count']}",
+        f"- Standardized samples written: {summary['sample_count']}",
+        f"- Training-ready samples: {summary['ready_sample_count']}",
+        f"- Review samples: {summary['review_sample_count']}",
+        "",
+        "## Samples By Element",
+        "",
+    ]
+    for element, count in sorted(summary["by_element"].items()):
+        md_lines.append(f"- `{element}`: {count}")
+    md_lines.extend(["", "## Training-Ready Samples By Element", ""])
+    for element, count in sorted(summary["ready_by_element"].items()):
+        md_lines.append(f"- `{element}`: {count}")
+    md_lines.extend(
+        [
+            "",
+            "## Important Notes",
+            "",
+            "- Manifests use `hf://datasets/<repo>/<path>` references and do not imply files were downloaded.",
+            "- Licenses and label semantics must be checked before a dataset is used for training.",
+            "- Unknown or unpaired assets are kept for review, not treated as training-ready negatives.",
+        ]
+    )
+    (report_dir / "hf_dataset_discovery.md").write_text("\n".join(md_lines) + "\n", encoding="utf-8")
+    print(json.dumps(summary, indent=2, ensure_ascii=False))
+
+
+if __name__ == "__main__":
+    main()

scripts/train_2gpu.ps1

@@ -0,0 +1,28 @@
+param(
+  [string]$Config = "configs/train_2gpu_4090d.json",
+  [int]$BatchSize = 10,
+  [int]$Epochs = 500,
+  [string]$Weights = ""
+)
+
+$ErrorActionPreference = "Stop"
+$env:CUDA_VISIBLE_DEVICES = "0,1"
+$env:PYTHONUTF8 = "1"
+$env:USE_LIBUV = "0"
+
+$argsList = @(
+  "scripts/launch_ddp_no_libuv.py",
+  "--nproc_per_node=2",
+  "--master_addr=127.0.0.1",
+  "--master_port=29500",
+  "train_seaweed_segmentation.py",
+  "--config", $Config,
+  "--batch-size", "$BatchSize",
+  "--epochs", "$Epochs"
+)
+
+if ($Weights -ne "") {
+  $argsList += @("--weights", $Weights)
+}
+
+python @argsList

scripts/train_yolo_detection.py

@@ -0,0 +1,56 @@
+"""Train a YOLO detector from a prepared data.yaml file."""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--data", required=True)
+    parser.add_argument("--model", default="yolov8n.pt")
+    parser.add_argument("--epochs", type=int, default=80)
+    parser.add_argument("--imgsz", type=int, default=640)
+    parser.add_argument("--batch", type=int, default=4)
+    parser.add_argument("--device", default="0")
+    parser.add_argument("--workers", type=int, default=2)
+    parser.add_argument("--project", required=True)
+    parser.add_argument("--name", required=True)
+    parser.add_argument("--patience", type=int, default=20)
+    parser.add_argument("--amp", action=argparse.BooleanOptionalAction, default=True)
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    from ultralytics import YOLO
+    import torch
+
+    print(f"torch={torch.__version__} cuda={torch.cuda.is_available()} devices={torch.cuda.device_count()}", flush=True)
+    print(f"data={args.data}", flush=True)
+    print(f"model={args.model}", flush=True)
+    print(f"amp={args.amp}", flush=True)
+
+    Path(args.project).mkdir(parents=True, exist_ok=True)
+    model = YOLO(args.model)
+    result = model.train(
+        data=args.data,
+        epochs=args.epochs,
+        imgsz=args.imgsz,
+        batch=args.batch,
+        device=args.device,
+        workers=args.workers,
+        project=args.project,
+        name=args.name,
+        exist_ok=True,
+        patience=args.patience,
+        amp=args.amp,
+        cache=False,
+        plots=True,
+    )
+    print(result, flush=True)
+
+
+if __name__ == "__main__":
+    main()

scripts/validate_yolo_map.py

@@ -0,0 +1,47 @@
+"""Run Ultralytics validation and save compact mAP metrics."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--weights", required=True)
+    parser.add_argument("--data", required=True)
+    parser.add_argument("--split", default="test", choices=["train", "val", "test"])
+    parser.add_argument("--imgsz", type=int, default=640)
+    parser.add_argument("--batch", type=int, default=16)
+    parser.add_argument("--device", default="0")
+    parser.add_argument("--project", required=True)
+    parser.add_argument("--name", required=True)
+    parser.add_argument("--output", required=True)
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    from ultralytics import YOLO
+
+    model = YOLO(args.weights)
+    result = model.val(
+        data=args.data,
+        split=args.split,
+        imgsz=args.imgsz,
+        batch=args.batch,
+        device=args.device,
+        plots=True,
+        project=args.project,
+        name=args.name,
+    )
+    metrics = {key: float(value) for key, value in result.results_dict.items()}
+    output = Path(args.output)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    output.write_text(json.dumps(metrics, indent=2), encoding="utf-8")
+    print(json.dumps(metrics, indent=2))
+
+
+if __name__ == "__main__":
+    main()

scripts/visualize_yolo_errors.py

@@ -0,0 +1,92 @@
+"""Create a contact sheet for YOLO error cases with GT and prediction boxes."""
+
+from __future__ import annotations
+
+import argparse
+import json
+from pathlib import Path
+
+from PIL import Image, ImageDraw, ImageFont
+
+from evaluate_yolo_detection import label_for_image, read_label
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--weights", required=True)
+    parser.add_argument("--metrics-json", required=True)
+    parser.add_argument("--output", required=True)
+    parser.add_argument("--imgsz", type=int, default=640)
+    parser.add_argument("--conf", type=float, default=0.25)
+    parser.add_argument("--device", default="0")
+    parser.add_argument("--limit", type=int, default=12)
+    parser.add_argument("--thumb", type=int, default=360)
+    return parser.parse_args()
+
+
+def draw_scaled_box(draw: ImageDraw.ImageDraw, box: list[float], scale: float, color: str, width: int = 3) -> None:
+    coords = [box[0] * scale, box[1] * scale, box[2] * scale, box[3] * scale]
+    draw.rectangle(coords, outline=color, width=width)
+
+
+def main() -> None:
+    args = parse_args()
+    from ultralytics import YOLO
+
+    metrics = json.loads(Path(args.metrics_json).read_text(encoding="utf-8"))
+    error_images = sorted(
+        metrics["error_images"],
+        key=lambda item: int(item["false_negatives"]) + int(item["false_positives"]),
+        reverse=True,
+    )[: args.limit]
+    model = YOLO(args.weights)
+
+    tiles: list[Image.Image] = []
+    for item in error_images:
+        image_path = Path(str(item["image"]))
+        image = Image.open(image_path).convert("RGB")
+        width, height = image.size
+        scale = args.thumb / max(width, height)
+        tile_size = (int(width * scale), int(height * scale))
+        tile = image.resize(tile_size)
+        draw = ImageDraw.Draw(tile)
+
+        gt_boxes = read_label(label_for_image(image_path), width, height)
+        for box in gt_boxes:
+            draw_scaled_box(draw, box, scale, "lime")
+
+        result = model.predict(
+            source=str(image_path),
+            imgsz=args.imgsz,
+            conf=args.conf,
+            device=args.device,
+            verbose=False,
+        )[0]
+        pred_boxes = result.boxes.xyxy.cpu().tolist() if result.boxes is not None else []
+        for box in pred_boxes:
+            draw_scaled_box(draw, box, scale, "red")
+
+        caption = f"FN {item['false_negatives']} FP {item['false_positives']}  {image_path.name}"
+        caption_h = 34
+        panel = Image.new("RGB", (args.thumb, args.thumb + caption_h), "white")
+        panel.paste(tile, ((args.thumb - tile.width) // 2, 0))
+        panel_draw = ImageDraw.Draw(panel)
+        panel_draw.text((6, args.thumb + 8), caption[:70], fill="black", font=ImageFont.load_default())
+        tiles.append(panel)
+
+    cols = 3
+    rows = (len(tiles) + cols - 1) // cols
+    sheet = Image.new("RGB", (cols * args.thumb, rows * (args.thumb + 34)), "white")
+    for idx, tile in enumerate(tiles):
+        x = (idx % cols) * args.thumb
+        y = (idx // cols) * (args.thumb + 34)
+        sheet.paste(tile, (x, y))
+
+    output = Path(args.output)
+    output.parent.mkdir(parents=True, exist_ok=True)
+    sheet.save(output)
+    print(output)
+
+
+if __name__ == "__main__":
+    main()

seaweed_segmentation_dataset.py

@@ -0,0 +1,139 @@
+"""Dataset utilities for seaweed binary segmentation."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import albumentations as A
+import numpy as np
+import rasterio
+import torch
+from PIL import Image
+from torch.utils.data import Dataset
+from torchvision import transforms
+
+
+IMAGE_EXTS = {".tif", ".tiff", ".png", ".jpg", ".jpeg"}
+MASK_EXTS = (".png", ".tif", ".tiff", ".jpg", ".jpeg")
+
+
+class SeaweedSegmentationDataset(Dataset):
+    def __init__(self, image_dir, mask_dir, transform=None, target_size=256, use_4channel=True):
+        self.image_dir = Path(image_dir)
+        self.mask_dir = Path(mask_dir)
+        self.transform = transform
+        self.target_size = int(target_size)
+        self.use_4channel = bool(use_4channel)
+
+        if not self.image_dir.exists():
+            raise FileNotFoundError(f"Image directory not found: {self.image_dir}")
+        if not self.mask_dir.exists():
+            raise FileNotFoundError(f"Mask directory not found: {self.mask_dir}")
+
+        self.images = sorted(p.name for p in self.image_dir.iterdir() if p.is_file() and p.suffix.lower() in IMAGE_EXTS)
+
+        self.normalize_3ch = transforms.Normalize(mean=(0.430, 0.411, 0.296), std=(0.213, 0.156, 0.143))
+        self.normalize_4ch = transforms.Normalize(mean=(0.430, 0.411, 0.296, 0.350), std=(0.213, 0.156, 0.143, 0.180))
+
+    def __len__(self):
+        return len(self.images)
+
+    def find_mask_path(self, image_name: str) -> Path:
+        stem = Path(image_name).stem
+        for ext in MASK_EXTS:
+            for suffix in (ext, ext.upper()):
+                candidate = self.mask_dir / f"{stem}{suffix}"
+                if candidate.exists():
+                    return candidate
+        return self.mask_dir / f"{stem}.png"
+
+    @staticmethod
+    def read_raster(path: Path) -> np.ndarray:
+        if path.suffix.lower() in {".tif", ".tiff"}:
+            with rasterio.open(path) as src:
+                return np.transpose(src.read(), (1, 2, 0))
+        image = Image.open(path)
+        return np.asarray(image)
+
+    @staticmethod
+    def extract_432_bands(image: np.ndarray) -> np.ndarray:
+        if image.ndim == 2:
+            image = image[:, :, None]
+        if image.shape[2] >= 4:
+            return image[:, :, [3, 2, 1]]
+        output = image[:, :, : min(3, image.shape[2])]
+        while output.shape[2] < 3:
+            output = np.concatenate([output, output[:, :, -1:]], axis=2)
+        return output
+
+    @staticmethod
+    def read_mask(path: Path, fallback_shape: tuple[int, int]) -> np.ndarray:
+        if not path.exists():
+            return np.zeros(fallback_shape, dtype=np.uint8)
+        if path.suffix.lower() in {".tif", ".tiff"}:
+            with rasterio.open(path) as src:
+                mask = src.read(1)
+        else:
+            mask = np.asarray(Image.open(path).convert("L"))
+        return (mask > 127).astype(np.uint8)
+
+    def __getitem__(self, idx):
+        img_name = self.images[idx]
+        img_path = self.image_dir / img_name
+        mask_path = self.find_mask_path(img_name)
+
+        try:
+            image = self.read_raster(img_path)
+            if image.ndim == 2:
+                image = image[:, :, None]
+            mask = self.read_mask(mask_path, image.shape[:2])
+
+            if self.use_4channel and image.shape[2] >= 4:
+                processed = image[:, :, :4]
+                processed = torch.from_numpy(processed.astype(np.float32))
+                if processed.max() > 1.0:
+                    processed = processed / 65535.0
+                processed = self.normalize_4ch(processed.permute(2, 0, 1))
+            else:
+                processed = self.extract_432_bands(image)
+                processed = torch.from_numpy(processed.astype(np.float32))
+                if processed.max() > 1.0:
+                    processed = processed / 65535.0
+                processed = self.normalize_3ch(processed.permute(2, 0, 1))
+
+            mask_tensor = torch.from_numpy(mask).long()
+
+            if self.target_size != processed.shape[1] or self.target_size != processed.shape[2]:
+                processed = transforms.Resize((self.target_size, self.target_size), antialias=True)(processed)
+                mask_tensor = transforms.Resize(
+                    (self.target_size, self.target_size),
+                    interpolation=transforms.InterpolationMode.NEAREST,
+                )(mask_tensor.unsqueeze(0)).squeeze(0)
+
+            if self.transform:
+                augmented = self.transform(image=processed.permute(1, 2, 0).numpy(), mask=mask_tensor.numpy())
+                processed = torch.from_numpy(augmented["image"]).permute(2, 0, 1).float()
+                mask_tensor = torch.from_numpy(augmented["mask"]).long()
+
+            return {"image": processed, "mask": mask_tensor, "filename": img_name}
+        except Exception as exc:
+            print(f"Error loading {img_name}: {exc}")
+            return None
+
+
+def get_train_transforms(target_size=256, use_4channel=True):
+    return A.Compose(
+        [
+            A.Resize(target_size, target_size),
+            A.HorizontalFlip(p=0.5),
+            A.VerticalFlip(p=0.3),
+            A.RandomRotate90(p=0.3),
+            A.ShiftScaleRotate(shift_limit=0.1, scale_limit=0.1, rotate_limit=15, p=0.5),
+            A.RandomBrightnessContrast(p=0.3),
+            A.GaussNoise(p=0.2),
+        ]
+    )
+
+
+def get_val_transforms(target_size=256, use_4channel=True):
+    return None

seaweed_segmentation_improved_epoch500/config.json

@@ -0,0 +1,32 @@
+{
+  "train_image_dir": "data/train/images",
+  "train_mask_dir": "data/train/masks",
+  "val_image_dir": "data/val/images",
+  "val_mask_dir": "data/val/masks",
+  "num_classes": 2,
+  "backbone_name": "dinov3_vitl16",
+  "pretrained": true,
+  "backbone_weights": "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth",
+  "use_4channel": true,
+  "image_size": 256,
+  "batch_size": 8,
+  "num_epochs": 500,
+  "backbone_lr": 1e-05,
+  "decoder_lr": 0.0001,
+  "weight_decay": 0.0001,
+  "scheduler": "cosine",
+  "grad_clip": 1.0,
+  "focal_alpha": [
+    1.0,
+    3.0
+  ],
+  "focal_gamma": 2,
+  "dice_weight": 0.5,
+  "focal_weight": 1.0,
+  "num_workers": 0,
+  "output_dir": "outputs/seaweed_segmentation_improved_epoch500",
+  "plot_interval": 10,
+  "device": "cuda",
+  "background_weight": 1.0,
+  "foreground_weight": 3.0
+}

test-org.xml

@@ -0,0 +1,73 @@
+<?xml version="1.0" encoding="utf-8" ?>
+ <Job_Order>
+	<!--配置参数-->
+	<Conf> 
+		<!--处理组件名称-->
+		<Processor_Name>coral</Processor_Name>
+		<!--处理组件版本号-->
+		<Version>v01.01</Version>
+		<!--订单号-->
+		<Order_ID>0000029777</Order_ID>
+		<!--工作目录-->
+		<working_directory>E:\yqj\code\GF\coral\v01.01\processing</working_directory>
+		<!--标准日志位置-->
+		<Stdout_Log_Level>Logs</Stdout_Log_Level>  
+		<!--错误日志位置-->
+		<Stderr_Log_Level>Logs</Stderr_Log_Level> 
+		<!--是否测试组件-->		
+		<Test>false</Test> 
+		<!--arcpy地址-->	
+		<Proc1>C:\Python27\ArcGIS10.3\python.exe</Proc1>  		
+		<!--配置文件-->			
+		<Config_Files> 
+			<!---->
+			<Conf_File_Name>null</Conf_File_Name> 
+			<Conf_File_Name>null</Conf_File_Name>
+			<Conf_File_Name>null</Conf_File_Name>
+		</Config_Files>  
+		<!--触发时间-->			
+		<Sensing_Time>       
+			<Start>2021-11-19T21：10：00Z</Start>       
+			<Stop>2021-11-19T21：18：00Z</Stop>     
+		</Sensing_Time>   
+		<!--处理组件参数-->	
+		<Parameters>        
+			<Parameter>         
+				<Name>thematic_map_title</Name>         
+				<Value>养殖区遥感监测信息提取专题产品</Value>       
+			</Parameter>            			
+		</Parameters>  
+		<!--是否生产专题图-->	
+		<Proc1>True</Proc1> 		
+	</Conf>   
+	<!--执行进程-->
+	<Procs>  
+		<!--单个进程-->	
+		<Proc>     
+			<!--任务名称-->	
+			<Task_Name>RAA</Task_Name>
+			<!--任务版本-->				
+			<Task_Version>01.00</Task_Version>       
+			<Inputs>   
+				<!--输入数据-->				
+				<Input>  
+					<!--文件类型-->					
+					<File_Type>1</File_Type>                     
+					<File_Names>
+					     <!--多光谱原始影像-->
+						<File_Name>G:\webGis\line\GF1_PMS1_E119.3_N35.0_20160818_L1A0001770060-MSS1.tiff</File_Name>
+
+					</File_Names>         
+				</Input>       
+			</Inputs>       
+			<Outputs>  
+				<!--输出数据-->				    
+				<Output> 
+					<!--只包含输出路径+文件名，不带扩展名-->
+					<File_Name>G:\webGis\line\result</File_Name>    					
+				</Output> 
+							
+			</Outputs>     
+		</Proc>   
+	</Procs> 
+</Job_Order>

test.xml

@@ -0,0 +1 @@
+<JSONObject><Conf><Processor_Name>coral</Processor_Name><Version>v01.01</Version><Order_ID>0000029777</Order_ID><working_directory>E:\yqj\code\GF\coral\v01.01\processing</working_directory><Stdout_Log_Level>Logs</Stdout_Log_Level><Stderr_Log_Level>Logs</Stderr_Log_Level><Test>false</Test><Proc1>C:\Python27\ArcGIS10.3\python.exe</Proc1><Proc1>True</Proc1><Config_Files><Conf_File_Name>null</Conf_File_Name><Conf_File_Name>null</Conf_File_Name><Conf_File_Name>null</Conf_File_Name></Config_Files><Sensing_Time><Start>2021-11-19T21：10：00Z</Start><Stop>2021-11-19T21：18：00Z</Stop></Sensing_Time><Parameters><Parameter><Name>thematic_map_title</Name><Value>养殖区遥感监测信息提取专题产品</Value></Parameter></Parameters></Conf><Procs><Proc><Task_Name>RAA</Task_Name><Task_Version>01.00</Task_Version><Inputs><Input><File_Type>1</File_Type><File_Names><File_Name>F:/skzh/uploadPath/upload/temp/GF1_WFV1_E122.2_N34.7_20210525_L1A0005664506.tiff</File_Name></File_Names></Input></Inputs><Outputs><Output><File_Name>F:/Results/admin/GF1/157</File_Name></Output></Outputs></Proc></Procs></JSONObject>

train_config.json

@@ -0,0 +1,27 @@
+{
+  "train_image_dir": "data/train/images",
+  "train_mask_dir": "data/train/masks",
+  "val_image_dir": "data/val/images",
+  "val_mask_dir": "data/val/masks",
+  "num_classes": 2,
+  "backbone_name": "dinov3_vitl16",
+  "pretrained": true,
+  "backbone_weights": "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth",
+  "use_4channel": true,
+  "image_size": 256,
+  "batch_size": 8,
+  "num_epochs": 200,
+  "backbone_lr": 1e-05,
+  "decoder_lr": 0.0001,
+  "weight_decay": 0.0001,
+  "scheduler": "cosine",
+  "grad_clip": 1.0,
+  "focal_alpha": 1,
+  "focal_gamma": 2,
+  "dice_weight": 0.5,
+  "focal_weight": 1.0,
+  "num_workers": 0,
+  "output_dir": "outputs/seaweed_segmentation_full_epoch200",
+  "plot_interval": 10,
+  "device": "cuda"
+}

train_config_improved.json

@@ -0,0 +1,33 @@
+{
+  "train_image_dir": "data/train/images",
+  "train_mask_dir": "data/train/masks",
+  "val_image_dir": "data/val/images",
+  "val_mask_dir": "data/val/masks",
+  "num_classes": 2,
+  "backbone_name": "dinov3_vitl16",
+  "pretrained": true,
+  "backbone_weights": "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth",
+  "use_4channel": true,
+  "image_size": 256,
+  "batch_size": 8,
+  "num_epochs": 500,
+  "backbone_lr": 1e-05,
+  "decoder_lr": 0.0001,
+  "weight_decay": 0.0001,
+  "scheduler": "cosine",
+  "grad_clip": 1.0,
+  "focal_alpha": [1.0, 3.0],
+  "focal_gamma": 2,
+  "dice_weight": 0.5,
+  "focal_weight": 1.0,
+  "background_weight": 1.0,
+  "foreground_weight": 3.0,
+  "num_workers": 0,
+  "output_dir": "outputs/seaweed_segmentation_improved_epoch500",
+  "plot_interval": 10,
+  "device": "cuda"
+}
+
+
+
+

train_seaweed_segmentation.py

@@ -0,0 +1,537 @@
+"""Train DINOv3 + DeepLabV3+ for seaweed segmentation.
+
+The script supports single GPU, torchrun DDP, AMP, gradient accumulation,
+checkpoint resume, and rank-0-only logging/checkpointing.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import random
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt
+import numpy as np
+import torch
+import torch.distributed as dist
+import torch.nn as nn
+import torch.optim as optim
+from torch.cuda.amp import GradScaler, autocast
+from torch.nn.parallel import DistributedDataParallel as DDP
+from torch.utils.data import DataLoader
+from torch.utils.data.distributed import DistributedSampler
+from tqdm import tqdm
+
+from dinov3_deeplabv3plus import DinoV3DeepLabV3Plus, SeaweedSegmentationLoss
+from seaweed_segmentation_dataset import SeaweedSegmentationDataset, get_train_transforms, get_val_transforms
+
+
+def load_json(path: str | Path) -> dict[str, Any]:
+    with open(path, "r", encoding="utf-8") as f:
+        return json.load(f)
+
+
+def is_dist() -> bool:
+    return dist.is_available() and dist.is_initialized()
+
+
+def rank() -> int:
+    return dist.get_rank() if is_dist() else 0
+
+
+def world_size() -> int:
+    return dist.get_world_size() if is_dist() else 1
+
+
+def is_main_process() -> bool:
+    return rank() == 0
+
+
+def setup_distributed() -> tuple[torch.device, int]:
+    local_rank = int(os.environ.get("LOCAL_RANK", "0"))
+    distributed = int(os.environ.get("WORLD_SIZE", "1")) > 1
+
+    if distributed:
+        backend = "nccl" if os.name != "nt" else "gloo"
+        rank_id = int(os.environ["RANK"])
+        size = int(os.environ["WORLD_SIZE"])
+        if os.environ.get("USE_LIBUV", "1") == "0":
+            from datetime import timedelta
+
+            store = dist.TCPStore(
+                os.environ.get("MASTER_ADDR", "127.0.0.1"),
+                int(os.environ.get("MASTER_PORT", "29500")),
+                world_size=size,
+                is_master=rank_id == 0,
+                timeout=timedelta(seconds=180),
+                use_libuv=False,
+            )
+            dist.init_process_group(backend=backend, store=store, rank=rank_id, world_size=size)
+        else:
+            dist.init_process_group(backend=backend)
+
+    if torch.cuda.is_available():
+        if distributed:
+            torch.cuda.set_device(local_rank)
+        device = torch.device(f"cuda:{local_rank}")
+    else:
+        device = torch.device("cpu")
+    return device, local_rank
+
+
+def cleanup_distributed() -> None:
+    if is_dist():
+        dist.barrier()
+        dist.destroy_process_group()
+
+
+def seed_everything(seed: int) -> None:
+    random.seed(seed)
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+
+
+def resolve_path(path: str | None, roots: list[str]) -> str | None:
+    if not path:
+        return path
+    candidate = Path(path)
+    if candidate.exists():
+        return str(candidate)
+    for root in roots:
+        rooted = Path(root) / path
+        if rooted.exists():
+            return str(rooted)
+    return path
+
+
+def collate_skip_none(batch: list[dict[str, Any] | None]) -> dict[str, Any]:
+    valid = [item for item in batch if item is not None]
+    if not valid:
+        raise RuntimeError("All samples in this batch failed to load.")
+    images = torch.stack([item["image"] for item in valid], dim=0)
+    masks = torch.stack([item["mask"] for item in valid], dim=0)
+    filenames = [item["filename"] for item in valid]
+    return {"image": images, "mask": masks, "filename": filenames}
+
+
+class SeaweedSegmentationTrainer:
+    def __init__(self, config: dict[str, Any], device: torch.device, local_rank: int):
+        self.config = config
+        self.device = device
+        self.local_rank = local_rank
+        self.use_amp = bool(config.get("amp", True)) and device.type == "cuda"
+        self.grad_accum_steps = int(config.get("grad_accum_steps", 1))
+
+        weight_roots = config.get("weight_search_roots", [])
+        config["backbone_weights"] = resolve_path(config.get("backbone_weights"), weight_roots)
+
+        seed_everything(int(config.get("seed", 42)) + rank())
+
+        model = DinoV3DeepLabV3Plus(
+            num_classes=config["num_classes"],
+            backbone_name=config["backbone_name"],
+            pretrained=config["pretrained"],
+            weights=config["backbone_weights"],
+            use_4channel=config["use_4channel"],
+            freeze_backbone=config.get("freeze_backbone", False),
+        ).to(device)
+
+        if is_dist():
+            self.model = DDP(
+                model,
+                device_ids=[local_rank] if device.type == "cuda" else None,
+                find_unused_parameters=True,
+            )
+        else:
+            self.model = model
+
+        self.criterion = SeaweedSegmentationLoss(
+            num_classes=config["num_classes"],
+            focal_alpha=config.get("focal_alpha", [1.0, 2.0]),
+            focal_gamma=config.get("focal_gamma", 2),
+            dice_weight=config.get("dice_weight", 0.5),
+            focal_weight=config.get("focal_weight", 1.0),
+            background_weight=config.get("background_weight", 1.0),
+            foreground_weight=config.get("foreground_weight", 2.0),
+        ).to(device)
+
+        self.optimizer = self._create_optimizer()
+        self.scheduler = self._create_scheduler()
+        self.scaler = GradScaler(enabled=self.use_amp)
+        self.train_loader, self.val_loader = self._create_data_loaders()
+
+        self.train_history = {
+            "train_loss": [],
+            "train_focal": [],
+            "train_dice": [],
+            "val_loss": [],
+            "val_focal": [],
+            "val_dice": [],
+            "val_iou": [],
+            "val_accuracy": [],
+        }
+        self.best_val_loss = float("inf")
+        self.best_val_iou = 0.0
+        self.start_epoch = 0
+
+        self.output_dir = Path(config["output_dir"])
+        if is_main_process():
+            self.output_dir.mkdir(parents=True, exist_ok=True)
+            with open(self.output_dir / "config.json", "w", encoding="utf-8") as f:
+                json.dump(config, f, indent=2, ensure_ascii=False)
+
+        if config.get("resume"):
+            self.load_checkpoint(config["resume"])
+
+    @property
+    def raw_model(self) -> nn.Module:
+        return self.model.module if isinstance(self.model, DDP) else self.model
+
+    def _create_optimizer(self):
+        raw_model = self.raw_model
+        if self.config.get("freeze_backbone", False):
+            params = raw_model.get_trainable_parameters()
+            lr = self.config.get("decoder_lr", 1e-4)
+            if is_main_process():
+                print(f"Frozen backbone mode. Trainable tensors: {len(params)}")
+            return optim.AdamW(params, lr=lr, weight_decay=self.config.get("weight_decay", 1e-4))
+
+        backbone_params = list(raw_model.get_backbone_params())
+        decoder_params = list(raw_model.get_decoder_params())
+        return optim.AdamW(
+            [
+                {"params": backbone_params, "lr": self.config.get("backbone_lr", 1e-5)},
+                {"params": decoder_params, "lr": self.config.get("decoder_lr", 1e-4)},
+            ],
+            weight_decay=self.config.get("weight_decay", 1e-4),
+        )
+
+    def _create_scheduler(self):
+        scheduler_type = self.config.get("scheduler", "cosine")
+        if scheduler_type == "cosine":
+            return optim.lr_scheduler.CosineAnnealingWarmRestarts(self.optimizer, T_0=10, T_mult=2, eta_min=1e-7)
+        if scheduler_type == "step":
+            return optim.lr_scheduler.StepLR(self.optimizer, step_size=30, gamma=0.1)
+        return None
+
+    def _create_data_loaders(self):
+        use_aug = self.config.get("use_data_augmentation", True)
+        train_transform = (
+            get_train_transforms(self.config["image_size"], self.config["use_4channel"])
+            if use_aug
+            else get_val_transforms(self.config["image_size"], self.config["use_4channel"])
+        )
+
+        train_dataset = SeaweedSegmentationDataset(
+            image_dir=self.config["train_image_dir"],
+            mask_dir=self.config["train_mask_dir"],
+            transform=train_transform,
+            target_size=self.config["image_size"],
+            use_4channel=self.config["use_4channel"],
+        )
+        val_dataset = SeaweedSegmentationDataset(
+            image_dir=self.config["val_image_dir"],
+            mask_dir=self.config["val_mask_dir"],
+            transform=get_val_transforms(self.config["image_size"], self.config["use_4channel"]),
+            target_size=self.config["image_size"],
+            use_4channel=self.config["use_4channel"],
+        )
+
+        if len(train_dataset) == 0 or len(val_dataset) == 0:
+            raise RuntimeError(
+                f"Empty dataset: train={len(train_dataset)}, val={len(val_dataset)}. "
+                "Check image/mask directories in the config."
+            )
+
+        train_sampler = DistributedSampler(train_dataset, shuffle=True) if is_dist() else None
+        val_sampler = DistributedSampler(val_dataset, shuffle=False) if is_dist() else None
+        num_workers = int(self.config.get("num_workers", 4))
+        loader_kwargs = {
+            "batch_size": int(self.config["batch_size"]),
+            "num_workers": num_workers,
+            "pin_memory": self.device.type == "cuda",
+            "collate_fn": collate_skip_none,
+            "persistent_workers": num_workers > 0,
+        }
+        train_loader = DataLoader(
+            train_dataset,
+            shuffle=train_sampler is None,
+            sampler=train_sampler,
+            drop_last=True,
+            **loader_kwargs,
+        )
+        val_loader = DataLoader(
+            val_dataset,
+            shuffle=False,
+            sampler=val_sampler,
+            drop_last=False,
+            **loader_kwargs,
+        )
+        return train_loader, val_loader
+
+    def calculate_metrics(self, pred, target):
+        if isinstance(pred, dict):
+            pred = pred["out"]
+        pred_classes = torch.argmax(pred, dim=1)
+        foreground_pred = pred_classes == 1
+        foreground_target = target == 1
+        intersection = (foreground_pred & foreground_target).sum().float()
+        union = (foreground_pred | foreground_target).sum().float()
+        iou = intersection / union.clamp_min(1)
+        accuracy = (pred_classes == target).float().mean()
+        return {"iou": iou.detach(), "accuracy": accuracy.detach()}
+
+    def _reduce_scalar(self, value: torch.Tensor) -> float:
+        value = value.detach().float()
+        if is_dist():
+            dist.all_reduce(value, op=dist.ReduceOp.SUM)
+            value /= world_size()
+        return value.item()
+
+    def run_epoch(self, epoch: int, train: bool):
+        self.model.train(train)
+        loader = self.train_loader if train else self.val_loader
+        if train and isinstance(loader.sampler, DistributedSampler):
+            loader.sampler.set_epoch(epoch)
+
+        totals = {"total_loss": 0.0, "focal_loss": 0.0, "dice_loss": 0.0, "iou": 0.0, "accuracy": 0.0}
+        steps = 0
+        desc = f"Epoch {epoch + 1}/{self.config['num_epochs']} - {'Train' if train else 'Val'}"
+        iterator = tqdm(loader, desc=desc, disable=not is_main_process())
+
+        if train:
+            self.optimizer.zero_grad(set_to_none=True)
+
+        for step, batch in enumerate(iterator):
+            max_batches = self.config.get("max_train_batches" if train else "max_val_batches")
+            if max_batches is not None and step >= int(max_batches):
+                break
+
+            images = batch["image"].to(self.device, non_blocking=True)
+            masks = batch["mask"].to(self.device, non_blocking=True)
+
+            with torch.set_grad_enabled(train):
+                with autocast(enabled=self.use_amp):
+                    outputs = self.model(images)
+                    losses = self.criterion(outputs, masks)
+                    loss = losses["total_loss"] / self.grad_accum_steps
+
+                if train:
+                    self.scaler.scale(loss).backward()
+                    should_step = (step + 1) % self.grad_accum_steps == 0 or (step + 1) == len(loader)
+                    if should_step:
+                        if self.config.get("grad_clip"):
+                            self.scaler.unscale_(self.optimizer)
+                            torch.nn.utils.clip_grad_norm_(self.raw_model.parameters(), self.config["grad_clip"])
+                        self.scaler.step(self.optimizer)
+                        self.scaler.update()
+                        self.optimizer.zero_grad(set_to_none=True)
+
+            metrics = self.calculate_metrics(outputs, masks)
+            totals["total_loss"] += self._reduce_scalar(losses["total_loss"])
+            totals["focal_loss"] += self._reduce_scalar(losses["focal_loss"])
+            totals["dice_loss"] += self._reduce_scalar(losses["dice_loss"])
+            totals["iou"] += self._reduce_scalar(metrics["iou"])
+            totals["accuracy"] += self._reduce_scalar(metrics["accuracy"])
+            steps += 1
+
+            if is_main_process():
+                iterator.set_postfix(
+                    loss=f"{totals['total_loss'] / steps:.4f}",
+                    iou=f"{totals['iou'] / steps:.4f}",
+                    acc=f"{totals['accuracy'] / steps:.4f}",
+                )
+
+        return {k: v / max(steps, 1) for k, v in totals.items()}
+
+    def save_checkpoint(self, epoch: int, is_best: bool) -> None:
+        if not is_main_process():
+            return
+        checkpoint = {
+            "epoch": epoch,
+            "model_state_dict": self.raw_model.state_dict(),
+            "optimizer_state_dict": self.optimizer.state_dict(),
+            "scheduler_state_dict": self.scheduler.state_dict() if self.scheduler else None,
+            "scaler_state_dict": self.scaler.state_dict(),
+            "train_history": self.train_history,
+            "config": self.config,
+            "best_val_loss": self.best_val_loss,
+            "best_val_iou": self.best_val_iou,
+        }
+        torch.save(checkpoint, self.output_dir / "latest_checkpoint.pth")
+        if is_best:
+            torch.save(checkpoint, self.output_dir / "best_checkpoint.pth")
+            print(f"Saved best checkpoint: {self.output_dir / 'best_checkpoint.pth'}")
+
+    def load_checkpoint(self, path: str) -> None:
+        checkpoint = torch.load(path, map_location=self.device)
+        self.raw_model.load_state_dict(checkpoint["model_state_dict"], strict=True)
+        self.optimizer.load_state_dict(checkpoint["optimizer_state_dict"])
+        if self.scheduler and checkpoint.get("scheduler_state_dict"):
+            self.scheduler.load_state_dict(checkpoint["scheduler_state_dict"])
+        if checkpoint.get("scaler_state_dict"):
+            self.scaler.load_state_dict(checkpoint["scaler_state_dict"])
+        self.train_history = checkpoint.get("train_history", self.train_history)
+        self.best_val_loss = checkpoint.get("best_val_loss", self.best_val_loss)
+        self.best_val_iou = checkpoint.get("best_val_iou", self.best_val_iou)
+        self.start_epoch = int(checkpoint.get("epoch", -1)) + 1
+        if is_main_process():
+            print(f"Resumed from {path} at epoch {self.start_epoch}")
+
+    def plot_training_history(self) -> None:
+        if not is_main_process() or not self.train_history["train_loss"]:
+            return
+        epochs = range(1, len(self.train_history["train_loss"]) + 1)
+        fig, axes = plt.subplots(2, 2, figsize=(15, 10))
+        axes[0, 0].plot(epochs, self.train_history["train_loss"], label="Train")
+        axes[0, 0].plot(epochs, self.train_history["val_loss"], label="Val")
+        axes[0, 0].set_title("Loss")
+        axes[0, 0].legend()
+        axes[0, 1].plot(epochs, self.train_history["train_focal"], label="Train")
+        axes[0, 1].plot(epochs, self.train_history["val_focal"], label="Val")
+        axes[0, 1].set_title("Focal Loss")
+        axes[0, 1].legend()
+        axes[1, 0].plot(epochs, self.train_history["train_dice"], label="Train")
+        axes[1, 0].plot(epochs, self.train_history["val_dice"], label="Val")
+        axes[1, 0].set_title("Dice Loss")
+        axes[1, 0].legend()
+        axes[1, 1].plot(epochs, self.train_history["val_iou"], label="Val IoU")
+        axes[1, 1].plot(epochs, self.train_history["val_accuracy"], label="Val Acc")
+        axes[1, 1].set_title("Metrics")
+        axes[1, 1].legend()
+        for ax in axes.ravel():
+            ax.grid(True)
+        plt.tight_layout()
+        plt.savefig(self.output_dir / "training_history.png", dpi=200, bbox_inches="tight")
+        plt.close(fig)
+
+    def train(self):
+        if is_main_process():
+            effective_batch = self.config["batch_size"] * world_size() * self.grad_accum_steps
+            print(f"Device: {self.device}; world_size={world_size()}; AMP={self.use_amp}")
+            print(f"Per-GPU batch: {self.config['batch_size']}; effective batch: {effective_batch}")
+            print(f"Train samples: {len(self.train_loader.dataset)}; val samples: {len(self.val_loader.dataset)}")
+            print(f"Parameters: {sum(p.numel() for p in self.raw_model.parameters()):,}")
+
+        for epoch in range(self.start_epoch, int(self.config["num_epochs"])):
+            train_stats = self.run_epoch(epoch, train=True)
+            val_stats = self.run_epoch(epoch, train=False)
+            if self.scheduler:
+                self.scheduler.step()
+
+            self.train_history["train_loss"].append(train_stats["total_loss"])
+            self.train_history["train_focal"].append(train_stats["focal_loss"])
+            self.train_history["train_dice"].append(train_stats["dice_loss"])
+            self.train_history["val_loss"].append(val_stats["total_loss"])
+            self.train_history["val_focal"].append(val_stats["focal_loss"])
+            self.train_history["val_dice"].append(val_stats["dice_loss"])
+            self.train_history["val_iou"].append(val_stats["iou"])
+            self.train_history["val_accuracy"].append(val_stats["accuracy"])
+
+            is_best = val_stats["total_loss"] < self.best_val_loss or val_stats["iou"] > self.best_val_iou
+            self.best_val_loss = min(self.best_val_loss, val_stats["total_loss"])
+            self.best_val_iou = max(self.best_val_iou, val_stats["iou"])
+
+            if is_main_process():
+                print(
+                    f"Epoch {epoch + 1}: train_loss={train_stats['total_loss']:.4f}, "
+                    f"val_loss={val_stats['total_loss']:.4f}, val_iou={val_stats['iou']:.4f}"
+                )
+            self.save_checkpoint(epoch, is_best)
+
+            if (epoch + 1) % int(self.config.get("plot_interval", 10)) == 0:
+                self.plot_training_history()
+
+        self.plot_training_history()
+        if is_main_process():
+            print(f"Done. best_val_loss={self.best_val_loss:.4f}, best_val_iou={self.best_val_iou:.4f}")
+
+
+def default_config() -> dict[str, Any]:
+    return {
+        "train_image_dir": "data/train/images",
+        "train_mask_dir": "data/train/masks",
+        "val_image_dir": "data/val/images",
+        "val_mask_dir": "data/val/masks",
+        "num_classes": 2,
+        "backbone_name": "dinov3_vitl16",
+        "pretrained": True,
+        "backbone_weights": "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth",
+        "weight_search_roots": ["D:/DINOv3_pretrained_weights", "D:/DINOv3预训练权重", "dinov3-main"],
+        "use_4channel": True,
+        "image_size": 256,
+        "batch_size": 8,
+        "num_epochs": 100,
+        "backbone_lr": 1e-5,
+        "decoder_lr": 1e-4,
+        "weight_decay": 1e-4,
+        "scheduler": "cosine",
+        "grad_clip": 1.0,
+        "grad_accum_steps": 1,
+        "amp": True,
+        "focal_alpha": [1.0, 3.0],
+        "focal_gamma": 2,
+        "dice_weight": 0.5,
+        "focal_weight": 1.0,
+        "background_weight": 1.0,
+        "foreground_weight": 3.0,
+        "num_workers": 4,
+        "output_dir": f"outputs/seaweed_segmentation_{datetime.now().strftime('%Y%m%d_%H%M%S')}",
+        "plot_interval": 10,
+        "seed": 42,
+        "freeze_backbone": False,
+        "use_data_augmentation": True,
+    }
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Train seaweed segmentation")
+    parser.add_argument("--config", type=str, default=None, help="JSON config path")
+    parser.add_argument("--resume", type=str, default=None, help="Checkpoint to resume")
+    parser.add_argument("--batch-size", type=int, default=None, help="Override per-GPU batch size")
+    parser.add_argument("--epochs", type=int, default=None, help="Override epoch count")
+    parser.add_argument("--output-dir", type=str, default=None, help="Override output directory")
+    parser.add_argument("--weights", type=str, default=None, help="Override DINOv3 weights path")
+    parser.add_argument("--max-train-batches", type=int, default=None, help="Debug limit for train batches per epoch")
+    parser.add_argument("--max-val-batches", type=int, default=None, help="Debug limit for val batches per epoch")
+    return parser.parse_args()
+
+
+def main() -> None:
+    args = parse_args()
+    config = default_config()
+    if args.config:
+        config.update(load_json(args.config))
+    if args.resume:
+        config["resume"] = args.resume
+    if args.batch_size:
+        config["batch_size"] = args.batch_size
+    if args.epochs:
+        config["num_epochs"] = args.epochs
+    if args.output_dir:
+        config["output_dir"] = args.output_dir
+    if args.weights:
+        config["backbone_weights"] = args.weights
+    if args.max_train_batches is not None:
+        config["max_train_batches"] = args.max_train_batches
+    if args.max_val_batches is not None:
+        config["max_val_batches"] = args.max_val_batches
+
+    device, local_rank = setup_distributed()
+    try:
+        trainer = SeaweedSegmentationTrainer(config, device, local_rank)
+        trainer.train()
+    finally:
+        cleanup_distributed()
+
+
+if __name__ == "__main__":
+    main()

train_segmentation_frozen_config.json

@@ -0,0 +1,29 @@
+{
+  "train_image_dir": "data/train/images",
+  "train_mask_dir": "data/train/masks",
+  "val_image_dir": "data/val/images",
+  "val_mask_dir": "data/val/masks",
+  "num_classes": 2,
+  "backbone_name": "dinov3_vitl16",
+  "pretrained": true,
+  "backbone_weights": "dinov3_vitl16_pretrain_sat493m-eadcf0ff.pth",
+  "use_4channel": true,
+  "image_size": 256,
+  "batch_size": 8,
+  "num_epochs": 50,
+  "backbone_lr": 1e-05,
+  "decoder_lr": 5e-05,
+  "weight_decay": 0.0001,
+  "scheduler": "cosine",
+  "grad_clip": 0.5,
+  "focal_alpha": 0.25,
+  "focal_gamma": 2,
+  "dice_weight": 0.5,
+  "focal_weight": 1,
+  "num_workers": 0,
+  "output_dir": "outputs/seaweed_segmentation_frozen",
+  "plot_interval": 10,
+  "device": "cuda",
+  "freeze_backbone": true,
+  "use_data_augmentation": false
+}

vector.py

@@ -0,0 +1,247 @@
+# -*- coding: utf-8 -*-
+"""
+一张大图端到端：瓦片预测 → 拼接 → 矢量化
+"""
+import os
+import json
+import time
+import torch
+import numpy as np
+import cv2
+from pathlib import Path
+from tqdm import tqdm
+import rasterio
+from rasterio.windows import Window
+from osgeo import gdal, ogr, osr
+import warnings
+warnings.filterwarnings('ignore')
+
+import xml.etree.ElementTree as ET
+
+def parse_xml(xml_path: str):
+    """返回 (影像绝对路径, 输出目录绝对路径)"""
+    root = ET.parse(xml_path).getroot()
+    img_node = root.find(".//Input/File_Names/File_Name")
+    out_node = root.find(".//Output/File_Name")
+    if img_node is None or out_node is None:
+        raise RuntimeError("XML 中缺少 Input//File_Name 或 Output//File_Name")
+    img_path = os.path.abspath(img_node.text.strip())
+    out_path = os.path.abspath(out_node.text.strip())
+    return img_path, out_path
+
+
+
+
+
+# ---------- 模型加载 ----------
+def load_model(model_path, config, device):
+    from dinov3_deeplabv3plus import DinoV3DeepLabV3Plus
+    print("加载模型...")
+    model = DinoV3DeepLabV3Plus(
+        num_classes=config['num_classes'],
+        backbone_name=config['backbone_name'],
+        pretrained=False,
+        weights=config['backbone_weights'],
+        use_4channel=config['use_4channel']
+    ).to(device)
+
+    ckpt = torch.load(model_path, map_location=device)
+    model.load_state_dict(ckpt.get('model_state_dict', ckpt))
+    model.eval()
+    return model
+
+
+# ---------- 瓦片预测 ----------
+def predict_tile(model, tile, device, cfg, threshold):
+    with torch.no_grad():
+        out = model(tile.to(device))
+        if isinstance(out, dict):
+            out = out['out']
+        prob = torch.softmax(out, dim=1)[0, 1].cpu().numpy()
+        mask = (prob > threshold).astype(np.uint8)
+    return mask, prob
+
+
+# ---------- 预处理 ----------
+def preprocess_tile(tile_arr, cfg):
+    # tile_arr: HWC float32
+    img = cv2.resize(tile_arr, (cfg['image_size'], cfg['image_size']),
+                     interpolation=cv2.INTER_LINEAR)
+    if img.max() > 1:
+        img = img / 65535.0
+    # 标准化
+    mean = np.array([0.430, 0.411, 0.296, 0.350]) if cfg['use_4channel'] else np.array([0.430, 0.411, 0.296])
+    std  = np.array([0.213, 0.156, 0.143, 0.180]) if cfg['use_4channel'] else np.array([0.213, 0.156, 0.143])
+    for i in range(img.shape[2]):
+        img[:, :, i] = (img[:, :, i] - mean[i]) / std[i]
+    tensor = torch.from_numpy(img).permute(2, 0, 1).float().unsqueeze(0)
+    return tensor
+
+
+# ---------- 主流程 ----------
+def run_one_shot(cfg: dict):
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    # 0. 路径准备
+    tif_path   = Path(cfg['tif_path'])
+    base_name  = tif_path.stem
+    out_dir    = tif_path.parent / f"{base_name}_1"
+    out_dir.mkdir(exist_ok=True)
+
+    # 1. 加载模型 & 配置
+    with open(cfg['config_path'], 'r', encoding='utf-8') as f:
+        config = json.load(f)
+    model = load_model(cfg['model_path'], config, device)
+
+    # 2. 打开大图
+    with rasterio.open(tif_path) as src:
+        h, w, bands = src.height, src.width, src.count
+        transform, crs = src.transform, src.crs
+        print(f"图像尺寸: {w}×{h}  波段: {bands}")
+
+        # 3. 计算瓦片信息
+        tile_size, overlap = cfg['tile_size'], cfg['overlap']
+        stride = tile_size - overlap
+        tiles_x = (w - tile_size) // stride + 1 + ((w - tile_size) % stride != 0)
+        tiles_y = (h - tile_size) // stride + 1 + ((h - tile_size) % stride != 0)
+        total   = tiles_x * tiles_y
+        print(f"瓦片数量: {total} ({tiles_x}×{tiles_y})")
+
+        # 4. 创建输出内存文件
+        pred_driver = gdal.GetDriverByName('MEM')
+        prob_driver = gdal.GetDriverByName('MEM')
+        pred_ds = pred_driver.Create('', w, h, 1, gdal.GDT_Byte)
+        prob_ds = prob_driver.Create('', w, h, 1, gdal.GDT_Float32)
+        for ds in [pred_ds, prob_ds]:
+            ds.SetGeoTransform(transform.to_gdal())
+            ds.SetProjection(crs.to_wkt())
+
+        pred_band = pred_ds.GetRasterBand(1)
+        prob_band = prob_ds.GetRasterBand(1)
+        count = np.zeros((h, w), dtype=np.uint16)
+
+        # 5. 分块预测
+        tile_id = 0
+        for y in range(0, h - tile_size + 1, stride):
+            for x in range(0, w - tile_size + 1, stride):
+                actual_x, actual_y = min(x, w - tile_size), min(y, h - tile_size)
+                window = Window(actual_x, actual_y, tile_size, tile_size)
+                tile = src.read(window=window)                      # CHW
+                tile = np.transpose(tile, (1, 2, 0))                # HWC
+                if tile.shape[2] >= 4:
+                    tile = tile[:, :, :4]
+                else:
+                    tile = tile[:, :, [2, 1, 0]] if tile.shape[2] >= 3 else tile[:, :, :3]
+
+                tensor = preprocess_tile(tile.astype(np.float32), config)
+                mask, prob = predict_tile(model, tensor, device, config, cfg['threshold'])
+
+                # 写入内存
+                pred_band.WriteArray(mask, actual_x, actual_y)
+                prob_band.WriteArray(prob, actual_x, actual_y)
+                count[actual_y:actual_y+tile_size, actual_x:actual_x+tile_size] += 1
+                tile_id += 1
+                if tile_id % 500 == 0:
+                    print(f"  已预测 {tile_id}/{total}")
+
+        # 6. 平均化重叠区域
+        count[count == 0] = 1
+        pred_final = (pred_band.ReadAsArray().astype(np.float32) / count).round().astype(np.uint8)
+        prob_final = prob_band.ReadAsArray() / count
+
+        # 7. 保存 GeoTIFF
+        gtiff_path = out_dir / f"{base_name}_prediction.tif"
+        drv = gdal.GetDriverByName('GTiff')
+        ds_out = drv.Create(str(gtiff_path), w, h, 2, gdal.GDT_Byte,
+                            options=['COMPRESS=DEFLATE', 'TILED=YES'])
+        ds_out.SetGeoTransform(transform.to_gdal())
+        ds_out.SetProjection(crs.to_wkt())
+        # band1: 二值掩膜
+        b1 = ds_out.GetRasterBand(1)
+        b1.WriteArray(pred_final * 255)
+        b1.SetNoDataValue(0)
+        # band2: 概率 [0-255]
+        b2 = ds_out.GetRasterBand(2)
+        b2.WriteArray((prob_final * 255).astype(np.uint8))
+        ds_out = None
+        print(f"✅ GeoTIFF 已保存: {gtiff_path}")
+
+        # 8. 矢量化
+        shp_path = out_dir / f"{base_name}_sargassum.shp"
+        vectorize(gtiff_path, shp_path, cfg['sieve_pixels'], cfg['min_area'])
+        print(f"✅ 矢量文件已保存: {shp_path}")
+
+    print("🎉 全部完成！")
+
+
+# ---------- 矢量化 ----------
+def vectorize(tif_path: Path, shp_path: Path, sieve_pixels: int, min_area_m2: float):
+    ds = gdal.Open(str(tif_path), gdal.GA_ReadOnly)
+    band = ds.GetRasterBand(1)
+    data = band.ReadAsArray()
+    geo = ds.GetGeoTransform()
+    proj = ds.GetProjection()
+
+    # 碎斑过滤
+    if sieve_pixels > 0:
+        kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (sieve_pixels//2, sieve_pixels//2))
+        data = cv2.morphologyEx(data, cv2.MORPH_OPEN, kernel)
+
+    mask = (data == 255).astype(np.uint8)
+    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
+
+    driver = ogr.GetDriverByName('ESRI Shapefile')
+    if shp_path.exists():
+        driver.DeleteDataSource(str(shp_path))
+    out_ds = driver.CreateDataSource(str(shp_path))
+    srs = osr.SpatialReference()
+    srs.ImportFromWkt(proj)
+    layer = out_ds.CreateLayer("sargassum", srs, ogr.wkbPolygon)
+    fd = ogr.FieldDefn('DN', ogr.OFTInteger)
+    layer.CreateField(fd)
+
+    pixel_area = abs(geo[1] * geo[5])
+    for cnt in contours:
+        area_pixel = cv2.contourArea(cnt)
+        if area_pixel * pixel_area < min_area_m2:
+            continue
+        ring = ogr.Geometry(ogr.wkbLinearRing)
+        for pt in cnt[:, 0, :]:
+            x = geo[0] + pt[0] * geo[1] + pt[1] * geo[2]
+            y = geo[3] + pt[0] * geo[4] + pt[1] * geo[5]
+            ring.AddPoint(x, y)
+        ring.CloseRings()
+        poly = ogr.Geometry(ogr.wkbPolygon)
+        poly.AddGeometry(ring)
+        feat = ogr.Feature(layer.GetLayerDefn())
+        feat.SetGeometry(poly)
+        feat.SetField('DN', 255)
+        layer.CreateFeature(feat)
+    out_ds = None
+    ds = None
+
+
+# ---------- 入口 ----------
+if __name__ == "__main__":
+    # 1. 读 XML（唯一需要改的地方）
+    xml_file = r"test.xml"          # 也可 sys.argv[1] 传入
+    tif_path, out_root = parse_xml(xml_file)
+
+    # 2. 组装配置字典（其余逻辑零改动）
+    base_name = Path(tif_path).stem
+    out_dir   = Path(out_root) / f"{base_name}_1"   # F:/Results/admin/GF1/157/xxx_1
+    out_dir.mkdir(parents=True, exist_ok=True)
+
+    MAIN_CFG = {
+        "tif_path"     : str(tif_path),
+        "model_path"   : r"seaweed_segmentation_improved_epoch500\best_checkpoint.pth",
+        "config_path"  : r"seaweed_segmentation_improved_epoch500\config.json",
+        "tile_size"    : 256,
+        "overlap"      : 64,
+        "threshold"    : 0.5,
+        "sieve_pixels" : 10,
+        "min_area"     : 0.0,
+    }
+
+    # 3. 跑流程
+    run_one_shot(MAIN_CFG)