# SFR-Net
English | ็ฎไฝไธญๆ
Learning Scale-Frustum Representations for Ultra-Wide Area
Remote Sensing Image Segmentation
## Overview ๐งญ
SFR-Net is designed for semantic segmentation of ultra-wide area (UWA) remote sensing images, where both the pixel count and geographical coverage are extremely large. It constructs aligned local, short-range, and long-range observations around the same Projection Reference Point (PRP), resizes them to a unified input size, and distinguishes them with learnable scale embeddings. A Cascaded Cross-Scale Fusion (CCSF) module then injects contextual information into the local representation progressively, preserving fine details while improving long-range semantic continuity.
## News ๐ฐ
- **2026-08-26:** We updated the codebase, fixed known bugs, improved the inference, testing, and visualization scripts, and released trained weights for GID, FBPS, and Inria Aerial.
- **2026-07-11:** We received the first-round review decision from IEEE Transactions on Geoscience and Remote Sensing (IEEE TGRS), and the manuscript was invited for major revision.
- **2026-05-25:** Our paper, [โSFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentationโ](https://arxiv.org/abs/2605.25737), was released on arXiv.
- **2026-05-20:** Our paper, โSFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentation,โ was submitted to IEEE TGRS.
- **2026-05-11:** We released the initial code version with training and testing scripts and pretrained weights.
## Highlights โจ
- We formulate ultra-wide area remote sensing image segmentation as a task that jointly considers large pixel counts, extremely wide geographical coverage, significantly varying object scales, and long-range semantic continuity.
- Scale-Frustum Representations unify local, short-range, and long-range observations around the same PRP. The released GID/FBPS configs use distances `[1, 3, 14]`, while the Inria Aerial config uses `[1, 3, 10]`.
- Learnable scale embeddings explicitly identify resized observations from different spatial ranges.
- The CCSF module progressively introduces nearby and broader contextual cues into detailed local features.
- SFR-Net achieves state-of-the-art results on the UWA GID and FBPS benchmarks. The SFR representation can also improve the accuracy and convergence speed of generic segmentation networks.
## Performance ๐
The following table is taken from the paper. SFR-Net reaches `74.67%` mIoU on GID and `77.24%` mIoU on FBPS in the paper setting.
## Repository Layout ๐๏ธ
```text
SFR-Net/
โโโ configs/
โ โโโ _base_/
โ โ โโโ datasets/
โ โ โโโ schedules/
โ โ โโโ default_runtime.py
โ โโโ gid/sfrnet_swinl_320k_gid.py
โ โโโ fbps/sfrnet_swinl_320k_fbps.py
โ โโโ inria_aerial/sfrnet_swinl_320k_inria_aerial.py
โโโ mmseg/
โ โโโ datasets/transforms/sfr_loading.py
โ โโโ datasets/uwa_dataset.py
โ โโโ models/backbones/sfr_net.py
โ โโโ models/necks/ccsf_neck.py
โโโ tools/
โ โโโ train.py
โ โโโ test.py
โ โโโ sfr_inference.py
โ โโโ get_res_iou.py
โ โโโ visualizer.py
โโโ pics/
โโโ pretrain/
โโโ weights/
โโโ README.md
โโโ README_zh-CN.md
```
The release keeps the default SFR-Net pathway and the GID, FBPS, and Inria Aerial configurations. Multi-distance ablations and other experimental-only modules are intentionally excluded.
## Weights ๐
All pretrained backbones and released SFR-Net checkpoints are hosted in the [SFR-Net Hugging Face repository](https://huggingface.co/shadowwalk/SFR-Net).
### Available files
| Type | File | Expected location |
| ----------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
| ResNet-18 ImageNet pretraining | [`resnet18_v1c-b5776b93.pth`](https://huggingface.co/shadowwalk/SFR-Net/blob/main/pretrain/resnet18_v1c-b5776b93.pth) | `pretrain/resnet18_v1c-b5776b93.pth` |
| Swin-Large ImageNet-22K pretraining | [`swin_large_patch4_window12_384_22k_20220412-6580f57d.pth`](https://huggingface.co/shadowwalk/SFR-Net/blob/main/pretrain/swin_large_patch4_window12_384_22k_20220412-6580f57d.pth) | `pretrain/swin_large_patch4_window12_384_22k_20220412-6580f57d.pth` |
| GID checkpoint | [`iter_320000_gid.pth`](https://huggingface.co/shadowwalk/SFR-Net/blob/main/weights/iter_320000_gid.pth) | `weights/iter_320000_gid.pth` |
| FBPS checkpoint | [`iter_320000_fbps.pth`](https://huggingface.co/shadowwalk/SFR-Net/blob/main/weights/iter_320000_fbps.pth) | `weights/iter_320000_fbps.pth` |
| Inria Aerial checkpoint | [`iter_320000_inria.pth`](https://huggingface.co/shadowwalk/SFR-Net/blob/main/weights/iter_320000_inria.pth) | `weights/iter_320000_inria.pth` |
You can download the files with the Hugging Face CLI:
```bash
pip install -U huggingface_hub
hf download shadowwalk/SFR-Net --local-dir downloads/SFR-Net
cp -r downloads/SFR-Net/pretrain/. pretrain/
cp -r downloads/SFR-Net/weights/. weights/
```
### Released checkpoint results
| Dataset | OA (%) | mIoU (%) | mF1 (%) | Checkpoint |
| ------------ | -----: | -------: | ------: | ------------------------------- |
| GID | 86.82 | 74.46 | 85.73 | `weights/iter_320000_gid.pth` |
| FBPS | 93.50 | 77.86 | 66.72 | `weights/iter_320000_fbps.pth` |
| Inria Aerial | 96.91 | 83.96* | 91.28* | `weights/iter_320000_inria.pth` |
`*` For Inria Aerial, IoU and F1 are reported for the building class only. The released checkpoints were trained with random seed `42`; their results therefore differ slightly from the values reported in the paper.
The backbone paths are currently defined in `mmseg/models/backbones/sfr_net.py`. No code change is required if the two pretrained files are kept under `pretrain/` and commands are executed from the repository root.
## Installation ๐ ๏ธ
Create an environment with a PyTorch/CUDA combination suitable for your GPU, then install SFR-Net from the repository root:
```bash
conda create -n sfrnet python=3.10 -y
conda activate sfrnet
# Install PyTorch first according to https://pytorch.org/get-started/locally/
pip install -U openmim
mim install mmengine "mmcv>=2.0.0"
pip install -r requirements.txt
pip install -v -e .
pip install mxnet
```
`mxnet` is used by `tools/sfr_inference.py` to read the original ultra-wide images.
## Data Preparation ๐๏ธ
Official dataset pages:
| Dataset | Website |
| ------------ | ------------------------------------------------------------ |
| GID | [Gaofen Image Dataset](https://x-ytong.github.io/project/GID) |
| FBPS | [Five-Billion-Pixels](https://x-ytong.github.io/project/Five-Billion-Pixels.html) |
| Inria Aerial | [Inria Aerial Image Labeling Dataset](https://project.inria.fr/aerialimagelabeling/) |
Organize the datasets as follows:
```text
SFR-Net/
โโโ data/
โโโ GID/
โ โโโ Image_train/
โ โโโ Image_test/
โ โโโ annos_train_5l/
โ โโโ annos_test_5l/
โ โโโ annos_train_24l/
โ โโโ annos_test_24l/
โโโ inria_aerial/
โโโ images/
โ โโโ train/
โ โโโ val/
โ โโโ test/
โโโ Label/
โโโ train/
โโโ val/
โโโ test/
```
GID and FBPS use the same GF-2 images but different label folders. GID uses the 5-category annotations and produces 6 class indices including background; FBPS uses the 24-category annotations and produces 25 class indices including background. Inria Aerial uses two class indices: background and building.
The released configs still contain the original local absolute paths. Before training or validation, update these three files:
```python
# configs/_base_/datasets/gid.py
data_root = 'data/GID'
# configs/_base_/datasets/fbps.py
data_root = 'data/GID'
# configs/_base_/datasets/inria_aerial.py
data_root = 'data/inria_aerial'
```
Alternatively, keep the datasets elsewhere and set each `data_root` to the corresponding absolute path. The folder names below `data_root` must still match the structure shown above.
## Training ๐๏ธ
Before training:
1. Set `data_root` in the appropriate file under `configs/_base_/datasets/` as described in Data Preparation.
2. Check `batch_size` and `num_workers` in the selected experiment config. The released configs use batch size `4` and override `num_workers` to `64`; reduce them if your GPU memory or CPU resources are limited.
3. Keep the two backbone checkpoints under `pretrain/`, or update `depth2ckpt` in `mmseg/models/backbones/sfr_net.py` if you use different locations.
Train with random seed `42` (the default in `configs/_base_/default_runtime.py` and `tools/train.py`):
```bash
python tools/train.py configs/gid/sfrnet_swinl_320k_gid.py \
--work-dir work_dirs/gid
python tools/train.py configs/fbps/sfrnet_swinl_320k_fbps.py \
--work-dir work_dirs/fbps
python tools/train.py configs/inria_aerial/sfrnet_swinl_320k_inria_aerial.py \
--work-dir work_dirs/inria_aerial
```
Add `--amp` to enable automatic mixed precision. Use `--resume` with the same `--work-dir` to continue from its latest checkpoint.
## Inference ๐ฐ๏ธ
`tools/sfr_inference.py` contains original-machine defaults in the `DATASETS` dictionary, including `/mnt/dataset/zhongchuyu/...`. Either replace the `src` entries with `data/GID/Image_test` and `data/inria_aerial/images/test`, or pass `--src` explicitly as shown below. Command-line values take precedence over those defaults.
```bash
python tools/sfr_inference.py \
--dataset gid \
--src data/GID/Image_test \
--dst work_dirs/gid_predictions \
--config configs/gid/sfrnet_swinl_320k_gid.py \
--ckpt weights/iter_320000_gid.pth \
--stride 128
python tools/sfr_inference.py \
--dataset fbps \
--src data/GID/Image_test \
--dst work_dirs/fbps_predictions \
--config configs/fbps/sfrnet_swinl_320k_fbps.py \
--ckpt weights/iter_320000_fbps.pth \
--stride 128
python tools/sfr_inference.py \
--dataset inria_aerial \
--src data/inria_aerial/images/test \
--dst work_dirs/inria_aerial_predictions \
--config configs/inria_aerial/sfrnet_swinl_320k_inria_aerial.py \
--ckpt weights/iter_320000_inria.pth \
--stride 128
```
The default `--load-type random` builds the complete scale-frustum representation. Predictions are saved as single-channel class-index PNG masks.
## Metrics and Visualization ๐จ
### Metrics
`tools/get_res_iou.py` currently stores the original ground-truth paths in its `DATASETS` dictionary and does not provide a `--gt` argument. Update that dictionary before evaluation:
```python
DATASETS = {
'gid': ('data/GID/annos_test_5l', 6),
'fbps': ('data/GID/annos_test_24l', 25),
'inria_aerial': ('data/inria_aerial/Label/test', 2),
}
```
Then compute the metrics:
```bash
python tools/get_res_iou.py --dataset gid \
--pred work_dirs/gid_predictions
python tools/get_res_iou.py --dataset fbps \
--pred work_dirs/fbps_predictions
python tools/get_res_iou.py --dataset inria_aerial \
--pred work_dirs/inria_aerial_predictions
```
### Visualization
`tools/visualizer.py` has no fixed dataset path; provide the input and output directories on the command line. Its `PALETTES` dictionary contains the GID, FBPS, and Inria Aerial color maps and only needs modification if your class-index convention changes.
```bash
python tools/visualizer.py --dataset gid \
--src work_dirs/gid_predictions \
--dst work_dirs/gid_visualizations
python tools/visualizer.py --dataset fbps \
--src work_dirs/fbps_predictions \
--dst work_dirs/fbps_visualizations
python tools/visualizer.py --dataset inria_aerial \
--src work_dirs/inria_aerial_predictions \
--dst work_dirs/inria_aerial_visualizations
```
## Contact โ๏ธ
If you find this work useful, please cite our [paper](https://arxiv.org/abs/2605.25737):
```bibtex
@article{zhong2026sfr,
title={SFR-Net: Learning Scale-Frustum Representations for Ultra-Wide Area Remote Sensing Image Segmentation},
author={Zhong, Chuyu and Chen, Keyan and Yang, Qinzhe and Chen, Bowen and Zou, Zhengxia and Shi, Zhenwei},
journal={arXiv preprint arXiv:2605.25737},
year={2026}
}
```
Questions and bug reports are welcome at **buaazcy@buaa.edu.cn**.
If you find this repository helpful, please give it a star. Finally, here is Phoebe. You are not allowed to bully her.
