---
license: cc-by-nc-4.0
task_categories:
  - image-segmentation
tags:
  - glass-surface-detection
  - semantic-segmentation
  - scene-understanding
  - pytorch
pretty_name: GlassSemNet (Glass Semantic Network)
---

# GlassSemNet — Glass Semantic Network

Pre-trained weights for **GlassSemNet**, introduced in:

> **Exploiting Semantic Relations for Glass Surface Detection**  
> Jiaying Lin, Yuen-Hei Yeung, Rynson W. H. Lau  
> NeurIPS 2022  
> [Paper](https://openreview.net/forum?id=WrIrYMCZgbb) · [Project Page](https://jiaying.link/neurips2022-gsds/) · [Dataset (GSD-S)](https://huggingface.co/datasets/garrying/GSD-S)

## Model Summary

GlassSemNet detects glass surfaces by exploiting semantic relations between the glass region and its surrounding scene context. It uses a dual-backbone design:

- **Spatial backbone (SegFormer)**: extracts multi-scale spatial features.
- **Semantic backbone (ResNet-50 + DeepLabV3+)**: encodes 43-class semantic scene features into compact per-class encodings.
- **Semantic-Aware Attention (SAA)**: fuses spatial and semantic features at three scales using the semantic encodings as guidance.
- **Cross-modal Context Aggregation (CCA)**: aggregates cross-scale context at the deepest level.
- **UPerNet decoder**: combines the fused multi-scale features into the final glass surface prediction.

| File | Description |
|------|-------------|
| `GlassSemNet.pth` | Best checkpoint (917 MB), saved as a raw `state_dict` |

## Loading the Weights

```python
import torch
from model.GlassSemNet import GlassSemNet   # from the code release

model = GlassSemNet()
state_dict = torch.load("GlassSemNet.pth", map_location="cpu")
model.load_state_dict(state_dict)
model.eval()
```

Download the checkpoint:
```bash
huggingface-cli download garrying/GlassSemNet GlassSemNet.pth --local-dir ./weights
```

## Inference

```bash
python predict.py -c GlassSemNet.pth -i /path/to/images/ -o /path/to/output/
```

Images are resized to **384 × 384** internally. Predictions are post-processed with CRF refinement and thresholded to produce binary glass surface masks.

## Training Dataset

This model was trained and evaluated on **GSD-S**, the first glass surface detection dataset with semantic annotations:

- 4,519 images (3,511 train / 1,008 test) with binary glass masks, instance segmentation maps, and 43-class semantic labels
- Available at [garrying/GSD-S](https://huggingface.co/datasets/garrying/GSD-S)

## Citation

```bibtex
@article{neurips2022:gsds2022,
  author    = {Lin, Jiaying and Yeung, Yuen-Hei and Lau, Rynson W.H.},
  title     = {Exploiting Semantic Relations for Glass Surface Detection},
  journal   = {NeurIPS},
  year      = {2022},
}
```

## License

Non-commercial use only — [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/).