README.md

---
license: mit
library_name: pytorch
pipeline_tag: image-segmentation
tags:
- semantic-segmentation
- microstructure
- materials-science
- segment-anything
- lora
- hq-sam
---

# HQ-SAM Fine-Tuned Checkpoints for Microstructure Segmentation

## Model Description

This repository contains dataset-specific fine-tuned parameter checkpoints for
HQ-SAM with its ViT-B backbone. The files do not contain the full base model.
Load them together with `sam_hq_vit_b.pth` using the matching source script.

Source code:
https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis

## Fine-Tuning Routes and Checkpoints

| Dataset | Route | Classes | File |
| --- | --- | ---: | --- |
| Aachen-Heerlen | LoRA plus mask decoder | 2 | `checkpoints/Aachen-Heerlen/hqsam_lora_decoder_best.pth` |
| EMPS | LoRA plus mask decoder | 2 | `checkpoints/EMPS/hqsam_lora_decoder_best.pth` |
| Grain | LoRA plus mask decoder | 2 | `checkpoints/Grain/hqsam_lora_decoder_best.pth` |
| EBC | LoRA, mask decoder, and class tokens | 3 | `checkpoints/EBC/semantic_hqsam_best.pth` |
| Super | LoRA, mask decoder, and class tokens | 3 | `checkpoints/Super/semantic_hqsam_best.pth` |
| MetalDAM | LoRA, mask decoder, and class tokens | 5 | `checkpoints/MetalDAM/semantic_hqsam_best.pth` |
| UHCS | LoRA, mask decoder, and class tokens | 7 | `checkpoints/UHCS/semantic_hqsam_best.pth` |

Class counts include background.

## Base Model Requirement

Download the official HQ-SAM ViT-B checkpoint from:
https://huggingface.co/lkeab/hq-sam/resolve/main/sam_hq_vit_b.pth

The files in this repository contain LoRA and mask-decoder parameters.
Multiclass files additionally contain learned class tokens.

## Training Details

- Base architecture: HQ-SAM ViT-B
- SAM encoder input: longest side resized and padded to 1024 x 1024
- Training batch size: 1
- Epochs: 200
- Optimizer: AdamW
- Learning rate: 0.0001
- Weight decay: 0.0001
- Scheduler: CosineAnnealingLR with minimum learning rate 0.000001
- Early-stopping patience: 50
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets: `qkv` and `proj`
- Binary objective: BCE, Dice, and IoU MSE
- Multiclass objective: cross-entropy and Dice

## Usage

Example for binary EMPS segmentation:

```bash
hf download NAMESPACE/microstructure-hq-sam checkpoints/EMPS/hqsam_lora_decoder_best.pth --local-dir weights/hq-sam
python hqsam/test_lora_decoder.py \
  --dataset-root /path/to/EMPS \
  --checkpoint weights/pretrained/sam_hq_vit_b.pth \
  --finetuned-checkpoint weights/hq-sam/checkpoints/EMPS/hqsam_lora_decoder_best.pth \
  --output-dir results/hq_sam_emps
```

For multiclass checkpoints, use `hqsam/test_semantic_hqsam.py` and pass the
class count from the table.

## Inference Parameters

The original automatic-mask path uses `points_per_side=32`,
`points_per_batch=64`, `pred_iou_thresh=0.85`,
`stability_score_thresh=0.8`, `box_nms_thresh=0.7`, and `crop_n_layers=0`. The
binary fine-tuned path uses `pred_iou_thresh=0.78` and
`stability_score_thresh=0.8`.

## Evaluation, Intended Use, and Limitations

Evaluation saves original-resolution masks and calculates the common benchmark
metrics through `Myutils/metrics.py`. These checkpoints are intended for
research reproduction on the named datasets. They require the matching HQ-SAM
ViT-B base checkpoint and may not generalize to unseen materials or imaging
conditions. They are not validated for safety-critical or industrial
quality-control decisions.

## License and Citation

The released experiment files are provided under the MIT license. HQ-SAM and
its base checkpoint remain subject to upstream terms and are not redistributed
here. A paper citation will be added after publication.