README.md

---
license: mit
library_name: pytorch
pipeline_tag: image-segmentation
tags:
- semantic-segmentation
- microstructure
- materials-science
- segment-anything-2
- lora
- sam2
---

# SAM2 Fine-Tuned Checkpoints for Microstructure Segmentation

## Model Description

This repository contains dataset-specific fine-tuned parameter checkpoints for
SAM2.1 Hiera Base+. The files are partial checkpoints and must be loaded with
the exact `sam2.1_hiera_base_plus.pt` base checkpoint and
`configs/sam2.1/sam2.1_hiera_b+.yaml`.

Source code:
https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis

## Fine-Tuning Routes and Checkpoints

| Dataset | Route | Classes | File |
| --- | --- | ---: | --- |
| Aachen-Heerlen | LoRA plus mask decoder | 2 | `checkpoints/Aachen-Heerlen/sam2_lora_decoder_best.pth` |
| EMPS | LoRA plus mask decoder | 2 | `checkpoints/EMPS/sam2_lora_decoder_best.pth` |
| Grain | LoRA plus mask decoder | 2 | `checkpoints/Grain/sam2_lora_decoder_best.pth` |
| EBC | LoRA, mask decoder, and class tokens | 3 | `checkpoints/EBC/semantic_sam2_best.pth` |
| Super | LoRA, mask decoder, and class tokens | 3 | `checkpoints/Super/semantic_sam2_best.pth` |
| MetalDAM | LoRA, mask decoder, and class tokens | 5 | `checkpoints/MetalDAM/semantic_sam2_best.pth` |
| UHCS | LoRA, mask decoder, and class tokens | 7 | `checkpoints/UHCS/semantic_sam2_best.pth` |

Class counts include background.

## Base Model Requirement

Download the official SAM2.1 Hiera Base+ checkpoint from:
https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_base_plus.pt

Do not substitute another SAM2 architecture without changing the model config.

## Training Details

- Base architecture: SAM2.1 Hiera Base+
- Input preprocessing: longest side resized to 1024, padded to 1024 x 1024
- Training batch size: 1
- Epochs: 200
- Optimizer: AdamW
- Learning rate: 0.0001
- Weight decay: 0.0001
- Scheduler: CosineAnnealingLR with minimum learning rate 0.000001
- Early-stopping patience: 50
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets: `qkv` and `proj`
- Binary objective: BCE, Dice, and IoU MSE
- Multiclass objective: cross-entropy and Dice

## Usage

Example for binary EMPS segmentation:

```bash
hf download NAMESPACE/microstructure-sam2 checkpoints/EMPS/sam2_lora_decoder_best.pth --local-dir weights/sam2
python sam2/test_lora_decoder.py \
  --dataset-root /path/to/EMPS \
  --checkpoint weights/pretrained/sam2.1_hiera_base_plus.pt \
  --finetuned-checkpoint weights/sam2/checkpoints/EMPS/sam2_lora_decoder_best.pth \
  --output-dir results/sam2_emps
```

For multiclass checkpoints, use `sam2/test_semantic_sam2.py` and pass the
class count from the table.

## Inference Parameters

Automatic mask generation uses `points_per_side=32`, `points_per_batch=64`,
`pred_iou_thresh=0.8`, `stability_score_thresh=0.8`, `box_nms_thresh=0.7`, and
`crop_n_layers=0`. Padded regions are removed before predictions are restored
to the original image size.

## Evaluation, Intended Use, and Limitations

Evaluation saves original-resolution masks and calculates the common benchmark
metrics through `Myutils/metrics.py`. These checkpoints require the exact
SAM2.1 Hiera Base+ base model and matching source code. They are intended for
research reproduction and may not generalize to unseen materials or imaging
conditions. They are not validated for safety-critical or industrial
quality-control decisions.

## License and Citation

The released experiment files are provided under the MIT license. SAM2 and its
base checkpoint remain subject to Meta's upstream terms and are not
redistributed here. A paper citation will be added after publication.