README.md

---
license: mit
library_name: pytorch
pipeline_tag: image-segmentation
tags:
- semantic-segmentation
- microstructure
- materials-science
- segment-anything
- lora
- matsam
---

# MatSAM Fine-Tuned Checkpoints for Microstructure Segmentation

## Model Description

This repository contains dataset-specific fine-tuned parameter checkpoints for
MatSAM with a ViT-H image encoder. The files do not contain the full base
model. Each checkpoint must be loaded together with the official SAM ViT-H
checkpoint `sam_vit_h_4b8939.pth` using the matching test script.

Source code:
https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis

## Fine-Tuning Routes and Checkpoints

| Dataset | Route | Classes | File |
| --- | --- | ---: | --- |
| Aachen-Heerlen | LoRA plus mask decoder | 2 | `checkpoints/Aachen-Heerlen/matsam_lora_decoder_best.pth` |
| EMPS | LoRA plus mask decoder | 2 | `checkpoints/EMPS/matsam_lora_decoder_best.pth` |
| Grain | LoRA plus mask decoder | 2 | `checkpoints/Grain/matsam_lora_decoder_best.pth` |
| EBC | LoRA, mask decoder, and class tokens | 3 | `checkpoints/EBC/semantic_matsam_best.pth` |
| Super | LoRA, mask decoder, and class tokens | 3 | `checkpoints/Super/semantic_matsam_best.pth` |
| MetalDAM | LoRA, mask decoder, and class tokens | 5 | `checkpoints/MetalDAM/semantic_matsam_best.pth` |
| UHCS | LoRA, mask decoder, and class tokens | 7 | `checkpoints/UHCS/semantic_matsam_best.pth` |

Class counts include background.

## Base Model Requirement

Download the SAM ViT-H base checkpoint from Meta:
https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

The fine-tuned files contain LoRA and mask-decoder parameters. Multiclass files
also contain learned class-token parameters. They are not standalone weights.

## Training Details

- Base architecture: ViT-H
- SAM encoder input: longest side resized and padded to 1024 x 1024
- Training batch size: 1
- Epochs: 200
- Optimizer: AdamW
- Learning rate: 0.0001
- Weight decay: 0.0001
- Scheduler: CosineAnnealingLR with minimum learning rate 0.000001
- Early-stopping patience: 50
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets: `qkv` and `proj`
- Binary objective: BCE, Dice, and IoU MSE
- Multiclass objective: cross-entropy and Dice

## Usage

Example for binary EMPS segmentation:

```bash
hf download NAMESPACE/microstructure-matsam checkpoints/EMPS/matsam_lora_decoder_best.pth --local-dir weights/matsam
python matsam/test_lora_decoder.py \
  --dataset-root /path/to/EMPS \
  --checkpoint weights/pretrained/sam_vit_h_4b8939.pth \
  --finetuned-checkpoint weights/matsam/checkpoints/EMPS/matsam_lora_decoder_best.pth \
  --output-dir results/matsam_emps \
  --method-type 2
```

For Grain, use `--method-type 1`. For other metallographic datasets, use
`--method-type 2`. For multiclass checkpoints, use
`matsam/test_semantic_matsam.py` and pass the class count from the table.

## Inference Parameters

The original automatic-mask path uses `n_per_side_base=54`,
`pred_iou_thresh=0.88`, `stability_score_thresh=0.9`,
`box_nms_thresh=0.7`, `crop_nms_thresh=0.7`, and `crop_n_layers=0`. The binary
fine-tuned path uses point prompts and thresholds of 0.7 for predicted IoU and
stability score.

## Evaluation, Intended Use, and Limitations

Evaluation restores masks to their original image dimensions and calculates
the common benchmark metrics through `Myutils/metrics.py`. These checkpoints
are intended for research reproduction on the named datasets. They require the
exact ViT-H base architecture and matching script. They may not generalize to
unseen materials or imaging conditions and are not validated for safety-critical
or industrial quality-control decisions.

## License and Citation

The released experiment files are provided under the MIT license. The SAM base
checkpoint and MatSAM implementation remain subject to their upstream terms.
The base checkpoint is not redistributed here. A paper citation will be added
after publication.