File size: 3,715 Bytes
90f068f dd98b72 90f068f dd98b72 90f068f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 | ---
license: mit
library_name: pytorch
pipeline_tag: image-segmentation
tags:
- semantic-segmentation
- microstructure
- materials-science
- segment-anything-2
- lora
- sam2
---
# SAM2 Fine-Tuned Checkpoints for Microstructure Segmentation
## Model Description
This repository contains dataset-specific fine-tuned parameter checkpoints for
SAM2.1 Hiera Base+. The files are partial checkpoints and must be loaded with
the exact `sam2.1_hiera_base_plus.pt` base checkpoint and
`configs/sam2.1/sam2.1_hiera_b+.yaml`.
Source code:
https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis
## Fine-Tuning Routes and Checkpoints
| Dataset | Route | Classes | File |
| --- | --- | ---: | --- |
| Aachen-Heerlen | LoRA plus mask decoder | 2 | `checkpoints/Aachen-Heerlen/sam2_lora_decoder_best.pth` |
| EMPS | LoRA plus mask decoder | 2 | `checkpoints/EMPS/sam2_lora_decoder_best.pth` |
| Grain | LoRA plus mask decoder | 2 | `checkpoints/Grain/sam2_lora_decoder_best.pth` |
| EBC | LoRA, mask decoder, and class tokens | 3 | `checkpoints/EBC/semantic_sam2_best.pth` |
| Super | LoRA, mask decoder, and class tokens | 3 | `checkpoints/Super/semantic_sam2_best.pth` |
| MetalDAM | LoRA, mask decoder, and class tokens | 5 | `checkpoints/MetalDAM/semantic_sam2_best.pth` |
| UHCS | LoRA, mask decoder, and class tokens | 7 | `checkpoints/UHCS/semantic_sam2_best.pth` |
Class counts include background.
## Base Model Requirement
Download the official SAM2.1 Hiera Base+ checkpoint from:
https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_base_plus.pt
Do not substitute another SAM2 architecture without changing the model config.
## Training Details
- Base architecture: SAM2.1 Hiera Base+
- Input preprocessing: longest side resized to 1024, padded to 1024 x 1024
- Training batch size: 1
- Epochs: 200
- Optimizer: AdamW
- Learning rate: 0.0001
- Weight decay: 0.0001
- Scheduler: CosineAnnealingLR with minimum learning rate 0.000001
- Early-stopping patience: 50
- LoRA rank: 8
- LoRA alpha: 16
- LoRA dropout: 0.05
- LoRA targets: `qkv` and `proj`
- Binary objective: BCE, Dice, and IoU MSE
- Multiclass objective: cross-entropy and Dice
## Usage
Example for binary EMPS segmentation:
```bash
hf download NAMESPACE/microstructure-sam2 checkpoints/EMPS/sam2_lora_decoder_best.pth --local-dir weights/sam2
python sam2/test_lora_decoder.py \
--dataset-root /path/to/EMPS \
--checkpoint weights/pretrained/sam2.1_hiera_base_plus.pt \
--finetuned-checkpoint weights/sam2/checkpoints/EMPS/sam2_lora_decoder_best.pth \
--output-dir results/sam2_emps
```
For multiclass checkpoints, use `sam2/test_semantic_sam2.py` and pass the
class count from the table.
## Inference Parameters
Automatic mask generation uses `points_per_side=32`, `points_per_batch=64`,
`pred_iou_thresh=0.8`, `stability_score_thresh=0.8`, `box_nms_thresh=0.7`, and
`crop_n_layers=0`. Padded regions are removed before predictions are restored
to the original image size.
## Evaluation, Intended Use, and Limitations
Evaluation saves original-resolution masks and calculates the common benchmark
metrics through `Myutils/metrics.py`. These checkpoints require the exact
SAM2.1 Hiera Base+ base model and matching source code. They are intended for
research reproduction and may not generalize to unseen materials or imaging
conditions. They are not validated for safety-critical or industrial
quality-control decisions.
## License and Citation
The released experiment files are provided under the MIT license. SAM2 and its
base checkpoint remain subject to Meta's upstream terms and are not
redistributed here. A paper citation will be added after publication.
|