README.md

metadata

license: mit
library_name: pytorch
pipeline_tag: image-segmentation
tags:
  - semantic-segmentation
  - microstructure
  - materials-science
  - segment-anything
  - lora
  - matsam

MatSAM Fine-Tuned Checkpoints for Microstructure Segmentation

Model Description

This repository contains dataset-specific fine-tuned parameter checkpoints for MatSAM with a ViT-H image encoder. The files do not contain the full base model. Each checkpoint must be loaded together with the official SAM ViT-H checkpoint sam_vit_h_4b8939.pth using the matching test script.

Source code: https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis

Fine-Tuning Routes and Checkpoints

Dataset	Route	Classes	File
Aachen-Heerlen	LoRA plus mask decoder	2	checkpoints/Aachen-Heerlen/matsam_lora_decoder_best.pth
EMPS	LoRA plus mask decoder	2	checkpoints/EMPS/matsam_lora_decoder_best.pth
Grain	LoRA plus mask decoder	2	checkpoints/Grain/matsam_lora_decoder_best.pth
EBC	LoRA, mask decoder, and class tokens	3	checkpoints/EBC/semantic_matsam_best.pth
Super	LoRA, mask decoder, and class tokens	3	checkpoints/Super/semantic_matsam_best.pth
MetalDAM	LoRA, mask decoder, and class tokens	5	checkpoints/MetalDAM/semantic_matsam_best.pth
UHCS	LoRA, mask decoder, and class tokens	7	checkpoints/UHCS/semantic_matsam_best.pth

Class counts include background.

Base Model Requirement

Download the SAM ViT-H base checkpoint from Meta: https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth

The fine-tuned files contain LoRA and mask-decoder parameters. Multiclass files also contain learned class-token parameters. They are not standalone weights.

Training Details

• Base architecture: ViT-H
• SAM encoder input: longest side resized and padded to 1024 x 1024
• Training batch size: 1
• Epochs: 200
• Optimizer: AdamW
• Learning rate: 0.0001
• Weight decay: 0.0001
• Scheduler: CosineAnnealingLR with minimum learning rate 0.000001
• Early-stopping patience: 50
• LoRA rank: 8
• LoRA alpha: 16
• LoRA dropout: 0.05
• LoRA targets: qkv and proj
• Binary objective: BCE, Dice, and IoU MSE
• Multiclass objective: cross-entropy and Dice

Usage

Example for binary EMPS segmentation:

hf download NAMESPACE/microstructure-matsam checkpoints/EMPS/matsam_lora_decoder_best.pth --local-dir weights/matsam
python matsam/test_lora_decoder.py \
  --dataset-root /path/to/EMPS \
  --checkpoint weights/pretrained/sam_vit_h_4b8939.pth \
  --finetuned-checkpoint weights/matsam/checkpoints/EMPS/matsam_lora_decoder_best.pth \
  --output-dir results/matsam_emps \
  --method-type 2

For Grain, use --method-type 1. For other metallographic datasets, use --method-type 2. For multiclass checkpoints, use matsam/test_semantic_matsam.py and pass the class count from the table.

Inference Parameters

The original automatic-mask path uses n_per_side_base=54, pred_iou_thresh=0.88, stability_score_thresh=0.9, box_nms_thresh=0.7, crop_nms_thresh=0.7, and crop_n_layers=0. The binary fine-tuned path uses point prompts and thresholds of 0.7 for predicted IoU and stability score.

Evaluation, Intended Use, and Limitations

Evaluation restores masks to their original image dimensions and calculates the common benchmark metrics through Myutils/metrics.py. These checkpoints are intended for research reproduction on the named datasets. They require the exact ViT-H base architecture and matching script. They may not generalize to unseen materials or imaging conditions and are not validated for safety-critical or industrial quality-control decisions.

License and Citation

The released experiment files are provided under the MIT license. The SAM base checkpoint and MatSAM implementation remain subject to their upstream terms. The base checkpoint is not redistributed here. A paper citation will be added after publication.