MobileSam: Optimized for Qualcomm Devices

Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.

This is based on the implementation of MobileSam found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime	Precision	Chipset	SDK Versions	Download
ONNX	float	Universal	QAIRT 2.45, ONNX Runtime 1.27.1	Download
QNN_DLC	float	Universal	QAIRT 2.45	Download
TFLITE	float	Universal	QAIRT 2.45	Download

For more device-specific assets and performance metrics, visit MobileSam on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

• Custom weights (e.g., fine-tuned checkpoints)
• Custom input shapes
• Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for MobileSam on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

• Input resolution: 720p (720x1280)
• Model checkpoint: vit_t
• Model size (sam_decoder) (float): 23.7 MB
• Model size (sam_encoder) (float): 26.6 MB
• Number of parameters (sam_decoder): 6.16M
• Number of parameters (sam_encoder): 6.95M

Performance Summary

Model	Runtime	Precision	Chipset	Inference Time (ms)	Peak Memory Range (MB)	Primary Compute Unit
decoder	ONNX	float	Snapdragon® X2 Elite	2.634 ms	4 - 4 MB	NPU
decoder	ONNX	float	Snapdragon® X Elite	6.155 ms	11 - 11 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	4.064 ms	1 - 269 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Gen 1 Mobile	14.638 ms	5 - 236 MB	NPU
decoder	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	6.286 ms	4 - 8 MB	NPU
decoder	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	5.914 ms	0 - 14 MB	NPU
decoder	ONNX	float	Qualcomm® QCS8450	14.638 ms	5 - 236 MB	NPU
decoder	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	6.861 ms	4 - 7 MB	NPU
decoder	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	6.155 ms	11 - 11 MB	NPU
decoder	ONNX	float	Qualcomm® Dragonwing™ Q-8750	3.186 ms	0 - 235 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Elite Mobile	3.186 ms	0 - 235 MB	NPU
decoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	2.539 ms	2 - 215 MB	NPU
decoder	QNN_DLC	float	Snapdragon® X2 Elite	3.03 ms	4 - 4 MB	NPU
decoder	QNN_DLC	float	Snapdragon® X Elite	5.911 ms	4 - 4 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	3.782 ms	4 - 221 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Gen 1 Mobile	11.744 ms	0 - 220 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-8275	5.775 ms	4 - 11 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-8275	12.735 ms	0 - 192 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	5.451 ms	4 - 38 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8775P	6.38 ms	0 - 193 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8650P	6.38 ms	0 - 193 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8255P	6.38 ms	0 - 193 MB	NPU
decoder	QNN_DLC	float	Qualcomm® QCS8450	11.744 ms	0 - 220 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-9075	6.167 ms	6 - 12 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-X7181	5.911 ms	4 - 4 MB	NPU
decoder	QNN_DLC	float	Qualcomm® Dragonwing™ Q-8750	2.852 ms	0 - 221 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA7255P	12.735 ms	0 - 192 MB	NPU
decoder	QNN_DLC	float	Qualcomm® SA8295P	7.83 ms	0 - 196 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Elite Mobile	2.852 ms	0 - 221 MB	NPU
decoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	2.557 ms	4 - 194 MB	NPU
decoder	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	3.435 ms	0 - 210 MB	NPU
decoder	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	8.355 ms	0 - 216 MB	NPU
decoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	5.361 ms	0 - 19 MB	NPU
decoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	11.796 ms	0 - 185 MB	NPU
decoder	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	5.002 ms	0 - 8 MB	NPU
decoder	TFLITE	float	Qualcomm® SA8775P	5.9 ms	0 - 188 MB	NPU
decoder	TFLITE	float	Qualcomm® SA8650P	5.9 ms	0 - 188 MB	NPU
decoder	TFLITE	float	Qualcomm® SA8255P	5.9 ms	0 - 188 MB	NPU
decoder	TFLITE	float	Qualcomm® QCS8450	8.355 ms	0 - 216 MB	NPU
decoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	5.672 ms	0 - 19 MB	NPU
decoder	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	2.617 ms	0 - 183 MB	NPU
decoder	TFLITE	float	Qualcomm® SA7255P	11.796 ms	0 - 185 MB	NPU
decoder	TFLITE	float	Qualcomm® SA8295P	7.332 ms	0 - 190 MB	NPU
decoder	TFLITE	float	Snapdragon® 8 Elite Mobile	2.617 ms	0 - 183 MB	NPU
decoder	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	2.192 ms	0 - 185 MB	NPU
encoder	ONNX	float	Snapdragon® X2 Elite	58.903 ms	12 - 12 MB	NPU
encoder	ONNX	float	Snapdragon® X Elite	116.029 ms	13 - 13 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Gen 3 Mobile	83.329 ms	21 - 1139 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Gen 1 Mobile	327.149 ms	20 - 1193 MB	NPU
encoder	ONNX	float	Qualcomm® Dragonwing™ IQ-8275	112.413 ms	12 - 16 MB	NPU
encoder	ONNX	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	116.071 ms	13 - 31 MB	NPU
encoder	ONNX	float	Qualcomm® QCS8450	327.149 ms	20 - 1193 MB	NPU
encoder	ONNX	float	Qualcomm® Dragonwing™ IQ-9075	121.602 ms	12 - 15 MB	NPU
encoder	ONNX	float	Qualcomm® Dragonwing™ IQ-X7181	116.029 ms	13 - 13 MB	NPU
encoder	ONNX	float	Qualcomm® Dragonwing™ Q-8750	65.12 ms	9 - 931 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Elite Mobile	65.12 ms	9 - 931 MB	NPU
encoder	ONNX	float	Snapdragon® 8 Elite Gen 5 Mobile	52.309 ms	8 - 1024 MB	NPU
encoder	QNN_DLC	float	Snapdragon® X2 Elite	45.987 ms	12 - 12 MB	NPU
encoder	QNN_DLC	float	Snapdragon® X Elite	103.247 ms	12 - 12 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Gen 3 Mobile	66.112 ms	12 - 1334 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Gen 1 Mobile	327.155 ms	12 - 1342 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-8275	92.17 ms	12 - 30 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-8275	207.955 ms	1 - 1125 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	100.908 ms	12 - 14 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8775P	100.324 ms	0 - 1549 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8650P	100.324 ms	0 - 1549 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8255P	100.324 ms	0 - 1549 MB	NPU
encoder	QNN_DLC	float	Qualcomm® QCS8450	327.155 ms	12 - 1342 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-9075	101.187 ms	12 - 30 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ IQ-X7181	103.247 ms	12 - 12 MB	NPU
encoder	QNN_DLC	float	Qualcomm® Dragonwing™ Q-8750	48.689 ms	12 - 1130 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA7255P	207.955 ms	1 - 1125 MB	NPU
encoder	QNN_DLC	float	Qualcomm® SA8295P	324.965 ms	0 - 1556 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Elite Mobile	48.689 ms	12 - 1130 MB	NPU
encoder	QNN_DLC	float	Snapdragon® 8 Elite Gen 5 Mobile	40.347 ms	10 - 1258 MB	NPU
encoder	TFLITE	float	Snapdragon® 8 Gen 3 Mobile	66.978 ms	4 - 3185 MB	NPU
encoder	TFLITE	float	Snapdragon® 8 Gen 1 Mobile	310.091 ms	4 - 1363 MB	NPU
encoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	86.768 ms	0 - 47 MB	NPU
encoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-8275	196.85 ms	4 - 1633 MB	NPU
encoder	TFLITE	float	Qualcomm® Dragonwing™ QCS8550 (Proxy)	90.598 ms	4 - 7 MB	NPU
encoder	TFLITE	float	Qualcomm® SA8775P	100.606 ms	5 - 1670 MB	NPU
encoder	TFLITE	float	Qualcomm® SA8650P	100.606 ms	5 - 1670 MB	NPU
encoder	TFLITE	float	Qualcomm® SA8255P	100.606 ms	5 - 1670 MB	NPU
encoder	TFLITE	float	Qualcomm® QCS8450	310.091 ms	4 - 1363 MB	NPU
encoder	TFLITE	float	Qualcomm® Dragonwing™ IQ-9075	120.497 ms	3 - 48 MB	NPU
encoder	TFLITE	float	Qualcomm® Dragonwing™ Q-8750	48.054 ms	3 - 1629 MB	NPU
encoder	TFLITE	float	Qualcomm® SA7255P	196.85 ms	4 - 1633 MB	NPU
encoder	TFLITE	float	Qualcomm® SA8295P	258.282 ms	5 - 1713 MB	NPU
encoder	TFLITE	float	Snapdragon® 8 Elite Mobile	48.054 ms	3 - 1629 MB	NPU
encoder	TFLITE	float	Snapdragon® 8 Elite Gen 5 Mobile	40.193 ms	4 - 1797 MB	NPU

License

• The license for the original implementation of MobileSam can be found here.

References

• Segment Anything
• Source Model Implementation

Community

• Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
• For questions or feedback please reach out to us.