SlimSAM-uniform-50 Open-source Model - Efficiently Generate Similar Performance with Pruning and Distillation Framework, Significantly Save Training Data!

Slimsam Uniform 50

Developed by Zigeng

SlimSAM is an innovative SAM model compression method that efficiently reuses pre-trained SAM through a pruning-distillation framework, achieving comparable performance with only 0.1% of training data.

Image Segmentation

Transformers

OtherOpen Source License:Apache-2.0 #Lightweight segmentation model #Low data dependency #Pruning and distillation techniques

Downloads 9,459

Release Time : 1/8/2024

Model Overview

SlimSAM is a lightweight version of the Segment Anything Model (SAM), which significantly reduces model parameters and computational requirements while maintaining the original SAM's segmentation performance through an innovative alternating slimming strategy and pruning-distillation framework.

Model Features

Efficient compression

Reduces SAM model parameters to 0.9% (5.7 million) and computational operations to 0.8% (21 billion)

Data efficiency

Achieves performance comparable to original SAM with only 0.1% (10,000 images) of training data

Alternating slimming strategy

Progressively compresses the model through an innovative pruning-distillation framework to enhance knowledge transfer

Label-free pruning criteria

Aligns pruning objectives with optimization metrics to improve post-pruning distillation effects

Model Capabilities

Image segmentation

Object recognition

Visual feature extraction

Use Cases

Computer vision

General image segmentation

Segments objects in arbitrary images

Maintains segmentation accuracy comparable to original SAM while reducing parameters by 99.9%

Deployment in resource-constrained environments

Enables efficient image segmentation on devices with limited computational resources

Reduces computational operations to 0.8% of original SAM

🚀 SlimSAM: 0.1% Data Makes Segment Anything Slim

SlimSAM is a novel SAM compression method that efficiently reuses pre - trained SAMs without extensive retraining, achieving great performance with significantly reduced parameters, MACs, and training data.

🚀 Quick Start

SlimSAM is a novel SAM compression method, which efficiently reuses pre - trained SAMs without the necessity for extensive retraining. This is achieved by the efficient reuse of pre - trained SAMs through a unified pruning - distillation framework. To enhance knowledge inheritance from the original SAM, we employ an innovative alternate slimming strategy that partitions the compression process into a progressive procedure. Diverging from prior pruning techniques, we meticulously prune and distill decoupled model structures in an alternating fashion. Furthermore, a novel label - free pruning criterion is also proposed to align the pruning objective with the optimization target, thereby boosting the post - distillation after pruning.

SlimSAM achieves approaching performance while reducing the parameter counts to 0.9% (5.7M), MACs to 0.8% (21G), and requiring mere 0.1% (10k) of the training data when compared to the original SAM - H. Extensive experiments demonstrate that our method realizes significant superior performance while utilizing over 10 times less training data when compared to other SAM compression methods.

💻 Usage Examples

Basic Usage

Fast state_dict loading for local uniform pruning SlimSAM - 50 model:

model = SamModel.from_pretrained("Zigeng/SlimSAM-uniform-50").to("cuda")
processor = SamProcessor.from_pretrained("Zigeng/SlimSAM-uniform-50")

img_url = "https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png"
raw_image = Image.open(requests.get(img_url, stream=True).raw).convert("RGB")
input_points = [[[450, 600]]] # 2D localization of a window
inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to("cuda")
outputs = model(**inputs)
masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu())
scores = outputs.iou_scores

📚 Documentation

BibTex of our SlimSAM

If you use SlimSAM in your research, please use the following BibTeX entry. Thank you!

@misc{chen202301,
      title={0.1% Data Makes Segment Anything Slim}, 
      author={Zigeng Chen and Gongfan Fang and Xinyin Ma and Xinchao Wang},
      year={2023},
      eprint={2312.05284},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

SAM (Segment Anything) [bib]

@article{kirillov2023segany,
  title={Segment Anything}, 
  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan - Yen and Doll{\'a}r, Piotr and Girshick, Ross},
  journal={arXiv:2304.02643},
  year={2023}
}

Torch Pruning (DepGraph: Towards Any Structural Pruning) [bib]

@inproceedings{fang2023depgraph,
  title={Depgraph: Towards any structural pruning},
  author={Fang, Gongfan and Ma, Xinyin and Song, Mingli and Mi, Michael Bi and Wang, Xinchao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={16091--16101},
  year={2023}
}

📄 License

This project is licensed under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご