Slimsam Uniform 77

🚀 SlimSAM: 0.1% Data Makes Segment Anything Slim

SlimSAM is a novel SAM compression method that efficiently reuses pre - trained SAMs without extensive retraining, achieving high performance with significantly reduced parameters, MACs, and training data.

🚀 Quick Start

SlimSAM is a remarkable innovation in the field of SAM compression. It offers a highly efficient way to reuse pre - trained SAMs, eliminating the need for large - scale retraining.

✨ Features

• Efficient Compression: Through a unified pruning - distillation framework, SlimSAM efficiently reuses pre - trained SAMs.
• Alternate Slimming Strategy: An innovative alternate slimming strategy is employed to partition the compression process into a progressive one, enhancing knowledge inheritance from the original SAM.
• Novel Pruning Criterion: A label - free pruning criterion is proposed to align the pruning objective with the optimization target, boosting post - distillation performance.
• High Performance with Less Resources: SlimSAM achieves approaching performance while reducing the parameter counts to 0.9% (5.7M), MACs to 0.8% (21G), and requiring only 0.1% (10k) of the training data compared to the original SAM - H. It outperforms other SAM compression methods with over 10 times less training data.

📦 Installation

No specific installation steps are provided in the original README.

💻 Usage Examples

Basic Usage

Fast state_dict loading for local uniform pruning SlimSAM - 50 model:

model = SamModel.from_pretrained("Zigeng/SlimSAM-uniform-77").to("cuda")
processor = SamProcessor.from_pretrained("Zigeng/SlimSAM-uniform-77")

img_url = "https://huggingface.co/ybelkada/segment-anything/resolve/main/assets/car.png"
raw_image = Image.open(requests.get(img_url, stream=True).raw).convert("RGB")
input_points = [[[450, 600]]] # 2D localization of a window
inputs = processor(raw_image, input_points=input_points, return_tensors="pt").to("cuda")
outputs = model(**inputs)
masks = processor.image_processor.post_process_masks(outputs.pred_masks.cpu(), inputs["original_sizes"].cpu(), inputs["reshaped_input_sizes"].cpu())
scores = outputs.iou_scores

📚 Documentation

Introduction

SlimSAM is a novel SAM compression method, which efficiently reuses pre - trained SAMs without the necessity for extensive retraining. This is achieved by the efficient reuse of pre - trained SAMs through a unified pruning - distillation framework. To enhance knowledge inheritance from the original SAM, we employ an innovative alternate slimming strategy that partitions the compression process into a progressive procedure. Diverging from prior pruning techniques, we meticulously prune and distill decoupled model structures in an alternating fashion. Furthermore, a novel label - free pruning criterion is also proposed to align the pruning objective with the optimization target, thereby boosting the post - distillation after pruning.

SlimSAM achieves approaching performance while reducing the parameter counts to 0.9% (5.7M), MACs to 0.8% (21G), and requiring mere 0.1% (10k) of the training data when compared to the original SAM - H. Extensive experiments demonstrate that our method realizes significant superior performance while utilizing over 10 times less training data when compared to other SAM compression methods.

🔧 Technical Details

SlimSAM uses a unified pruning - distillation framework to efficiently reuse pre - trained SAMs. It partitions the compression process into a progressive procedure with an alternate slimming strategy. A novel label - free pruning criterion is proposed to align the pruning objective with the optimization target, which helps in boosting the post - distillation performance.

📄 License

This project is licensed under the Apache - 2.0 license.

BibTex of our SlimSAM

If you use SlimSAM in your research, please use the following BibTeX entry. Thank you!

@misc{chen202301,
      title={0.1% Data Makes Segment Anything Slim}, 
      author={Zigeng Chen and Gongfan Fang and Xinyin Ma and Xinchao Wang},
      year={2023},
      eprint={2312.05284},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

@article{kirillov2023segany,
  title={Segment Anything}, 
  author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan - Yen and Doll{\'a}r, Piotr and Girshick, Ross},
  journal={arXiv:2304.02643},
  year={2023}
}

@inproceedings{fang2023depgraph,
  title={Depgraph: Towards any structural pruning},
  author={Fang, Gongfan and Ma, Xinyin and Song, Mingli and Mi, Michael Bi and Wang, Xinchao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={16091--16101},
  year={2023}
}