đ SAM 2: Segment Anything in Images and Videos
This repository is for SAM 2, a foundation model developed by FAIR to address promptable visual segmentation in both images and videos. For more information, refer to the SAM 2 paper.
The official code is publicly available in this repo.
đ Quick Start
This section provides a quick guide on using SAM 2 for image and video prediction.
⨠Features
- Promptable Visual Segmentation: SAM 2 can perform segmentation in images and videos based on given prompts.
- Foundation Model: A powerful model for various visual segmentation tasks.
đĻ Installation
The original document does not provide installation steps, so this section is skipped.
đģ Usage Examples
Basic Usage
Image Prediction
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor
predictor = SAM2ImagePredictor.from_pretrained("facebook/sam2-hiera-small")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
predictor.set_image(<your_image>)
masks, _, _ = predictor.predict(<input_prompts>)
Video Prediction
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor
predictor = SAM2VideoPredictor.from_pretrained("facebook/sam2-hiera-small")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
state = predictor.init_state(<your_video>)
frame_idx, object_ids, masks = predictor.add_new_points_or_box(state, <your_prompts>):
for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
...
Refer to the demo notebooks for more detailed usage.
đ Documentation
The original document does not have detailed documentation content, so this section is skipped.
đ§ Technical Details
The original document does not contain technical implementation details, so this section is skipped.
đ License
This project is licensed under the Apache-2.0 license.
Citation
To cite the paper, model, or software, please use the following BibTeX entry:
@article{ravi2024sam2,
title={SAM 2: Segment Anything in Images and Videos},
author={Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R{\"a}dle, Roman and Rolland, Chloe and Gustafson, Laura and Mintun, Eric and Pan, Junting and Alwala, Kalyan Vasudev and Carion, Nicolas and Wu, Chao-Yuan and Girshick, Ross and Doll{\'a}r, Piotr and Feichtenhofer, Christoph},
journal={arXiv preprint arXiv:2408.00714},
url={https://arxiv.org/abs/2408.00714},
year={2024}
}