Directsam 1800px 0424

🚀 Direct Segment Anything Model (DirectSAM)

Direct Segment Anything Model (DirectSAM) proposed in the paper "Subobject-level Image Tokenization", which focuses on image segmentation tasks such as boundary detection and edge estimation.

🚀 Quick Start

The Direct Segment Anything Model (DirectSAM) is introduced in the paper "Subobject-level Image Tokenization" by Delong Chen, Samuel Cahyawijaya, Jianfeng Liu, Baoyuan Wang, and Pascale Fung.

✨ Features

• Model: A Segformer is used as the backbone, with a total of 84.6M parameters. The final multi - way classifier is replaced with a one - way classifier, and full - parameter fine - tuning is performed.
• Data: The SA - 1B dataset is used to train the DirectSAM. The mask annotations are converted to boundaries by running opencv - implemented contour detection and plotting the extracted contours with a line width of 3. Random Gaussian blur is applied with a probability of 0.25.
• Training: DirectSAM is trained on the SA - 1B dataset using a single - node 8xNVIDIA A100 (80GB) server. It is first trained with an input resolution of 1024x1024 for one epoch, and then for another 0.6 epoch with 1800x1800 resolution (the maximum resolution for data parallel training on 80GB GPUs). For the first 1024x1024 epoch, a per - GPU batch size of 4, gradient accumulation of steps of 4, and a learning rate of 4e - 4 are used. For the second 1800x1800 epoch, a per - GPU batch size of 1, gradient accumulation of steps of 8, and a learning rate of 2e - 4 are used. These two epochs respectively take around 15 days and 20 days.

📚 Documentation

Please see our GitHub repo for more information.