Xclip Base Patch16 Hmdb 8 Shot

🚀 X-CLIP (base-sized model)

The X-CLIP model (base-sized, patch resolution of 16) is trained in a few-shot manner (K = 8) on HMDB - 51, which can be used for video - language understanding tasks such as video classification and video - text retrieval.

🚀 Quick Start

The X-CLIP model (base-sized, with a patch resolution of 16) is trained in a few-shot fashion (K = 8) on HMDB-51. It was introduced in the paper Expanding Language-Image Pretrained Models for General Video Recognition by Ni et al. and first released in this repository.

This model was trained using 32 frames per video, at a resolution of 224x224.

Disclaimer: The team releasing X-CLIP did not write a model card for this model, so this model card has been written by the Hugging Face team.

✨ Features

• X-CLIP is a minimal extension of CLIP for general video - language understanding.
• The model is trained in a contrastive way on (video, text) pairs, enabling it to be used for tasks like zero - shot, few - shot or fully supervised video classification and video - text retrieval.

📚 Documentation

Intended uses & limitations

You can use the raw model for determining how well text goes with a given video. See the model hub to look for fine - tuned versions on a task that interests you.

How to use

For code examples, we refer to the documentation.

Training data

This model was trained on HMDB-51.

Preprocessing

• The exact details of preprocessing during training can be found here.
• The exact details of preprocessing during validation can be found here.
• During validation, one resizes the shorter edge of each frame, after which center cropping is performed to a fixed - size resolution (like 224x224). Next, frames are normalized across the RGB channels with the ImageNet mean and standard deviation.

Evaluation results

This model achieves a top - 1 accuracy of 62.8%.

📄 License

This model is released under the MIT license.