Xclip Base Patch16 16 Frames

🚀 X-CLIP (base-sized model)

X-CLIP is a minimal extension of CLIP for general video - language understanding. It can be used for tasks like zero - shot, few - shot or fully supervised video classification and video - text retrieval.

🚀 Quick Start

The X - CLIP model (base - sized, patch resolution of 16) is trained fully - supervised on Kinetics - 400. It was introduced in the paper Expanding Language - Image Pretrained Models for General Video Recognition by Ni et al. and first released in [this repository](https://github.com/microsoft/VideoX/tree/master/X - CLIP). This model was trained using 16 frames per video, at a resolution of 224x224.

Disclaimer: The team releasing X - CLIP did not write a model card for this model so this model card has been written by the Hugging Face team.

✨ Features

• X - CLIP is a minimal extension of CLIP for general video - language understanding.
• The model is trained in a contrastive way on (video, text) pairs, allowing it to be used for tasks like zero - shot, few - shot or fully supervised video classification and video - text retrieval.

📚 Documentation

Intended uses & limitations

You can use the raw model for determining how well text goes with a given video. See the model hub to look for fine - tuned versions on a task that interests you.

How to use

For code examples, we refer to the documentation.

Training data

This model was trained on Kinetics - 400.

Preprocessing

The exact details of preprocessing during training can be found [here](https://github.com/microsoft/VideoX/blob/40f6d177e0a057a50ac69ac1de6b5938fd268601/X - CLIP/datasets/build.py#L247). The exact details of preprocessing during validation can be found [here](https://github.com/microsoft/VideoX/blob/40f6d177e0a057a50ac69ac1de6b5938fd268601/X - CLIP/datasets/build.py#L285). During validation, one resizes the shorter edge of each frame, after which center cropping is performed to a fixed - size resolution (like 224x224). Next, frames are normalized across the RGB channels with the ImageNet mean and standard deviation.

Evaluation results

📄 License

This model is released under the MIT license.