Xclip Base Patch16 Kinetics 600 16 Frames

🚀 X-CLIP (base-sized model)

An X-CLIP model (base-sized, patch resolution of 16) trained fully-supervised on Kinetics-600 for video classification tasks.

🚀 Quick Start

The X-CLIP (base-sized model) is a powerful tool for video classification. It was fully-supervised trained on Kinetics-600. It was introduced in the paper Expanding Language-Image Pretrained Models for General Video Recognition by Ni et al. and first released in this repository.

✨ Features

• General Video-Language Understanding: X-CLIP is a minimal extension of CLIP for general video-language understanding. It is trained in a contrastive way on (video, text) pairs.
• Multiple Task Applications: This allows the model to be used for tasks like zero-shot, few-shot or fully supervised video classification and video-text retrieval.

📚 Documentation

Intended uses & limitations

You can use the raw model for determining how well text goes with a given video. See the model hub to look for fine-tuned versions on a task that interests you.

How to use

For code examples, we refer to the documentation.

Training data

This model was trained on Kinetics-600.

Preprocessing

• Training: The exact details of preprocessing during training can be found here.
• Validation: The exact details of preprocessing during validation can be found here. During validation, one resizes the shorter edge of each frame, after which center cropping is performed to a fixed-size resolution (like 224x224). Next, frames are normalized across the RGB channels with the ImageNet mean and standard deviation.

Evaluation results

This model achieves a top-1 accuracy of 85.8% and a top-5 accuracy of 97.3%.

📄 License

This model is released under the MIT license.