Xclip Base Patch16 Ucf 8 Shot

🚀 X-CLIP (base-sized model)

An X-CLIP model (base-sized, patch resolution of 16) trained in a few-shot fashion (K = 8) for video classification.

🚀 Quick Start

The X-CLIP model presented here is a base-sized model with a patch resolution of 16. It's trained in a few-shot manner (K = 8) on the UCF101 dataset. It was first introduced in the paper Expanding Language-Image Pretrained Models for General Video Recognition by Ni et al. and initially released in this repository.

This model was trained using 32 frames per video at a resolution of 224x224.

Disclaimer: The team releasing X-CLIP did not write a model card for this model. This model card has been written by the Hugging Face team.

✨ Features

X-CLIP is a minimal extension of CLIP designed for general video-language understanding. The model is trained contrastively on (video, text) pairs.

This enables the model to be applied to tasks such as zero-shot, few-shot, or fully supervised video classification and video-text retrieval.

📚 Documentation

Intended uses & limitations

You can use the raw model to determine how well text matches a given video. Check the model hub to find fine-tuned versions for tasks that interest you.

How to use

For code examples, refer to the documentation.

Training data

This model was trained on UCF101.

Preprocessing

The exact details of preprocessing during training can be found here.

The exact details of preprocessing during validation can be found here.

During validation, the shorter edge of each frame is resized, followed by center cropping to a fixed-size resolution (e.g., 224x224). Then, the frames are normalized across the RGB channels using the ImageNet mean and standard deviation.

Evaluation results

This model achieves a top-1 accuracy of 88.3%.

📄 License

This project is licensed under the MIT license.

📦 Model Information