Timesformer Hr Finetuned Ssv2

🚀 TimeSformer (high-resolution variant, fine-tuned on Something Something v2)

A TimeSformer model pre - trained on Something Something v2, which can be used for video classification.

🚀 Quick Start

The TimeSformer model was introduced in the paper TimeSformer: Is Space-Time Attention All You Need for Video Understanding? by Tong et al. and first released in this repository. Note that the team releasing TimeSformer did not write a model card for this model, and this model card has been written by fcakyon.

✨ Features

You can use the raw model for video classification into one of the 174 possible Something Something v2 labels.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

# Here is how to use this model to classify a video:
from transformers import AutoImageProcessor, TimesformerForVideoClassification
import numpy as np
import torch

video = list(np.random.randn(16, 3, 448, 448))

processor = AutoImageProcessor.from_pretrained("facebook/timesformer-hr-finetuned-ssv2")
model = TimesformerForVideoClassification.from_pretrained("facebook/timesformer-hr-finetuned-ssv2")

inputs = feature_extractor(images=video, return_tensors="pt")

with torch.no_grad():
  outputs = model(**inputs)
  logits = outputs.logits

predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])

Advanced Usage

For more code examples, we refer to the documentation.

📚 Documentation

BibTeX entry and citation info

@inproceedings{bertasius2021space,
  title={Is Space-Time Attention All You Need for Video Understanding?},
  author={Bertasius, Gedas and Wang, Heng and Torresani, Lorenzo},
  booktitle={International Conference on Machine Learning},
  pages={813--824},
  year={2021},
  organization={PMLR}
}

📄 License

The license of this project is cc-by-nc-4.0.