🚀 VideoMAE-v2(超大模型,在UnlabeledHybrid-1M上预训练)
VideoMAE-v2 超大模型在 UnlabeledHybrid-1M 数据集上以自监督方式预训练了 1200 个轮次。该模型由 Wang 等人在论文 [CVPR23]VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking 中提出,并首次在 GitHub 上发布。
✨ 主要特性
🚀 快速开始
预期用途和限制
你可以使用该原始模型进行视频特征提取。
使用方法
以下是如何使用此模型提取视频特征的示例:
from transformers import VideoMAEImageProcessor, AutoModel, AutoConfig
import numpy as np
import torch
config = AutoConfig.from_pretrained("OpenGVLab/VideoMAEv2-Huge", trust_remote_code=True)
processor = VideoMAEImageProcessor.from_pretrained("OpenGVLab/VideoMAEv2-Huge")
model = AutoModel.from_pretrained('OpenGVLab/VideoMAEv2-Huge', config=config, trust_remote_code=True)
video = list(np.random.rand(16, 3, 224, 224))
inputs = processor(video, return_tensors="pt")
inputs['pixel_values'] = inputs['pixel_values'].permute(0, 2, 1, 3, 4)
with torch.no_grad():
outputs = model(**inputs)
📚 详细文档
BibTeX 引用和引用信息
@InProceedings{wang2023videomaev2,
author = {Wang, Limin and Huang, Bingkun and Zhao, Zhiyu and Tong, Zhan and He, Yinan and Wang, Yi and Wang, Yali and Qiao, Yu},
title = {VideoMAE V2: Scaling Video Masked Autoencoders With Dual Masking},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2023},
pages = {14549-14560}
}
@misc{videomaev2,
title={VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking},
author={Limin Wang and Bingkun Huang and Zhiyu Zhao and Zhan Tong and Yinan He and Yi Wang and Yali Wang and Yu Qiao},
year={2023},
eprint={2303.16727},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
📄 许可证
本项目采用 CC BY-NC 4.0 许可证。