Internvideo2 Stage2 6B

🚀 InternVideo2 6B模型

本倉庫包含論文 InternVideo2 第二階段的6B模型。該模型可用於視頻分類任務，為視頻內容的理解和分類提供強大支持。

🚀 快速開始

本項目主要用於視頻分類，提供了預訓練的6B模型，可通過代碼調用實現視頻與文本的匹配檢索等功能。

📄 許可證

本項目採用 MIT 許可證。

📦 安裝指南

請參考 安裝說明 進行安裝。

💻 使用示例

基礎用法

import cv2
from transformers import AutoModel
from modeling_internvideo2 import (retrieve_text, vid2tensor, _frame_from_video,)


model = AutoModel.from_pretrained("OpenGVLab/InternVideo2-Stage2_6B", trust_remote_code=True).eval()

video = cv2.VideoCapture('example1.mp4')
frames = [x for x in _frame_from_video(video)]
text_candidates = ["A playful dog and its owner wrestle in the snowy yard, chasing each other with joyous abandon.",
                "A man in a gray coat walks through the snowy landscape, pulling a sleigh loaded with toys.",
                "A person dressed in a blue jacket shovels the snow-covered pavement outside their house.",
                "A cat excitedly runs through the yard, chasing a rabbit.",
                "A person bundled up in a blanket walks through the snowy landscape, enjoying the serene winter scenery."]

texts, probs = retrieve_text(frames, text_candidates, model=model, topk=5)
for t, p in zip(texts, probs):
    print(f'text: {t} ~ prob: {p:.4f}')

vidtensor = vid2tensor('example1.mp4', fnum=4)
feat = model.get_vid_feat(vidtensor)

上述代碼展示瞭如何加載模型，從視頻中提取幀，然後根據視頻幀從候選文本中檢索匹配的文本，並獲取視頻特徵。