🚀 img2pose
img2pose是一個用於圖像特徵提取的模型,它藉助Faster R - CNN預測照片中所有人臉的6自由度姿態(DoF),還能將3D人臉投影到2D平面以識別每個人臉的邊界框,且無需其他人臉檢測模型。
🚀 快速開始
以下是使用img2pose模型的示例代碼:
import numpy as np
import os
import json
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from feat.facepose_detectors.img2pose.deps.models import FasterDoFRCNN, postprocess_img2pose
from feat.utils.io import get_resource_path
from torchvision.models.detection.backbone_utils import resnet_fpn_backbone
facepose_config_file = hf_hub_download(repo_id= "py-feat/img2pose", filename="config.json", cache_dir=get_resource_path())
with open(facepose_config_file, "r") as f:
facepose_config = json.load(f)
device = 'cpu'
backbone = resnet_fpn_backbone(backbone_name="resnet18", weights=None)
backbone.eval()
backbone.to(device)
facepose_detector = FasterDoFRCNN(backbone=backbone,
num_classes=2,
min_size=facepose_config['min_size'],
max_size=facepose_config['max_size'],
pose_mean=torch.tensor(facepose_config['pose_mean']),
pose_stddev=torch.tensor(facepose_config['pose_stddev']),
threed_68_points=torch.tensor(facepose_config['threed_points']),
rpn_pre_nms_top_n_test=facepose_config['rpn_pre_nms_top_n_test'],
rpn_post_nms_top_n_test=facepose_config['rpn_post_nms_top_n_test'],
bbox_x_factor=facepose_config['bbox_x_factor'],
bbox_y_factor=facepose_config['bbox_y_factor'],
expand_forehead=facepose_config['expand_forehead'])
facepose_model_file = hf_hub_download(repo_id= "py-feat/img2pose", filename="model.safetensors", cache_dir=get_resource_path())
facepose_checkpoint = load_file(facepose_model_file)
facepose_detector.load_state_dict(facepose_checkpoint)
facepose_detector.eval()
facepose_detector.to(device)
face_image = "path/to/your/test_image.jpg"
img2pose_output = facepose_detector(face_image)
img2pose_output = postprocess_img2pose(img2pose_output[0])
bbox = img2pose_output['boxes']
poses = img2pose_output['dofs']
facescores = img2pose_output['scores']
✨ 主要特性
- 使用Faster R - CNN預測人臉的6自由度姿態(DoF)。
- 可以將3D人臉投影到2D平面,識別每個人臉的邊界框。
- 無需額外的人臉檢測模型。
📚 詳細文檔
模型詳情
屬性 |
詳情 |
模型類型 |
卷積神經網絡(CNN) |
架構 |
Faster R - CNN |
框架 |
PyTorch |
模型來源
引用
如果您在研究或應用中使用了此模型,請引用以下論文:
Vítor Albiero, Xingyu Chen, Xi Yin, Guan Pang, Tal Hassner, "img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation," CVPR, 2021, arXiv:2012.07791
@inproceedings{albiero2021img2pose,
title={img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation},
author={Albiero, Vítor and Chen, Xingyu and Yin, Xi and Pang, Guan and Hassner, Tal},
booktitle={CVPR},
year={2021},
url={https://arxiv.org/abs/2012.07791},
}
致謝
我們感謝Albiero Vítor以寬鬆的許可協議分享他們的代碼和訓練權重。
📄 許可證
本模型採用CC - BY - NC - 4.0許可證。