Contriever Msmarco

🚀 特徵提取模型 - contriever微調版

本模型是預訓練的contriever模型的微調版本，預訓練模型可在 此處 獲取。該微調遵循了論文 Towards Unsupervised Dense Information Retrieval with Contrastive Learning 中描述的方法。相關的GitHub倉庫可在 此處 找到。

🚀 快速開始

✨ 主要特性

此模型可用於特徵提取，通過微調預訓練的contriever模型，能夠更好地適應特定的任務需求。

💻 使用示例

基礎用法

在HuggingFace Transformers中直接使用該模型時，需要添加均值池化操作以獲得句子嵌入。

import torch
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained('facebook/contriever-msmarco')
model = AutoModel.from_pretrained('facebook/contriever-msmarco')

sentences = [
    "Where was Marie Curie born?",
    "Maria Sklodowska, later known as Marie Curie, was born on November 7, 1867.",
    "Born in Paris on 15 May 1859, Pierre Curie was the son of Eugène Curie, a doctor of French Catholic origin from Alsace."
]

# Apply tokenizer
inputs = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
outputs = model(**inputs)

# Mean pooling
def mean_pooling(token_embeddings, mask):
    token_embeddings = token_embeddings.masked_fill(~mask[..., None].bool(), 0.)
    sentence_embeddings = token_embeddings.sum(dim=1) / mask.sum(dim=1)[..., None]
    return sentence_embeddings
embeddings = mean_pooling(outputs[0], inputs['attention_mask'])