Silver Retriever Base V1

🚀 银色检索器基础版 (v1)

银色检索器模型能够将波兰语的句子或段落编码为768维的密集向量空间，可用于文档检索或语义搜索等任务。

它基于 HerBERT-base 模型进行初始化，并在 PolQA 和 MAUPQA 数据集上进行了15,000步的微调，批量大小为1,024。更多详细信息请参考 SilverRetriever: Advancing Neural Passage Retrieval for Polish Question Answering。

🚀 快速开始

银色检索器模型可用于将波兰语句子或段落编码为768维的密集向量空间，适用于文档检索和语义搜索等任务。以下将介绍其使用方法。

✨ 主要特性

• 基于 HerBERT-base 模型初始化。
• 在 PolQA 和 MAUPQA 数据集上微调。
• 能够将波兰语句子或段落编码为768维的密集向量空间。

📦 安装指南

使用此模型前，需安装 sentence-transformers：

pip install -U sentence-transformers

💻 使用示例

基础用法

使用 sentence-transformers 库进行推理：

from sentence_transformers import SentenceTransformer
sentences = [
    "Pytanie: W jakim mieście urodził się Zbigniew Herbert?", 
    "Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]

model = SentenceTransformer('ipipan/silver-retriever-base-v1')
embeddings = model.encode(sentences)
print(embeddings)

高级用法

不使用 sentence-transformers 库，直接使用 HuggingFace Transformers 进行推理：

from transformers import AutoTokenizer, AutoModel
import torch


def cls_pooling(model_output, attention_mask):
    return model_output[0][:,0]


# Sentences we want sentence embeddings for
sentences = [
    "Pytanie: W jakim mieście urodził się Zbigniew Herbert?", 
    "Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]
# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('ipipan/silver-retriever-base-v1')
model = AutoModel.from_pretrained('ipipan/silver-retriever-base-v1')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, cls pooling.
sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

📚 详细文档

评估结果

图例说明：

• 准确率 指的是前10个结果的准确率。
• 归一化折损累计增益 指的是前10个结果的归一化折损累计增益。

模型完整架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

额外信息

模型创建者

该模型由来自 波兰科学院计算机科学研究所 的 Piotr Rybak 创建。

这项工作得到了欧洲区域发展基金的支持，是2014 - 2020年智能增长运营计划、CLARIN — 通用语言资源与技术基础设施项目（项目编号：POIR.04.02.00 - 00C002/19）的一部分。

许可信息

CC BY - SA 4.0

引用信息

@inproceedings{rybak-ogrodniczuk-2024-silver-retriever,
    title = "Silver Retriever: Advancing Neural Passage Retrieval for {P}olish Question Answering",
    author = "Rybak, Piotr  and
      Ogrodniczuk, Maciej",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1291",
    pages = "14826--14831",
    abstract = "Modern open-domain question answering systems often rely on accurate and efficient retrieval components to find passages containing the facts necessary to answer the question. Recently, neural retrievers have gained popularity over lexical alternatives due to their superior performance. However, most of the work concerns popular languages such as English or Chinese. For others, such as Polish, few models are available. In this work, we present Silver Retriever, a neural retriever for Polish trained on a diverse collection of manually or weakly labeled datasets. Silver Retriever achieves much better results than other Polish models and is competitive with larger multilingual models. Together with the model, we open-source five new passage retrieval datasets.",
}

输入准备

该模型是在问题 - 段落对上进行训练的，当输入格式与训练时的格式相同时，效果最佳：

• 在问题开头添加短语 Pytanie:。
• 训练段落由 标题 和 文本 用特殊标记 </s> 连接而成。即使你的段落没有 标题，在段落前添加 </s> 标记仍然有益。
• 尽管训练时使用了点积，但该模型通常使用余弦距离效果更好。

🔧 技术细节

该模型基于 HerBERT-base 模型进行初始化，并在 PolQA 和 MAUPQA 数据集上进行了15,000步的微调，批量大小为1,024。通过这种方式，模型能够将波兰语句子或段落编码为768维的密集向量空间，以用于文档检索和语义搜索等任务。

📄 许可证

CC BY - SA 4.0