Silver Retriever Base V1

🚀 銀色檢索器基礎版 (v1)

銀色檢索器模型能夠將波蘭語的句子或段落編碼為768維的密集向量空間，可用於文檔檢索或語義搜索等任務。

它基於 HerBERT-base 模型進行初始化，並在 PolQA 和 MAUPQA 數據集上進行了15,000步的微調，批量大小為1,024。更多詳細信息請參考 SilverRetriever: Advancing Neural Passage Retrieval for Polish Question Answering。

🚀 快速開始

銀色檢索器模型可用於將波蘭語句子或段落編碼為768維的密集向量空間，適用於文檔檢索和語義搜索等任務。以下將介紹其使用方法。

✨ 主要特性

• 基於 HerBERT-base 模型初始化。
• 在 PolQA 和 MAUPQA 數據集上微調。
• 能夠將波蘭語句子或段落編碼為768維的密集向量空間。

📦 安裝指南

使用此模型前，需安裝 sentence-transformers：

pip install -U sentence-transformers

💻 使用示例

基礎用法

使用 sentence-transformers 庫進行推理：

from sentence_transformers import SentenceTransformer
sentences = [
    "Pytanie: W jakim mieście urodził się Zbigniew Herbert?", 
    "Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]

model = SentenceTransformer('ipipan/silver-retriever-base-v1')
embeddings = model.encode(sentences)
print(embeddings)

高級用法

不使用 sentence-transformers 庫，直接使用 HuggingFace Transformers 進行推理：

from transformers import AutoTokenizer, AutoModel
import torch


def cls_pooling(model_output, attention_mask):
    return model_output[0][:,0]


# Sentences we want sentence embeddings for
sentences = [
    "Pytanie: W jakim mieście urodził się Zbigniew Herbert?", 
    "Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]
# Load model from HuggingFace Hub
tokenizer = AutoTokenizer.from_pretrained('ipipan/silver-retriever-base-v1')
model = AutoModel.from_pretrained('ipipan/silver-retriever-base-v1')

# Tokenize sentences
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

# Compute token embeddings
with torch.no_grad():
    model_output = model(**encoded_input)

# Perform pooling. In this case, cls pooling.
sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

📚 詳細文檔

評估結果

圖例說明：

• 準確率 指的是前10個結果的準確率。
• 歸一化折損累計增益 指的是前10個結果的歸一化折損累計增益。

模型完整架構

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)

額外信息

模型創建者

該模型由來自 波蘭科學院計算機科學研究所 的 Piotr Rybak 創建。

這項工作得到了歐洲區域發展基金的支持，是2014 - 2020年智能增長運營計劃、CLARIN — 通用語言資源與技術基礎設施項目（項目編號：POIR.04.02.00 - 00C002/19）的一部分。

許可信息

CC BY - SA 4.0

引用信息

@inproceedings{rybak-ogrodniczuk-2024-silver-retriever,
    title = "Silver Retriever: Advancing Neural Passage Retrieval for {P}olish Question Answering",
    author = "Rybak, Piotr  and
      Ogrodniczuk, Maciej",
    editor = "Calzolari, Nicoletta  and
      Kan, Min-Yen  and
      Hoste, Veronique  and
      Lenci, Alessandro  and
      Sakti, Sakriani  and
      Xue, Nianwen",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.1291",
    pages = "14826--14831",
    abstract = "Modern open-domain question answering systems often rely on accurate and efficient retrieval components to find passages containing the facts necessary to answer the question. Recently, neural retrievers have gained popularity over lexical alternatives due to their superior performance. However, most of the work concerns popular languages such as English or Chinese. For others, such as Polish, few models are available. In this work, we present Silver Retriever, a neural retriever for Polish trained on a diverse collection of manually or weakly labeled datasets. Silver Retriever achieves much better results than other Polish models and is competitive with larger multilingual models. Together with the model, we open-source five new passage retrieval datasets.",
}

輸入準備

該模型是在問題 - 段落對上進行訓練的，當輸入格式與訓練時的格式相同時，效果最佳：

• 在問題開頭添加短語 Pytanie:。
• 訓練段落由 標題 和 文本 用特殊標記 </s> 連接而成。即使你的段落沒有 標題，在段落前添加 </s> 標記仍然有益。
• 儘管訓練時使用了點積，但該模型通常使用餘弦距離效果更好。

🔧 技術細節

該模型基於 HerBERT-base 模型進行初始化，並在 PolQA 和 MAUPQA 數據集上進行了15,000步的微調，批量大小為1,024。通過這種方式，模型能夠將波蘭語句子或段落編碼為768維的密集向量空間，以用於文檔檢索和語義搜索等任務。

📄 許可證

CC BY - SA 4.0