🚀 Silver Retriever Base (v1.1)
Silver Retriever encodes Polish sentences or paragraphs into a 768 - dimensional dense vector space. It's highly useful for tasks like document retrieval and semantic search.

🚀 Quick Start
The Silver Retriever model can be used for various tasks related to sentence similarity and document retrieval. To get started, you need to understand its input requirements and how to perform inference.
✨ Features
- Encoding Capability: Encodes Polish sentences or paragraphs into a 768 - dimensional dense vector space.
- Fine - Tuned: Initialized from the HerBERT - base model and fine - tuned on the PolQA and MAUPQA datasets.
- High Performance: Achieves excellent results in evaluation metrics compared to other models.
Model Information
Property |
Details |
Pipeline Tag |
sentence - similarity |
Language |
pl |
Tags |
sentence - transformers, feature - extraction, sentence - similarity, transformers |
Datasets |
ipipan/polqa, ipipan/maupqa |
License |
cc - by - sa - 4.0 |
📦 Installation
To use this model, you need to install the sentence - transformers
library:
pip install -U sentence-transformers
💻 Usage Examples
Basic Usage
from sentence_transformers import SentenceTransformer
sentences = [
"Pytanie: W jakim mieście urodził się Zbigniew Herbert?",
"Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]
model = SentenceTransformer('ipipan/silver-retriever-base-v1.1')
embeddings = model.encode(sentences)
print(embeddings)
Advanced Usage
from transformers import AutoTokenizer, AutoModel
import torch
def cls_pooling(model_output, attention_mask):
return model_output[0][:,0]
sentences = [
"Pytanie: W jakim mieście urodził się Zbigniew Herbert?",
"Zbigniew Herbert</s>Zbigniew Bolesław Ryszard Herbert (ur. 29 października 1924 we Lwowie, zm. 28 lipca 1998 w Warszawie) – polski poeta, eseista i dramaturg.",
]
tokenizer = AutoTokenizer.from_pretrained('ipipan/silver-retriever-base-v1.1')
model = AutoModel.from_pretrained('ipipan/silver-retriever-base-v1.1')
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')
with torch.no_grad():
model_output = model(**encoded_input)
sentence_embeddings = cls_pooling(model_output, encoded_input['attention_mask'])
print("Sentence embeddings:")
print(sentence_embeddings)
📚 Documentation
Preparing inputs
The model was trained on question - passage pairs. For best results, format your input as follows:
- Add the phrase
Pytanie:
to the beginning of the question.
- Concatenate the
title
and text
of passages with the special token </s>
. Even without a title
, prefix passages with the </s>
token.
- Although the model was trained with the dot product, it usually performs better with cosine distance.
Evaluation
Legend:
- Acc is the Accuracy at 10
- NDCG is the Normalized Discounted Cumulative Gain at 10
🔧 Technical Details
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})
)
📄 License
CC BY - SA 4.0
Additional Information
Model Creators
The model was created by Piotr Rybak from the Institute of Computer Science, Polish Academy of Sciences.
This work was supported by the European Regional Development Fund as a part of 2014–2020 Smart Growth Operational Programme, CLARIN — Common Language Resources and Technology Infrastructure, project no. POIR.04.02.00 - 00C002/19.
Citation Information
@inproceedings{rybak-ogrodniczuk-2024-silver-retriever,
title = "Silver Retriever: Advancing Neural Passage Retrieval for {P}olish Question Answering",
author = "Rybak, Piotr and
Ogrodniczuk, Maciej",
editor = "Calzolari, Nicoletta and
Kan, Min-Yen and
Hoste, Veronique and
Lenci, Alessandro and
Sakti, Sakriani and
Xue, Nianwen",
booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
month = may,
year = "2024",
address = "Torino, Italia",
publisher = "ELRA and ICCL",
url = "https://aclanthology.org/2024.lrec-main.1291",
pages = "14826--14831",
abstract = "Modern open-domain question answering systems often rely on accurate and efficient retrieval components to find passages containing the facts necessary to answer the question. Recently, neural retrievers have gained popularity over lexical alternatives due to their superior performance. However, most of the work concerns popular languages such as English or Chinese. For others, such as Polish, few models are available. In this work, we present Silver Retriever, a neural retriever for Polish trained on a diverse collection of manually or weakly labeled datasets. Silver Retriever achieves much better results than other Polish models and is competitive with larger multilingual models. Together with the model, we open-source five new passage retrieval datasets.",
}