đ Cross-Encoder for MS Marco
This model is designed for the MS Marco Passage Ranking task, which can be effectively applied in Information Retrieval scenarios. Given a query, it encodes the query with all possible passages (e.g., retrieved by ElasticSearch) and then sorts the passages in descending order. For more details, refer to SBERT.net Retrieve & Re-rank. The training code is available at SBERT.net Training MS Marco.
đ Quick Start
⨠Features
- Trained on the MS Marco Passage Ranking task.
- Ideal for Information Retrieval, capable of sorting passages based on queries.
đĻ Installation
This section does not provide specific installation steps, so it is skipped.
đģ Usage Examples
Basic Usage
If you have SentenceTransformers installed, using the pre - trained models is straightforward:
from sentence_transformers import CrossEncoder
model = CrossEncoder('cross-encoder/ms-marco-TinyBERT-L2')
scores = model.predict([
("How many people live in Berlin?", "Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers."),
("How many people live in Berlin?", "Berlin is well known for its museums."),
])
print(scores)
Advanced Usage
You can also use the model with the transformers
library:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model = AutoModelForSequenceClassification.from_pretrained('cross-encoder/ms-marco-TinyBERT-L2')
tokenizer = AutoTokenizer.from_pretrained('cross-encoder/ms-marco-TinyBERT-L2')
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'], padding=True, truncation=True, return_tensors="pt")
model.eval()
with torch.no_grad():
scores = model(**features).logits
print(scores)
đ Documentation
Performance
The following table presents various pre - trained Cross - Encoders and their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.
Model-Name |
NDCG@10 (TREC DL 19) |
MRR@10 (MS Marco Dev) |
Docs / Sec |
Version 2 models |
|
|
|
cross-encoder/ms-marco-TinyBERT-L2-v2 |
69.84 |
32.56 |
9000 |
cross-encoder/ms-marco-MiniLM-L2-v2 |
71.01 |
34.85 |
4100 |
cross-encoder/ms-marco-MiniLM-L4-v2 |
73.04 |
37.70 |
2500 |
cross-encoder/ms-marco-MiniLM-L6-v2 |
74.30 |
39.01 |
1800 |
cross-encoder/ms-marco-MiniLM-L12-v2 |
74.31 |
39.02 |
960 |
Version 1 models |
|
|
|
cross-encoder/ms-marco-TinyBERT-L2 |
67.43 |
30.15 |
9000 |
cross-encoder/ms-marco-TinyBERT-L4 |
68.09 |
34.50 |
2900 |
cross-encoder/ms-marco-TinyBERT-L6 |
69.57 |
36.13 |
680 |
cross-encoder/ms-marco-electra-base |
71.99 |
36.41 |
340 |
Other models |
|
|
|
nboost/pt-tinybert-msmarco |
63.63 |
28.80 |
2900 |
nboost/pt-bert-base-uncased-msmarco |
70.94 |
34.75 |
340 |
nboost/pt-bert-large-msmarco |
73.36 |
36.48 |
100 |
Capreolus/electra-base-msmarco |
71.23 |
36.89 |
340 |
amberoad/bert-multilingual-passage-reranking-msmarco |
68.40 |
35.54 |
330 |
sebastian-hofstaetter/distilbert-cat-margin_mse-T2-msmarco |
72.82 |
37.88 |
720 |
Note: Runtime was computed on a V100 GPU.
đ License
This model is licensed under the Apache - 2.0 license.