Ms Marco TinyBERT L2

🚀 Cross-Encoder for MS Marco

This model is designed for the MS Marco Passage Ranking task, which can be effectively applied in Information Retrieval scenarios. Given a query, it encodes the query with all possible passages (e.g., retrieved by ElasticSearch) and then sorts the passages in descending order. For more details, refer to SBERT.net Retrieve & Re-rank. The training code is available at SBERT.net Training MS Marco.

🚀 Quick Start

✨ Features

• Trained on the MS Marco Passage Ranking task.
• Ideal for Information Retrieval, capable of sorting passages based on queries.

📦 Installation

This section does not provide specific installation steps, so it is skipped.

💻 Usage Examples

Basic Usage

If you have SentenceTransformers installed, using the pre - trained models is straightforward:

from sentence_transformers import CrossEncoder

model = CrossEncoder('cross-encoder/ms-marco-TinyBERT-L2')
scores = model.predict([
    ("How many people live in Berlin?", "Berlin had a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers."),
    ("How many people live in Berlin?", "Berlin is well known for its museums."),
])
print(scores)
# [0.82869005 0.00169255]

Advanced Usage

You can also use the model with the transformers library:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model = AutoModelForSequenceClassification.from_pretrained('cross-encoder/ms-marco-TinyBERT-L2')
tokenizer = AutoTokenizer.from_pretrained('cross-encoder/ms-marco-TinyBERT-L2')

features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], ['Berlin has a population of 3,520,031 registered inhabitants in an area of 891.82 square kilometers.', 'New York City is famous for the Metropolitan Museum of Art.'],  padding=True, truncation=True, return_tensors="pt")

model.eval()
with torch.no_grad():
    scores = model(**features).logits
    print(scores)

📚 Documentation

Performance

The following table presents various pre - trained Cross - Encoders and their performance on the TREC Deep Learning 2019 and the MS Marco Passage Reranking dataset.

Note: Runtime was computed on a V100 GPU.

📄 License

This model is licensed under the Apache - 2.0 license.