Viranker

🚀 Reranker

Reranker is different from embedding models. It takes questions and documents as input and directly outputs similarity scores instead of embeddings. You can obtain a relevance score by inputting a query and a passage, and this score can be mapped to a float value in the range of [0,1] using the sigmoid function.

🚀 Quick Start

✨ Features

Different from embedding model, reranker uses question and document as input and directly output similarity instead of embedding. You can get a relevance score by inputting query and passage to the reranker, and the score can be mapped to a float value in [0,1] by sigmoid function.

📦 Installation

There are two ways to install the necessary libraries for using the reranker:

Using FlagEmbedding

pip install -U FlagEmbedding

Using Huggingface transformers

pip install -U transformers

💻 Usage Examples

Basic Usage

Using FlagEmbedding

from FlagEmbedding import FlagReranker

reranker = FlagReranker('namdp-ptit/ViRanker',
                        use_fp16=True)  # Setting use_fp16 to True speeds up computation with a slight performance degradation

score = reranker.compute_score(['ai là vị vua cuối cùng của việt nam', 'vua bảo đại là vị vua cuối cùng của nước ta'])
print(score)  # 13.71875

# You can map the scores into 0-1 by set "normalize=True", which will apply sigmoid function to the score
score = reranker.compute_score(['ai là vị vua cuối cùng của việt nam', 'vua bảo đại là vị vua cuối cùng của nước ta'],
                               normalize=True)
print(score)  # 0.99999889840464

scores = reranker.compute_score(
    [
        ['ai là vị vua cuối cùng của việt nam', 'vua bảo đại là vị vua cuối cùng của nước ta'],
        ['ai là vị vua cuối cùng của việt nam', 'lý nam đế là vị vua đầu tiên của nước ta']
    ]
)
print(scores)  # [13.7265625, -8.53125]

# You can map the scores into 0-1 by set "normalize=True", which will apply sigmoid function to the score
scores = reranker.compute_score(
    [
        ['ai là vị vua cuối cùng của việt nam', 'vua bảo đại là vị vua cuối của nước ta'],
        ['ai là vị vua cuối cùng của việt nam', 'lý nam đế là vị vua đầu tiên của nước ta']
    ],
    normalize=True
)
print(scores)  # [0.99999889840464, 0.00019716942196222918]

Using Huggingface transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('namdp-ptit/ViRanker')
model = AutoModelForSequenceClassification.from_pretrained('namdp-ptit/ViRanker')
model.eval()

pairs = [
    ['ai là vị vua cuối cùng của việt nam', 'vua bảo đại là vị vua cuối cùng của nước ta'],
    ['ai là vị vua cuối cùng của việt nam', 'lý nam đế là vị vua đầu tiên của nước ta']
],
with torch.no_grad():
    inputs = tokenizer(pairs, padding=True, truncation=True, return_tensors='pt', max_length=512)
    scores = model(**inputs, return_dict=True).logits.view(-1, ).float()
    print(scores)

🔧 Technical Details

Fine tune

Data Format

Train data should be a json file, where each line is a dict like this:

{"query": str, "pos": List[str], "neg": List[str]}

query is the query, and pos is a list of positive texts, neg is a list of negative texts. If you have no negative texts for a query, you can random sample some from the entire corpus as the negatives. Besides, for each query in the train data, we used LLMs to generate hard negative for them by asking LLMs to create a document that is the opposite one of the documents in 'pos'.

Performance

Below is a comparison table of the results we achieved compared to some other pre-trained Cross-Encoders on the MS MMarco Passage Reranking - Vi - Dev dataset.

📄 License

This project is licensed under the Apache-2.0 license.

Contact

• Email: phuongnamdpn2k2@gmail.com
• LinkedIn: Dang Phuong Nam
• Facebook: Phương Nam

Support The Project

If you find this project helpful and wish to support its ongoing development, here are some ways you can contribute:

1. Star the Repository: Show your appreciation by starring the repository. Your support motivates further development and enhancements.
2. Contribute: We welcome your contributions! You can help by reporting bugs, submitting pull requests, or suggesting new features.
3. Donate: If you’d like to support financially, consider making a donation. You can donate through:
   • Vietcombank: 9912692172 - DANG PHUONG NAM

Thank you for your support!

Citation

Please cite as

@misc{ViRanker,
  title={ViRanker: A Cross-encoder Model for Vietnamese Text Ranking},
  author={Nam Dang Phuong},
  year={2024},
  publisher={Huggingface},
}