Langcache Crossencoder V1 Ms Marco MiniLM L6 V2

🚀 Redis semantic caching CrossEncoder model fine-tuned on Quora Question Pairs

This is a Cross Encoder model that computes scores for pairs of texts, which can be used for sentence pair classification. It is fine-tuned from cross-encoder/ms-marco-MiniLM-L6-v2 on the Quora Question Pairs LangCache Train Set dataset using the sentence-transformers library.

✨ Features

• Cross-Encoder Classification: Capable of performing cross-encoder classification tasks.
• Multiple Metrics Evaluation: Evaluated with metrics such as accuracy, F1, precision, recall, etc.
• Fine-tuned on Quora Data: Trained on the Quora Question Pairs dataset for better performance in relevant scenarios.

📦 Installation

First, install the Sentence Transformers library:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("aditeyabaral-redis/langcache-crossencoder-v1-ms-marco-MiniLM-L6-v2")
# Get scores for pairs of texts
pairs = [
    ['How can I get a list of my Gmail accounts?', 'How can I find all my old Gmail accounts?'],
    ['How can I stop Quora from modifying and editing other people’s questions on Quora?', 'Can I prevent a Quora user from editing my question on Quora?'],
    ['How much does it cost to design a logo in india?', 'How much does it cost to design a logo?'],
    ['What is screenedrenters.com?', 'What is allmyapps.com?'],
    ['What are the best colleges for an MBA in Australia?', 'What are the top MBA schools in Australia?'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

Advanced Usage

# Rank different texts based on similarity to a single text
ranks = model.rank(
    'How can I get a list of my Gmail accounts?',
    [
        'How can I find all my old Gmail accounts?',
        'Can I prevent a Quora user from editing my question on Quora?',
        'How much does it cost to design a logo?',
        'What is allmyapps.com?',
        'What are the top MBA schools in Australia?',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

📚 Documentation

Model Details

Model Description

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Evaluation

Metrics

Training Details

Training Dataset

• Quora Question Pairs LangCache Train Set
  • Size: 363,861 training samples
  • Columns: sentence1, sentence2, and label
  • Approximate statistics based on the first 1000 samples:

    	sentence1	sentence2	label
    type	string	string	int
    details	min: 15 characters mean: 60.22 characters max: 229 characters	min: 14 characters mean: 60.0 characters max: 274 characters	0: ~63.50% 1: ~36.50%

  • Samples:

    sentence1	sentence2	label
    Why do people believe in God and how can they say he/she exists?	Why do we kill each other in the name of God?	0
    What are the chances of a bee sting when a bee buzzes around you?	How can I tell if my bees are agitated/likely to sting?	0
    If a man from Syro Malankara church marries a Syro-Malabar girl, can they join a Syro-Malabar parish?	Is Malabar Hills of Mumbai anyhow related to Malabar of Kerala?	0

  • Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Evaluation Dataset

• Quora Question Pairs LangCache Validation Set
  • Size: 40,429 evaluation samples
  • Columns: sentence1, sentence2, and label
  • Approximate statistics based on the first 1000 samples:

    	sentence1	sentence2	label
    type	string	string	int
    details	min: 13 characters mean: 59.91 characters max: 266 characters	min: 13 characters mean: 59.51 characters max: 293 characters	0: ~63.80% 1: ~36.20%

  • Samples:

    sentence1	sentence2	label
    How can I get a list of my Gmail accounts?	How can I find all my old Gmail accounts?	1
    How can I stop Quora from modifying and editing other people’s questions on Quora?	Can I prevent a Quora user from editing my question on Quora?	1
    How much does it cost to design a logo in india?	How much does it cost to design a logo?	0

  • Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 0.0002
• num_train_epochs: 15
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: aditeyabaral-redis/langcache-crossencoder-v1-ms-marco-MiniLM-L6-v2

All Hyperparameters

Training Logs

📄 License

This project is licensed under the Apache-2.0 license.