Langcache Crossencoder V1 Ms Marco MiniLM L12 V2

🚀 Redis semantic caching CrossEncoder model fine-tuned on Quora Question Pairs

This is a Cross Encoder model that addresses the need for accurate semantic similarity scoring between sentence pairs. It's fine - tuned from a pre - existing model, offering enhanced performance for tasks like sentence - pair classification and semantic search.

🚀 Quick Start

Direct Usage (Sentence Transformers)

First, you need to install the Sentence Transformers library:

pip install -U sentence-transformers

Then, you can load this model and run inference:

from sentence_transformers import CrossEncoder

# Download from the Hub
model = CrossEncoder("aditeyabaral-redis/langcache-crossencoder-v1-ms-marco-MiniLM-L12-v2")
# Get scores for pairs of texts
pairs = [
    ['How can I get a list of my Gmail accounts?', 'How can I find all my old Gmail accounts?'],
    ['How can I stop Quora from modifying and editing other people’s questions on Quora?', 'Can I prevent a Quora user from editing my question on Quora?'],
    ['How much does it cost to design a logo in india?', 'How much does it cost to design a logo?'],
    ['What is screenedrenters.com?', 'What is allmyapps.com?'],
    ['What are the best colleges for an MBA in Australia?', 'What are the top MBA schools in Australia?'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'How can I get a list of my Gmail accounts?',
    [
        'How can I find all my old Gmail accounts?',
        'Can I prevent a Quora user from editing my question on Quora?',
        'How much does it cost to design a logo?',
        'What is allmyapps.com?',
        'What are the top MBA schools in Australia?',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

✨ Features

• Cross - Encoder Classification: Capable of performing cross - encoder classification tasks, which is useful for determining the relationship between sentence pairs.
• Multiple Metrics: Evaluated using multiple metrics such as accuracy, F1 score, precision, recall, and average precision, ensuring comprehensive performance assessment.

📦 Model Details

Model Description

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence - transformers)
• Hugging Face: [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence - transformers&other=cross - encoder)

📚 Evaluation

Metrics

Cross Encoder Classification

• Dataset: quora - eval
• Evaluated with CrossEncoderClassificationEvaluator

🔧 Training Details

Training Dataset

Quora Question Pairs LangCache Train Set

• Dataset: Quora Question Pairs LangCache Train Set
• Size: 363,861 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 15 characters mean: 60.22 characters max: 229 characters	min: 14 characters mean: 60.0 characters max: 274 characters	0: ~63.50% 1: ~36.50%

• Samples:

  sentence1	sentence2	label
  Why do people believe in God and how can they say he/she exists?	Why do we kill each other in the name of God?	0
  What are the chances of a bee sting when a bee buzzes around you?	How can I tell if my bees are agitated/likely to sting?	0
  If a man from Syro Malankara church marries a Syro - Malabar girl, can they join a Syro - Malabar parish?	Is Malabar Hills of Mumbai anyhow related to Malabar of Kerala?	0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Evaluation Dataset

Quora Question Pairs LangCache Validation Set

• Dataset: Quora Question Pairs LangCache Validation Set
• Size: 40,429 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	int
  details	min: 13 characters mean: 59.91 characters max: 266 characters	min: 13 characters mean: 59.51 characters max: 293 characters	0: ~63.80% 1: ~36.20%

• Samples:

  sentence1	sentence2	label
  How can I get a list of my Gmail accounts?	How can I find all my old Gmail accounts?	1
  How can I stop Quora from modifying and editing other people’s questions on Quora?	Can I prevent a Quora user from editing my question on Quora?	1
  How much does it cost to design a logo in india?	How much does it cost to design a logo?	0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non - Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 0.0002
• num_train_epochs: 15
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: aditeyabaral - redis/langcache - crossencoder - v1 - ms - marco - MiniLM - L12 - v2

All Hyperparameters

Training Logs

📄 License

This model is licensed under the apache - 2.0 license.