Ft Ms Marco MiniLM L12 V2 Claims Reranker V2

🚀 CrossEncoder based on cross-encoder/ms-marco-MiniLM-L12-v2

This is a Cross Encoder model that computes scores for text pairs, useful for text reranking and semantic search. It's finetuned from cross-encoder/ms-marco-MiniLM-L12-v2 using the sentence-transformers library.

🚀 Quick Start

Prerequisites

First, install the Sentence Transformers library:

pip install -U sentence-transformers

Usage Example

Then you can load this model and run inference:

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("Davidsamuel101/ft-ms-marco-MiniLM-L12-v2-claims-reranker-v2")
# Get scores for pairs of texts
pairs = [
    ['Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.', 'At very high concentrations (100 times atmospheric concentration, or greater), carbon dioxide can be toxic to animal life, so raising the concentration to 10,000 ppm (1%) or higher for several hours will eliminate pests such as whiteflies and spider mites in a greenhouse.'],
    ['Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.', 'Plants can grow as much as 50 percent faster in concentrations of 1,000 ppm CO 2 when compared with ambient conditions, though this assumes no change in climate and no limitation on other nutrients.'],
    ['Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.', 'Higher carbon dioxide concentrations will favourably affect plant growth and demand for water.'],
    ['Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.', "Carbon dioxide in the Earth's atmosphere is essential to life and to most of the planetary biosphere."],
    ['Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.', 'Rennie 2009: "Claim 1: Anthropogenic CO2 can\'t be changing climate, because CO2 is only a trace gas in the atmosphere and the amount produced by humans is dwarfed by the amount from volcanoes and other natural sources.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.',
    [
        'At very high concentrations (100 times atmospheric concentration, or greater), carbon dioxide can be toxic to animal life, so raising the concentration to 10,000 ppm (1%) or higher for several hours will eliminate pests such as whiteflies and spider mites in a greenhouse.',
        'Plants can grow as much as 50 percent faster in concentrations of 1,000 ppm CO 2 when compared with ambient conditions, though this assumes no change in climate and no limitation on other nutrients.',
        'Higher carbon dioxide concentrations will favourably affect plant growth and demand for water.',
        "Carbon dioxide in the Earth's atmosphere is essential to life and to most of the planetary biosphere.",
        'Rennie 2009: "Claim 1: Anthropogenic CO2 can\'t be changing climate, because CO2 is only a trace gas in the atmosphere and the amount produced by humans is dwarfed by the amount from volcanoes and other natural sources.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

✨ Features

• Cross Encoder Model: Computes scores for text pairs, enabling text reranking and semantic search.
• Finetuned: Based on cross-encoder/ms-marco-MiniLM-L12-v2, enhancing performance on specific tasks.

📚 Documentation

Model Details

Model Description

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Evaluation

Metrics - Cross Encoder Reranking

• Dataset: claims-evidence-dev
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 5,
      "always_rerank_positives": true
  }
  

Training Details

Training Dataset - Unnamed Dataset

• Size: 23,770 training samples
• Columns: text1, text2, and label
• Approximate statistics based on the first 1000 samples:

  	text1	text2	label
  type	string	string	int
  details	min: 38 characters mean: 118.57 characters max: 226 characters	min: 14 characters mean: 144.96 characters max: 1176 characters	0: ~83.70% 1: ~16.30%

• Samples:

  text1	text2	label
  Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.	At very high concentrations (100 times atmospheric concentration, or greater), carbon dioxide can be toxic to animal life, so raising the concentration to 10,000 ppm (1%) or higher for several hours will eliminate pests such as whiteflies and spider mites in a greenhouse.	1
  Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.	Plants can grow as much as 50 percent faster in concentrations of 1,000 ppm CO 2 when compared with ambient conditions, though this assumes no change in climate and no limitation on other nutrients.	1
  Not only is there no scientific evidence that CO2 is a pollutant, higher CO2 concentrations actually help ecosystems support more plant and animal life.	Higher carbon dioxide concentrations will favourably affect plant growth and demand for water.	1

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 10.0,
      "num_negatives": 4,
      "activation_fn": "torch.nn.modules.activation.Sigmoid"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• learning_rate: 3e-06
• num_train_epochs: 5
• bf16: True
• load_best_model_at_end: True

All Hyperparameters