Crossencoder Camembert Large

🚀 Sentence-Camembert-Base Model

A cross-encoder model for sentence similarity, trained to predict semantic similarity scores between sentences.

🚀 Quick Start

This model is a cross-encoder for sentence similarity, trained using the sentence-transformers Cross-Encoder class. It predicts a score between 0 and 1 for the semantic similarity of two sentences.

✨ Features

• Text Ranking: Ideal for text ranking tasks.
• Sentence Similarity: Accurately predicts the semantic similarity between sentences.
• Sentence Embedding: Generates effective sentence embeddings.

📦 Installation

Using this model becomes easy when you have sentence-transformers installed:

pip install -U sentence-transformers

💻 Usage Examples

Basic Usage

from sentence_transformers import CrossEncoder
model = CrossEncoder('dangvantuan/CrossEncoder-camembert-large', max_length=128)
scores = model.predict([('Un avion est en train de décoller.', "Un homme joue d'une grande flûte."), ("Un homme étale du fromage râpé sur une pizza.", "Une personne jette un chat au plafond") ])

📚 Documentation

Model

Cross-Encoder for sentence-similarity. This model was trained using sentence-transformers Cross-Encoder class.

Training Data

This model was trained on the STS benchmark dataset. The model will predict a score between 0 and 1 for the semantic similarity of two sentences.

Evaluation

The model can be evaluated as follows on the French test data of stsb.

from sentence_transformers.readers import InputExample
from sentence_transformers.cross_encoder.evaluation import CECorrelationEvaluator
from datasets import load_dataset
def convert_dataset(dataset):
    dataset_samples=[]
    for df in dataset:
        score = float(df['similarity_score'])/5.0  # Normalize score to range 0 ... 1
        inp_example = InputExample(texts=[df['sentence1'], 
                                    df['sentence2']], label=score)
        dataset_samples.append(inp_example)
    return dataset_samples

# Loading the dataset for evaluation
df_dev = load_dataset("stsb_multi_mt", name="fr", split="dev")
df_test = load_dataset("stsb_multi_mt", name="fr", split="test")

# Convert the dataset for evaluation

# For Dev set:
dev_samples = convert_dataset(df_dev)
val_evaluator = CECorrelationEvaluator.from_input_examples(dev_samples, name='sts-dev')
val_evaluator(model, output_path="./")

# For Test set

test_samples = convert_dataset(df_test)
test_evaluator = CECorrelationEvaluator.from_input_examples(test_samples, name='sts-test')
test_evaluator(models, output_path="./")

Test Result

The performance is measured using Pearson and Spearman correlation:

• On dev | Model | Pearson correlation | Spearman correlation | #params | | ------------- | ------------- | ------------- |------------- | | dangvantuan/CrossEncoder-camembert-large| 90.11 |90.01 | 336M |

• On test | Model | Pearson correlation | Spearman correlation | | ------------- | ------------- | ------------- | | dangvantuan/CrossEncoder-camembert-large| 88.16 | 87.57|

📄 License

This model is licensed under the Apache 2.0 license.

📋 Model Information