Pd Bert

🚀 Model Card for Fine-Tuned BERT for Paraphrase Detection

This is a fine-tuned BERT-base model for paraphrase detection. It's trained on four benchmark datasets (MRPC, QQP, PAWS-X, and PIT). The model is useful for applications like duplicate content detection, question answering, and semantic similarity analysis, with strong recall capabilities for identifying paraphrases in complex sentence structures.

🚀 Quick Start

To use the model, install transformers and load the fine-tuned model as follows:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

# Load the tokenizer and model
model_path = "viswadarshan06/pd-bert"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForSequenceClassification.from_pretrained(model_path)

# Encode sentence pairs
inputs = tokenizer("The car is fast.", "The vehicle moves quickly.", return_tensors="pt", padding=True, truncation=True)

# Get predictions
outputs = model(**inputs)
logits = outputs.logits
predicted_class = logits.argmax().item()
print("Paraphrase" if predicted_class == 1 else "Not a Paraphrase")

✨ Features

• Direct Use:
  • Identify duplicate questions in customer support and FAQs.
  • Improve semantic search in retrieval-based systems.
  • Enhance document deduplication and text similarity applications.
• Downstream Use: Can be further fine-tuned on domain - specific paraphrase datasets for industries like healthcare, legal, and finance.

📦 Installation

To use the model, you need to install the transformers library. You can install it via pip:

pip install transformers

📚 Documentation

Model Description

• Developed by: Viswadarshan R R
• Model Type: Transformer-based Sentence Pair Classifier
• Language: English
• Finetuned from: bert-base-cased

Model Sources

• Repository: Hugging Face Model Hub
• Research Paper: Comparative Insights into Modern Architectures for Paraphrase Detection (Accepted at ICCIDS 2025)
• Demo: (To be added upon deployment)

Uses

Direct Use

• Identifying duplicate questions in customer support and FAQs.
• Improving semantic search in retrieval-based systems.
• Enhancing document deduplication and text similarity applications.

Downstream Use

This model can be further fine-tuned on domain-specific paraphrase datasets for industries such as healthcare, legal, and finance.

Out-of-Scope Use

• The model is monolingual and trained only on English datasets, requiring additional fine-tuning for multilingual tasks.
• May struggle with idiomatic expressions or complex figurative language.

Bias, Risks, and Limitations

Known Limitations

• Higher recall but lower precision: The model tends to over-identify paraphrases, leading to increased false positives.
• Contextual ambiguity: May misinterpret sentences that require deep contextual reasoning.

Recommendations

Users can mitigate the false positive rate by applying post-processing techniques or confidence threshold tuning.

🔧 Technical Details

Training Details

This model was trained using a combination of four datasets:

• MRPC: News-based paraphrases.
• QQP: Duplicate question detection.
• PAWS-X: Adversarial paraphrases for robustness testing.
• PIT: Short-text paraphrase dataset.

Training Procedure

• Tokenizer: BERT Tokenizer
• Batch Size: 16
• Optimizer: AdamW
• Loss Function: Cross-entropy

Training Hyperparameters

• Learning Rate: 2e-5
• Sequence Length:
  • MRPC: 256
  • QQP: 336
  • PIT: 64
  • PAWS-X: 256

Speeds, Sizes, Times

• GPU Used: NVIDIA A100
• Total Training Time: ~6 hours
• Compute Units Used: 80

Testing Data, Factors & Metrics

Testing Data

The model was tested on combined test sets and evaluated using:

• Accuracy
• Precision
• Recall
• F1-Score
• Runtime

Results

BERT Model Evaluation Metrics

Summary

This BERT-based Paraphrase Detection Model demonstrates strong recall capabilities, making it highly effective at identifying paraphrases across varied linguistic structures. While it tends to overpredict paraphrases, it remains a strong baseline for semantic similarity tasks and can be fine-tuned further for domain-specific applications.

Citation

If you use this model, please cite:

@inproceedings{viswadarshan2025paraphrase,
   title={Comparative Insights into Modern Architectures for Paraphrase Detection},
   author={Viswadarshan R R, Viswaa Selvam S, Felcia Lilian J, Mahalakshmi S},
   booktitle={International Conference on Computational Intelligence, Data Science, and Security (ICCIDS)},
   year={2025},
   publisher={IFIP AICT Series by Springer}
}

📄 License

This project is licensed under the MIT license.

Model Card Contact

📧 Email: viswadarshanrramiya@gmail.com 🔗 GitHub: Viswadarshan R R