Bert Small Finetuned Squadv2

🚀 BERT-Small fine-tuned on SQuAD v2

This model, BERT-Small developed by Google Research, is fine-tuned on SQuAD 2.0 for the Q&A downstream task. It offers a practical solution for question - answering scenarios, with a model size (after training) of 109.74 MB.

✨ Features

Details of BERT-Small and its 'family'

Released on March 11th, 2020, this model is part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) as referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models. These smaller BERT models are designed for environments with limited computational resources. They can be fine-tuned like the original BERT models, and are particularly effective in knowledge distillation, where fine-tuning labels are generated by a larger and more accurate teacher.

Details of the downstream task (Q&A) - Dataset

SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to resemble answerable ones. To perform well on SQuAD2.0, systems need to not only answer questions when possible but also determine when no answer is supported by the paragraph and refrain from answering.

📦 Installation

There is no specific installation information provided in the original README. If you want to use this model, you can refer to the official Hugging Face documentation and the fine - tuning script here.

💻 Usage Examples

Basic Usage

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-small-finetuned-squadv2",
    tokenizer="mrm8488/bert-small-finetuned-squadv2"
)

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "Who has been working hard for hugginface/transformers lately?"

})

# Output:

{
  "answer": "Manuel Romero",
  "end": 13,
  "score": 0.9939319924374637,
  "start": 0
}

Advanced Usage

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "For which company has worked Manuel Romero?"
})

# Output:

{
  "answer": "hugginface/transformers",
  "end": 79,
  "score": 0.6024888734447131,
  "start": 56
}

🔧 Technical Details

Model training

The model was trained on a Tesla P100 GPU and 25GB of RAM. The script for fine - tuning can be found here

Results

Comparison

📄 License

No license information is provided in the original README.

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with ♥ in Spain