🚀 BERT-Small fine-tuned on SQuAD v2
This model, BERT-Small developed by Google Research, is fine-tuned on SQuAD 2.0 for the Q&A downstream task. It offers a practical solution for question - answering scenarios, with a model size (after training) of 109.74 MB.
✨ Features
Details of BERT-Small and its 'family'
Released on March 11th, 2020, this model is part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) as referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models. These smaller BERT models are designed for environments with limited computational resources. They can be fine-tuned like the original BERT models, and are particularly effective in knowledge distillation, where fine-tuning labels are generated by a larger and more accurate teacher.
Details of the downstream task (Q&A) - Dataset
SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to resemble answerable ones. To perform well on SQuAD2.0, systems need to not only answer questions when possible but also determine when no answer is supported by the paragraph and refrain from answering.
Dataset |
Split |
# samples |
SQuAD2.0 |
train |
130k |
SQuAD2.0 |
eval |
12.3k |
📦 Installation
There is no specific installation information provided in the original README. If you want to use this model, you can refer to the official Hugging Face documentation and the fine - tuning script here.
💻 Usage Examples
Basic Usage
from transformers import pipeline
qa_pipeline = pipeline(
"question-answering",
model="mrm8488/bert-small-finetuned-squadv2",
tokenizer="mrm8488/bert-small-finetuned-squadv2"
)
qa_pipeline({
'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
'question': "Who has been working hard for hugginface/transformers lately?"
})
{
"answer": "Manuel Romero",
"end": 13,
"score": 0.9939319924374637,
"start": 0
}
Advanced Usage
qa_pipeline({
'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
'question': "For which company has worked Manuel Romero?"
})
{
"answer": "hugginface/transformers",
"end": 79,
"score": 0.6024888734447131,
"start": 56
}
🔧 Technical Details
Model training
The model was trained on a Tesla P100 GPU and 25GB of RAM. The script for fine - tuning can be found here
Results
Metric |
# Value |
EM |
60.49 |
F1 |
64.21 |
Comparison
📄 License
No license information is provided in the original README.
Created by Manuel Romero/@mrm8488 | LinkedIn
Made with ♥ in Spain