BERT-Mini Fine-Tuned Open-Source Question-Answering Model - Suitable for Resource-Constrained Environments, Free Support for Question-Answering Tasks

Bert Mini Finetuned Squadv2

Developed by mrm8488

BERT-Mini is a small BERT model developed by Google Research, fine-tuned on the SQuAD 2.0 dataset for question-answering tasks, suitable for environments with limited computational resources.

Question Answering System English#Lightweight Q&A #SQuADv2 adaptation #Small model fine-tuning

Downloads 50

Release Time : 3/2/2022

Model Overview

This model is a compact version of BERT, specifically designed for question-answering tasks, capable of determining whether a passage supports an answer and rejecting unsolvable questions.

Model Features

Compact and Efficient

Designed for environments with limited computational resources, the model is small in size but delivers excellent performance.

Question-Answering Capability

Capable of handling both answerable and unanswerable questions, determining whether a passage supports an answer.

Knowledge Distillation Friendly

Particularly suitable as a student model to work alongside larger teacher models.

Model Capabilities

Question Answering System

Text Understanding

Answer Localization

Use Cases

Intelligent Customer Service

Automated Q&A System

Used to build customer service bots that understand questions and provide accurate answers.

Achieved 56.31 EM and 59.65 F1 on the SQuAD2.0 evaluation set.

Educational Technology

Learning Assistance System

Helps students quickly find answers to questions from textual materials.

🚀 BERT-Mini fine-tuned on SQuAD v2

This project presents BERT-Mini, developed by Google Research, and fine-tuned on SQuAD 2.0 for the Q&A downstream task. It offers an efficient solution for question - answering tasks, especially in resource - constrained environments.

Model size (after training): 42.63 MB

✨ Features

BERT-Mini and its 'family'

Released on March 11th, 2020, this model is part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) mentioned in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models. These smaller models are designed for environments with limited computational resources. They can be fine - tuned like the original BERT models, and are particularly effective in knowledge distillation scenarios, where fine - tuning labels are generated by a larger and more accurate teacher model.

Downstream task (Q&A) - Dataset

SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to resemble answerable ones. To perform well on SQuAD2.0, systems need to not only answer questions when possible but also determine when no answer is supported by the paragraph and refrain from answering.

Dataset	Split	# samples
SQuAD2.0	train	130k
SQuAD2.0	eval	12.3k

📦 Installation

The script for fine - tuning can be found here

💻 Usage Examples

Basic Usage

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-mini-finetuned-squadv2",
    tokenizer="mrm8488/bert-mini-finetuned-squadv2"
)

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "Who has been working hard for hugginface/transformers lately?"

})

# Output:

{
  "answer": "Manuel Romero",
  "end": 13,
  "score": 0.9676484207783673,
  "start": 0
}

Advanced Usage

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "For which company has worked Manuel Romero?"
})

# Output:

{
  "answer": "hugginface/transformers",
  "end": 79,
  "score": 0.5301655914731853,
  "start": 56
}

🔧 Technical Details

Model training

The model was trained on a Tesla P100 GPU and 25GB of RAM.

Results

Metric	# Value
EM	56.31
F1	59.65

Comparison

Model	EM	F1 score	SIZE (MB)
bert-tiny-finetuned-squadv2	48.60	49.73	16.74
bert-tiny-5-finetuned-squadv2	57.12	60.86	24.34
bert-mini-finetuned-squadv2	56.31	59.65	42.63
bert-mini-5-finetuned-squadv2	63.51	66.78	66.76

📄 License

Not provided in the original README.

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with ♥ in Spain

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご