Bert-tiny-5-finetuned-squadv2 Open-source Q&A Model - Compact and Lightweight, Suitable for Resource-constrained Environments

Bert Tiny 5 Finetuned Squadv2

Developed by mrm8488

This model is fine-tuned from BERT-Tiny on the SQuAD 2.0 dataset for question-answering tasks, with a compact size of only 24.33MB, making it suitable for resource-constrained environments.

Question Answering System English#Mini QA Model #SQuAD2.0 Adaptation #Lightweight BERT

Downloads 1,267

Release Time : 3/2/2022

Model Overview

A question-answering system fine-tuned from Google's BERT-Tiny model, specifically optimized for the SQuAD 2.0 dataset, capable of handling both answerable and unanswerable questions.

Model Features

Lightweight Design

The model is only 24.33MB in size, optimized for environments with limited computational resources.

Adversarial Question Handling

Capable of identifying and refusing to answer adversarial unanswerable questions in SQuAD2.0.

Efficient Fine-tuning

Based on a small BERT model, significantly reducing computational resource requirements while maintaining performance.

Model Capabilities

Reading Comprehension

Question Answering

Text Understanding

Adversarial Question Recognition

Use Cases

Educational Technology

Automated QA System

Used in educational platforms to automatically answer students' questions about textbook content.

Can accurately answer textbook-related questions and identify invalid queries.

Customer Service

FAQ Auto-Response

Handles common customer inquiries, identifying and filtering unanswerable questions.

Reduces workload for human agents and improves response speed.

🚀 BERT-Tiny (5) fine-tuned on SQuAD v2

This model is BERT-Tiny created by Google Research and fine-tuned on SQuAD 2.0 for the Q&A downstream task.

✨ Features

Compact Size: The model size after training is only 24.33 MB, suitable for environments with restricted computational resources.
Fine-tuned on SQuAD 2.0: Trained on the SQuAD 2.0 dataset, it can handle both answerable and unanswerable questions effectively.

📚 Documentation

Details of BERT-Tiny and its 'family'

Released on March 11th, 2020, this model is part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models. These smaller BERT models are designed for environments with limited computational resources. They can be fine-tuned in the same way as the original BERT models, and are most effective in knowledge distillation scenarios where fine-tuning labels are generated by a larger and more accurate teacher.

Details of the downstream task (Q&A) - Dataset

SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To perform well on SQuAD2.0, systems must not only answer questions when possible but also determine when no answer is supported by the paragraph and refrain from answering.

Dataset	Split	# samples
SQuAD2.0	train	130k
SQuAD2.0	eval	12.3k

Model training

The model was trained on a Tesla P100 GPU with 25GB of RAM. The script for fine-tuning can be found here.

Results

Metric	# Value
EM	57.12
F1	60.86

Model	EM	F1 score	SIZE (MB)
bert-tiny-finetuned-squadv2	48.60	49.73	16.74
bert-tiny-5-finetuned-squadv2	57.12	60.86	24.34

💻 Usage Examples

Basic Usage

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-tiny-5-finetuned-squadv2",
    tokenizer="mrm8488/bert-tiny-5-finetuned-squadv2"
)

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "Who has been working hard for hugginface/transformers lately?"

})

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with ♥ in Spain

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご