Open-source Question Answering Model cs224n-squad2.0-distilbert-base-uncased - Empowering Precise Question Answering Tasks

Cs224n Squad2.0 Distilbert Base Uncased

Developed by elgeish

This model was established as a benchmark for the CS224n student project, based on the DistilBERT architecture and fine-tuned on the SQuAD2.0 dataset for question-answering tasks.

Question Answering System

Transformers

#SQuAD2.0 Q&A #Education Scenario Optimization #Lightweight BERT

Downloads 15

Release Time : 3/2/2022

Model Overview

This model is a lightweight question-answering model based on the DistilBERT architecture, specifically optimized for the SQuAD2.0 dataset, capable of handling question-answering tasks that include unanswerable questions.

Model Features

Lightweight Architecture

Based on DistilBERT, smaller and faster than standard BERT models while maintaining high performance.

Handling Unanswerable Questions

Supports scenarios with unanswerable questions in SQuAD2.0.

Efficient Training

Optimized training parameters, suitable for student projects.

Model Capabilities

Reading Comprehension

Question-Answering System

Text Understanding

Use Cases

Education

CS224n Student Project

Serves as a benchmark model for student projects, helping students quickly start developing question-answering systems.

Provides an exact match score of 65.17 and an F1 score of 67.87.

Question-Answering System

Document-Based Question Answering

Answers user questions from given text.

Performs well on the SQuAD2.0 dataset.

🚀 CS224n SQuAD2.0 Project Dataset

This model aims to save CS224n students' GPU time when establishing baselines for the Default Final Project.

🚀 Quick Start

The goal of this model is to save CS224n students GPU time when establishing baselines to beat for the Default Final Project. The training set used to fine-tune this model is the same as the official one. However, evaluation and model selection were performed using roughly half of the official dev set, 6078 examples, picked at random. The data files can be found at https://github.com/elgeish/squad/tree/master/data — this is the Winter 2020 version.

⚠️ Important Note

Given that the official SQuAD2.0 dev set contains the project's test set, students must make sure not to use the official SQuAD2.0 dev set in any way — including the use of models fine-tuned on the official SQuAD2.0, since they used the official SQuAD2.0 dev set for model selection.

✨ Features

Results

{
  "exact": 65.16946363935504,
  "f1": 67.87348075352251,
  "total": 6078,
  "HasAns_exact": 69.51890034364261,
  "HasAns_f1": 75.16667217179045,
  "HasAns_total": 2910,
  "NoAns_exact": 61.17424242424242,
  "NoAns_f1": 61.17424242424242,
  "NoAns_total": 3168,
  "best_exact": 65.16946363935504,
  "best_exact_thresh": 0.0,
  "best_f1": 67.87348075352243,
  "best_f1_thresh": 0.0
}

Notable Arguments

{
  "do_lower_case": true,
  "doc_stride": 128,
  "fp16": false,
  "fp16_opt_level": "O1",
  "gradient_accumulation_steps": 24,
  "learning_rate": 3e-05,
  "max_answer_length": 30,
  "max_grad_norm": 1,
  "max_query_length": 64,
  "max_seq_length": 384,
  "model_name_or_path": "distilbert-base-uncased-distilled-squad",
  "model_type": "distilbert",
  "num_train_epochs": 4,
  "per_gpu_train_batch_size": 32,
  "save_steps": 5000,
  "seed": 42,
  "train_batch_size": 32,
  "version_2_with_negative": true,
  "warmup_steps": 0,
  "weight_decay": 0
}

Environment Setup

{
  "transformers": "2.5.1",
  "pytorch": "1.4.0=py3.6_cuda10.1.243_cudnn7.6.3_0",
  "python": "3.6.5=hc3d631a_2",
  "os": "Linux 4.15.0-1060-aws #62-Ubuntu SMP Tue Feb 11 21:23:22 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux",
  "gpu": "Tesla V100-SXM2-16GB"
}

📚 Documentation

How to Cite

@misc{elgeish2020gestalt,
  title={Gestalt: a Stacking Ensemble for SQuAD2.0},
  author={Mohamed El-Geish},
  journal={arXiv e-prints},
  archivePrefix={arXiv},
  eprint={2004.07067},
  year={2020},
}

Related Models

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご