Bert Base Uncased Finetuned Newsqa

🚀 BERT Base Uncased Finetuned on NewsQA

This project finetunes the BERT (Base) model on the NewsQA dataset. It uses a modified version of the run_squad.py legacy script from the Transformers library, which is provided in this repository. During the training process, examples with noAnswer and badQuestion are excluded.

🚀 Quick Start

To start the finetuning process, follow these steps:

$ cd ~/projects/transformers/examples/legacy/question-answering
$ mkdir bert_base_uncased_finetuned_newsqa
$ python run_newsqa.py \
    --model_type bert \
    --model_name_or_path "bert-base-uncased" \
    --do_train \
    --do_eval \
    --do_lower_case \
    --num_train_epochs 2 \
    --per_gpu_train_batch_size 8 \
    --per_gpu_eval_batch_size 32 \
    --max_seq_length 384 \
    --max_grad_norm inf \
    --doc_stride 128 \
    --train_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
    --predict_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
    --output_dir "./bert_base_uncased_finetuned_newsqa" \
    --save_steps 20000

💻 Usage Examples

Basic Usage

The above code snippet shows the basic process of finetuning the BERT Base Uncased model on the NewsQA dataset.

Advanced Usage

To evaluate the finetuned model, use the following command:

python run_newsqa.py \
  --model_type bert \
  --model_name_or_path "./bert_large_uncased_finetuned_newsqa/checkpoint-700000" \
  --do_eval \
  --do_lower_case \
  --per_gpu_eval_batch_size 32 \
  --max_seq_length 384 \
  --max_grad_norm inf \
  --doc_stride 128 \
  --predict_file "~/projects/data/newsqa/combined-newsqa-data-v1.json" \
  --output_dir "./bert_large_uncased_finetuned_newsqa_eval"

📦 Installation

Before running the above commands, you need to prepare the database. Follow the instructions on the NewsQA repository.

🔧 Technical Details

Results

The following are the evaluation results of the finetuned model:

{'exact': 60.19350380096752, 'f1': 73.29371985128037, 'total': 4341, 'HasAns_exact': 60.19350380096752, 'HasAns_f1': 73.29371985128037, 'HasAns_total': 4341, 'best_exact': 60.19350380096752, 'best_exact_thresh': 0.0, 'best_f1': 73.29371985128037, 'best_f1_thresh': 0.0}

📄 License

This project is licensed under the Apache-2.0 license.