Mobilebert Uncased Finetuned Squadv1

🚀 mobilebert-uncased-finetuned-squadv1

This model is a fine - tuned version of the mobilebert - uncased model, designed to solve question - answering tasks on the SQuADv1 dataset, offering high accuracy and efficiency.

🚀 Quick Start

This model is a finetuned version of the mobilebert-uncased model on the SQuADv1 task. To make this TPU - trained model stable when used in PyTorch on GPUs, the original model has been additionally pretrained for one epoch on BookCorpus and English Wikipedia with disabled dropout before finetuning on the SQuADv1 task.

✨ Features

• It is produced as part of the work on the paper The Optimal BERT Surgeon: Scalable and Accurate Second - Order Pruning for Large Language Models.
• On the SQuADv1 dev - set, it achieves the following performance:

EM = 83.96
F1 = 90.90

• The source code can be found at https://github.com/neuralmagic/sparseml/tree/main/research/optimal_BERT_surgeon_oBERT.

📚 Documentation

If you find the model useful, please consider citing our work.

📄 License

Citation info

@article{kurtic2022optimal,
  title={The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models},
  author={Kurtic, Eldar and Campos, Daniel and Nguyen, Tuan and Frantar, Elias and Kurtz, Mark and Fineran, Benjamin and Goin, Michael and Alistarh, Dan},
  journal={arXiv preprint arXiv:2203.07259},
  year={2022}
}