Bert Large Uncased Squadv1.1 Sparse 80 1x4 Block Pruneofa

🚀 80% 1x4 Block Sparse BERT-Large (uncased) Fine Tuned on SQuADv1.1

This model is fine-tuned from a Prune OFA 80% 1x4 block sparse pre-trained BERT-Large with knowledge distillation, offering high performance on the SQuADv1.1 dataset.

🚀 Quick Start

This model is the outcome of fine - tuning a Prune OFA 80% 1x4 block sparse pre - trained BERT - Large, integrated with knowledge distillation. On the SQuADv1.1 development set, this model achieves the following results: {"exact_match": 84.673, "f1": 91.174}

For more in - depth details, refer to our paper, Prune Once for All: Sparse Pre - Trained Language Models, and our open - source implementation available here.

📄 License

This project is licensed under the Apache - 2.0 license.