Deberta V3 Xsmall Squad2

🚀 DeBERTa v3 xsmall SQuAD 2.0

This model is designed for question - answering tasks. It improves upon previous models and shows significant performance on downstream tasks. Microsoft reports that this model can achieve 84.8/82.0 on f1/em on the dev set.

🚀 Quick Start

Microsoft reports that this model can get 84.8/82.0 on f1/em on the dev set.

I got 81.5/78.3 but I only did one run and I didn't use the official squad2 evaluation script. I will do some more runs and show the results on the official script soon.

✨ Features

• Improved Architecture: DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder.
• Enhanced Efficiency: In DeBERTa V3, the efficiency of DeBERTa is further improved using ELECTRA - Style pre - training with Gradient Disentangled Embedding Sharing.
• Better Performance: Compared to DeBERTa, the V3 version significantly improves the model performance on downstream tasks.

📄 License

This project is licensed under the CC - BY - 4.0 license.

📚 Documentation

Model Information

Widget Examples

• DeBERTa v3 Q1:
  • Context: DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data. In DeBERTa V3, we further improved the efficiency of DeBERTa using ELECTRA - Style pre - training with Gradient Disentangled Embedding Sharing. Compared to DeBERTa, our V3 version significantly improves the model performance on downstream tasks. You can find more technique details about the new model from our paper. Please check the official repository for more implementation details and updates.
  • Question: How is DeBERTa version 3 different than previous ones?
• DeBERTa v3 Q2:
  • Context: Same as above.
  • Question: Where do I go to see new info about DeBERTa?

Results