Deberta Base Mnli

🚀 DeBERTa: Decoding-enhanced BERT with Disentangled Attention

DeBERTa is a model that enhances BERT and RoBERTa by using disentangled attention and an enhanced mask decoder. It outperforms BERT and RoBERTa on most NLU tasks with 80GB of training data.

🚀 Quick Start

This model is the base DeBERTa model fine - tuned with the MNLI task. For more details and updates, please check the official repository.

✨ Features

Fine - tuning on NLU tasks

We present the dev results on SQuAD 1.1/2.0 and MNLI tasks.

📄 License

This model is released under the MIT license.

📚 Documentation

Citation

If you find DeBERTa useful for your work, please cite the following paper:

@inproceedings{
he2021deberta,
title={DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION},
author={Pengcheng He and Xiaodong Liu and Jianfeng Gao and Weizhu Chen},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=XPZIaotutsD}
}