Ernie 2.0 Large En

🚀 ERNIE-2.0-large

ERNIE 2.0-large is a powerful pre - trained model that outperforms many counterparts in multiple language tasks, offering high - quality language understanding capabilities.

🚀 Quick Start

from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-2.0-large-en")
model = AutoModel.from_pretrained("nghuyong/ernie-2.0-large-en")

✨ Features

ERNIE 2.0 is a continual pre - training framework proposed by Baidu in 2019. It builds and learns incrementally pre - training tasks through constant multi - task learning. Experimental results demonstrate that ERNIE 2.0 outperforms BERT and XLNet on 16 tasks including English tasks on GLUE benchmarks and several common tasks in Chinese.

More detail: https://arxiv.org/abs/1907.12412

📦 Installation

This released pytorch model is converted from the officially released PaddlePaddle ERNIE model and a series of experiments have been conducted to check the accuracy of the conversion.

• Official PaddlePaddle ERNIE repo: https://github.com/PaddlePaddle/ERNIE
• Pytorch Conversion repo: https://github.com/nghuyong/ERNIE-Pytorch

📄 License

Citation

@article{sun2019ernie20,
  title={ERNIE 2.0: A Continual Pre-training Framework for Language Understanding},
  author={Sun, Yu and Wang, Shuohuan and Li, Yukun and Feng, Shikun and Tian, Hao and Wu, Hua and Wang, Haifeng},
  journal={arXiv preprint arXiv:1907.12412},
  year={2019} 
}