Barthez

🚀 BARThez & mBARThez

A French sequence-to-sequence pretrained model based on BART, suitable for generative tasks

🚀 Quick Start

BARThez is a French sequence-to-sequence pretrained model built upon BART. It is pretrained by reconstructing corrupted input sentences, using a 66GB corpus of French raw text. Unlike existing BERT-based French language models like CamemBERT and FlauBERT, BARThez is well-suited for generative tasks (e.g., abstractive summarization) because both its encoder and decoder are pretrained.

In addition to BARThez, which is trained from scratch, we continued the pretraining of a multilingual BART, mBART, enhancing its performance in both discriminative and generative tasks. The French-adapted version is called mBARThez.

📚 Documentation

Model Information

Model Comparison

References

• Paper: BARThez: a Skilled Pretrained French Sequence-to-Sequence Model
• GitHub: https://github.com/moussaKam/BARThez

@article{eddine2020barthez,
  title={BARThez: a Skilled Pretrained French Sequence-to-Sequence Model},
  author={Eddine, Moussa Kamal and Tixier, Antoine J-P and Vazirgiannis, Michalis},
  journal={arXiv preprint arXiv:2010.12321},
  year={2020}
}

📄 License

This project is licensed under the Apache-2.0 license.