Plt5 Large

🚀 plT5 Large

plT5 models are T5-based language models trained on Polish corpora. They are optimized for the original T5 denoising target, offering capabilities in translation, summarization, question answering, and reading comprehension.

🚀 Quick Start

plT5 models are T5-based language models trained on Polish corpora. The models were optimized for the original T5 denoising target.

✨ Features

• Tags: T5, translation, summarization, question answering, reading comprehension
• Trained on multiple Polish corpora including CCNet, NKJP, Wikipedia, Open Subtitles, and Free Readings

📦 Installation

The installation process mainly involves using the transformers library. You can install it via pip if not already installed:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("allegro/plt5-large")
model = AutoModel.from_pretrained("allegro/plt5-large")

📚 Documentation

Corpus

plT5 was trained on six different corpora available for the Polish language:

Tokenizer

The training dataset was tokenized into subwords using a sentencepiece unigram model with a vocabulary size of 50k tokens.

📄 License

CC BY 4.0

📚 Citation

If you use this model, please cite the following paper:

@article{chrabrowa2022evaluation,
  title={Evaluation of Transfer Learning for Polish with a Text-to-Text Model},
  author={Chrabrowa, Aleksandra and Dragan, {\L}ukasz and Grzegorczyk, Karol and Kajtoch, Dariusz and Koszowski, Miko{\l}aj and Mroczkowski, Robert and Rybak, Piotr},
  journal={arXiv preprint arXiv:2205.08808},
  year={2022}
}

👥 Authors

The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences.

You can contact us at: klejbenchmark@allegro.pl