Plt5 Small

🚀 plT5 Small

plT5 models are T5-based language models trained on Polish corpora. They were optimized for the original T5 denoising target, offering capabilities in translation, summarization, question answering, and reading comprehension.

🚀 Quick Start

To quickly start using the plT5 Small model, you can use the following code:

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("allegro/plt5-small")
model = AutoModel.from_pretrained("allegro/plt5-small")

✨ Features

• Multifunctional: Suitable for translation, summarization, question answering, and reading comprehension tasks.
• Polish Corpora Training: Trained on various Polish corpora, making it well - adapted to the Polish language.

📦 Installation

The installation process is mainly about using the transformers library. You can install it via pip:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("allegro/plt5-small")
model = AutoModel.from_pretrained("allegro/plt5-small")

📚 Documentation

Corpus

plT5 was trained on six different corpora available for the Polish language:

Tokenizer

The training dataset was tokenized into subwords using a sentencepiece unigram model with a vocabulary size of 50k tokens.

🔧 Technical Details

The model is based on the T5 architecture and is optimized for the original T5 denoising target. The sentencepiece unigram model with a 50k token vocabulary size is used for tokenization, which helps the model better understand and process the Polish language.

📄 License

This model is licensed under CC BY 4.0.

📚 Citation

If you use this model, please cite the following paper:

@article{chrabrowa2022evaluation,
  title={Evaluation of Transfer Learning for Polish with a Text-to-Text Model},
  author={Chrabrowa, Aleksandra and Dragan, {\L}ukasz and Grzegorczyk, Karol and Kajtoch, Dariusz and Koszowski, Miko{\l}aj and Mroczkowski, Robert and Rybak, Piotr},
  journal={arXiv preprint arXiv:2205.08808},
  year={2022}
}

👥 Authors

The model was trained by Machine Learning Research Team at Allegro and Linguistic Engineering Group at Institute of Computer Science, Polish Academy of Sciences.

You can contact us at: klejbenchmark@allegro.pl