Ptt5 Large T5 Vocab

🚀 Portuguese T5 (aka "PTT5")

PTT5 is a T5 model pretrained on Portuguese data, enhancing performance on Portuguese language tasks.

🚀 Quick Start

PTT5 is a T5 model pretrained in the BrWac corpus, a large collection of web pages in Portuguese. It improves T5's performance on Portuguese sentence similarity and entailment tasks. It's available in three sizes (small, base, and large) and two vocabularies (Google's T5 original and ours, trained on Portuguese Wikipedia).

For further information or requests, please go to PTT5 repository.

✨ Features

• Pretrained on the BrWac corpus, a large Portuguese web - page collection.
• Available in three sizes: small, base, and large.
• Supports two vocabularies: Google's T5 original and a Portuguese - trained one.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

# Tokenizer 
from transformers import T5Tokenizer

# PyTorch (bare model, baremodel + language modeling head)
from transformers import T5Model, T5ForConditionalGeneration

# Tensorflow (bare model, baremodel + language modeling head)
from transformers import TFT5Model, TFT5ForConditionalGeneration

model_name = 'unicamp-dl/ptt5-base-portuguese-vocab'

tokenizer = T5Tokenizer.from_pretrained(model_name)

# PyTorch
model_pt = T5ForConditionalGeneration.from_pretrained(model_name)

# TensorFlow
model_tf = TFT5ForConditionalGeneration.from_pretrained(model_name)

📚 Documentation

Available models

📄 License

This project is under the MIT license.

📚 Citation

If you use PTT5, please cite:

@article{ptt5_2020,
  title={PTT5: Pretraining and validating the T5 model on Brazilian Portuguese data},
  author={Carmo, Diedre and Piau, Marcos and Campiotti, Israel and Nogueira, Rodrigo and Lotufo, Roberto},
  journal={arXiv preprint arXiv:2008.09144},
  year={2020}
}