Ptt5 V2 Base

🚀 ptt5-v2-base

The ptt5-v2-base is a pre - trained T5 model customized for the Portuguese language, offering high - quality text - to - text generation capabilities.

🚀 Quick Start

The ptt5-v2 models are pre - trained T5 models designed for the Portuguese language. They build on Google's original checkpoints, with sizes ranging from t5 - small to t5 - 3B. These checkpoints were used to train MonoT5 rerankers for Portuguese, which can be found in their HuggingFace collection. For more details about the pre - training process, refer to our paper, ptt5 - v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language.

💻 Usage Examples

Basic Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("unicamp-dl/ptt5-v2-base")
model = T5ForConditionalGeneration.from_pretrained("unicamp-dl/ptt5-v2-base")

📄 License

This project is licensed under the apache - 2.0 license.

📚 Documentation

Dataset Information

📚 Citation

If you use our models, please cite:

@article{piau2024ptt5v2,
      title={ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language}, 
      author={Marcos Piau and Roberto Lotufo and Rodrigo Nogueira},
      year={2024},
      eprint={2406.10806},
      archivePrefix={arXiv},
      primaryClass={id='cs.CL' full_name='Computation and Language' is_active=True alt_name='cmp-lg' in_archive='cs' is_general=False description='Covers natural language processing. Roughly includes material in ACM Subject Class I.2.7. Note that work on artificial languages (programming languages, logics, formal systems) that does not explicitly address natural-language issues broadly construed (natural-language processing, computational linguistics, speech, text retrieval, etc.) is not appropriate for this area.'}
}