# Multi-task Fine-tuning Adaptation

QARiB is a large-scale pre-trained BERT model for Arabic and its dialects, trained on 420 million tweets and 180 million text sentences.

T5 1.1 is Google's improved text-to-text transfer Transformer model, utilizing GEGLU activation function and optimized architecture, pretrained solely on the C4 dataset in an unsupervised manner