Jasmine 350M

🚀 Jasmine-350M

JASMINE: Arabic GPT Models for Few-Shot Learning

This repository is associated with our EMNLP2023 paper JASMINE: Arabic GPT Models for Few-Shot Learning. JASMINE is a collection of powerful Arabic autoregressive Transformer language models. These models have parameter sizes ranging from 300 million to 6.7 billion and are pre - trained on a large and diverse dataset (235 GB of text).

🚀 Quick Start

This section provides an overview of the JASMINE models and their associated research paper. The models are designed for few - shot learning in the Arabic language.

📚 Documentation

BibTex

If you utilize Jasmine models for your scientific publication or find the resources in this repository valuable, please cite our paper as follows (to be updated):

@inproceedings{billah-nagoudi-etal-2023-jasmine,
    title = "{JASMINE}: {A}rabic {GPT} Models for Few-Shot Learning",
    author = "Billah Nagoudi, El Moatez  and
      Abdul-Mageed, Muhammad  and
      Elmadany, AbdelRahim  and
      Inciarte, Alcides  and
      Islam Khondaker, Md Tawkat",
    editor = "Bouamor, Houda  and
      Pino, Juan  and
      Bali, Kalika",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.emnlp-main.1040",
    doi = "10.18653/v1/2023.emnlp-main.1040",
    pages = "16721--16744",
}

Acknowledgments

We sincerely appreciate the support from the Natural Sciences and Engineering Research Council of Canada, the Social Sciences and Humanities Research Council of Canada, the Canadian Foundation for Innovation, ComputeCanada, and UBC ARC - Sockeye. We also thank the Google TensorFlow Research Cloud (TFRC) program for providing us with free TPU access.