Saudibert

🚀 SaudiBERT

SaudiBERT is the first pre - trained large language model dedicated to Saudi dialect text, offering high - quality language processing capabilities for Saudi dialect.

✨ Features

• Exclusive Focus: It is the first pre - trained large language model solely focused on Saudi dialect text.
• Large - scale Training: Pretrained on two large - scale corpora, the Saudi Tweets Mega Corpus (STMC) with over 141 million tweets and the Saudi Forum Corpus with over 70 million sentences, totaling 26.3GB of text.
• Resource Availability: The code files and results are available on repo.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

SaudiBERT is the first pre - trained large language model focused exclusively on Saudi dialect text. The model was pretrained on two large - scale corpora: the Saudi Tweets Mega Corpus (STMC), which contains +141 million tweets, and the Saudi Forum Corpus, which includes +70 million sentences collected from various Saudi online forums. The datasets comprise 26.3GB of text. The code files along with the results are available on repo.

📄 License

The model is released under the CC - BY - NC - 4.0 license.

📚 BibTex

If you use SaudiBERT model in your scientific publication, or if you find the resources in this repository useful, please cite our paper as follows (citation details to be updated):

@article{qarah2024saudibert,
  title={SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora},
  author={Qarah, Faisal},
  journal={arXiv preprint arXiv:2405.06239},
  year={2024}
}