Arabertmo Base V4

🚀 Arabic BERT Model

AraBERTMo is an Arabic pre - trained language model based on Google's BERT architecture, offering multiple variants and achieving good results in fill - mask tasks.

✨ Features

• Based on BERT Architecture: AraBERTMo is an Arabic pre - trained language model built upon Google's BERT architecture. The AraBERTMo_base uses the same BERT - Base config and now comes in 10 new variants.
• Model Availability: All models are available on the HuggingFace model page under the Ebtihal name, and the checkpoints are in PyTorch formats.

📦 Installation

To use this model, you need to install torch or tensorflow and the Huggingface library transformers.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("Ebtihal/AraBertMo_base_V4")
model = AutoModelForMaskedLM.from_pretrained("Ebtihal/AraBertMo_base_V4")

📚 Documentation

Pretraining Corpus

The AraBertMo_base_V4 model was pre - trained on approximately 3 million words from the following source:

• OSCAR - Arabic version "unshuffled_deduplicated_ar".

Training Results

This model achieves the following results in the fill - mask task:

Research Organization

This model was built for master's degree research in the following organization:

• University of kufa.
• Faculty of Computer Science and Mathematics.
• Department of Computer Science