Multi-dialect-bert-base-arabic Open-source Model - Free Support for Recognizing Multiple Arabic Dialects

Multi Dialect Bert Base Arabic

Developed by bashar-talafha

A multi-dialect BERT model initialized with Arabic-BERT and trained on 10 million Arabic tweets, supporting identification of various Arabic dialects

Large Language Model Arabic#Arabic dialect identification #Multi-dialect BERT #Social media text processing

Downloads 357

Release Time : 3/2/2022

Model Overview

This model is a BERT model specifically developed for multiple Arabic dialects, particularly suitable for country-level dialect identification tasks. It is initialized with Arabic-BERT weights and trained on unlabeled data from the NADI Arabic dialect identification task.

Model Features

Multi-dialect support

Specifically trained for various Arabic dialects, capable of effectively identifying different regional Arabic dialects

Based on large-scale tweet data

Trained on 10 million unlabeled Arabic tweets, possessing strong language understanding capabilities

Transfer learning application

Initialized with Arabic-BERT weights, fully leveraging the advantages of pre-trained models

Model Capabilities

Arabic text understanding

Dialect identification

Masked language modeling

Text classification

Use Cases

Linguistic research

Arabic dialect analysis

Identifying specific Arabic dialects used in text

Can accurately identify dialects from different Arab countries

Social media analysis

Tweet origin prediction

Predicting the geographical location of the publisher based on tweet content

Determining the user's likely country or region of origin through dialect features

🚀 Multi-dialect-Arabic-BERT

This repository hosts the Multi-dialect Arabic BERT model, developed by Mawdoo3-AI.

Background reference: http://www.qfi.org/wp-content/uploads/2018/02/Qfi_Infographic_Mother-Language_Final.pdf

🚀 Quick Start

The model weights can be loaded using the transformers library by HuggingFace.

Basic Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("bashar-talafha/multi-dialect-bert-base-arabic")
model = AutoModel.from_pretrained("bashar-talafha/multi-dialect-bert-base-arabic")

Advanced Usage

Example using pipeline:

from transformers import pipeline

fill_mask = pipeline(
    "fill-mask",
    model="bashar-talafha/multi-dialect-bert-base-arabic ",
    tokenizer="bashar-talafha/multi-dialect-bert-base-arabic "
)

fill_mask(" سافر الرحالة من مطار [MASK] ")

[{'sequence': '[CLS] سافر الرحالة من مطار الكويت [SEP]', 'score': 0.08296813815832138, 'token': 3226},
 {'sequence': '[CLS] سافر الرحالة من مطار دبي [SEP]', 'score': 0.05123933032155037, 'token': 4747},
 {'sequence': '[CLS] سافر الرحالة من مطار مسقط [SEP]', 'score': 0.046838656067848206, 'token': 13205},
 {'sequence': '[CLS] سافر الرحالة من مطار القاهرة [SEP]', 'score': 0.03234650194644928, 'token': 4003},
 {'sequence': '[CLS] سافر الرحالة من مطار الرياض [SEP]', 'score': 0.02606341242790222, 'token': 2200}]

✨ Features

Instead of training the Multi-dialect Arabic BERT model from scratch, we initialized the weights of the model using Arabic-BERT and trained it on 10M arabic tweets from the unlabled data of The Nuanced Arabic Dialect Identification (NADI) shared task.

📚 Documentation

To cite this work

@misc{talafha2020multidialect,
    title={Multi-Dialect Arabic BERT for Country-Level Dialect Identification},
    author={Bashar Talafha and Mohammad Ali and Muhy Eddin Za'ter and Haitham Seelawi and Ibraheem Tuffaha and Mostafa Samir and Wael Farhan and Hussein T. Al-Natsheh},
    year={2020},
    eprint={2007.05612},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Repository

Please check the original repository for more information.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご