Arabic Clip Vit Base Patch32

🚀 Arabic CLIP

An adaptation of CLIP for the Arabic language, aiming to improve the understanding and interpretation of visual information in the Arabic context.

🚀 Quick Start

Arabic CLIP is an adaptation of the Contrastive Language - Image Pre - training (CLIP) for the Arabic language. CLIP, developed by OpenAI, learns conceptual concepts from images and relates them with textual descriptions. This work tries to enhance the model's understanding and interpretation of visual information in the Arabic language context.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, FlaxVisionTextDualEncoderModel
model = FlaxVisionTextDualEncoderModel.from_pretrained("LinaAlhuri/Arabic-clip-vit-base-patch32", logit_scale_init_value=1,from_pt=True)
model.save_pretrained("arabic_clip") 

tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-base-arabic", cache_dir=None, use_fast=True)

📚 Documentation

📦 Training Data

The goal was to create a comprehensive Arabic image - text dataset by combining multiple data sources due to the scarcity of Arabic resources. Challenges included limited Arabic data and the quality of translated datasets. The approach involved merging genuine datasets for rich information and using translated datasets to cover diverse domains, scenarios, and objects, balancing their respective pros and cons.

🔧 Technical Details

Performance and Limitations

We have tested the efficacy of Arabic CLIP across different benchmarks tailored for tasks like zero - shot learning, image retrieval, localization, and image search.

• Conceptual Captions
• COCO
• ImageNet
• Unsplash

Zero - shot Learning

Image Retrieval

Conceptual Captions Evaluation

COCO Evaluation

Limitations

• Arabic CLIP struggles to count after 3.
• Limited genuine samples for the Arabic language.
• Various noises and biases might be introduced into Arabic CLIP because no studies have been conducted yet to address this issue in the published Arabic dataset or Arabic language models.

Bias

For gender bias, it is important to note that Arabic uses a two - gender system in which all nouns are classified as masculine or feminine. However, this is not the case for English. Translating the text from English to Arabic may result in information loss or even make it prone to gender bias.