Tunbert Zied

🚀 Tunbert_zied

Tunbert_zied is a language model designed for the Tunisian dialect. It is based on a similar architecture to the RoBERTa model and was created by Zied Sbabti. The model was trained on over 600,000 phrases written in the Tunisian dialect, enabling it to understand and generate text in this unique dialect.

🚀 Quick Start

Load the Model and Tokenizer

To load the tunbert_zied model and its sub - word tokenizer, do not use the AutoTokenizer.from_pretrained(...) method. Instead, use the BertTokenizer.from_pretrained(...) method. This is because the built - in GPT tokenizer of the RoBERTa model was not used; instead, the BertTokenizer was employed.

Example

import transformers as tr

tokenizer = tr.BertTokenizer.from_pretrained("ziedsb19/tunbert_zied")
model = tr.AutoModelForMaskedLM.from_pretrained("ziedsb19/tunbert_zied")
pipeline = tr.pipeline("fill - mask", model=model, tokenizer=tokenizer)

# Test the model by masking a word in a phrase with [MASK]
pipeline("Ahla winek [MASK] lioum ?")

# Results
"""
[{'sequence': 'ahla winek cv lioum?',
  'score': 0.07968682795763016,
  'token': 869,
  'token_str': 'c v'},
 {'sequence': 'ahla winek enty lioum?',
  'score': 0.06116843968629837,
  'token': 448,
  'token_str': 'e n t y'},
 {'sequence': 'ahla winek ch3amla lioum?',
  'score': 0.057379286736249924,
  'token': 7342,
  'token_str': 'c h 3 a m l a'},
 {'sequence': 'ahla winek cha3malt lioum?',
  'score': 0.028112901374697685,
  'token': 4663,
  'token_str': 'c h a 3 m a l t'},
 {'sequence': 'ahla winek enti lioum?',
  'score': 0.025781650096178055,
  'token': 436,
  'token_str': 'e n t i'}]
"""

✍️ Authors

• zied sbabti - Idea & Initial work