đ roberta-medium-amharic
The roberta-medium-amharic model is designed for Amharic language processing, offering high - performance solutions for tasks like sentiment classification and named entity recognition.
đ Quick Start
This model has the same architecture as [xlm - roberta - base](https://huggingface.co/FacebookAI/xlm - roberta - base) and was pretrained from scratch using the Amharic subsets of the oscar, mc4, and [amharic - sentences - corpus](https://huggingface.co/datasets/rasyosef/amharic - sentences - corpus) datasets, on a total of 290 Million tokens. The tokenizer was trained from scratch on the same text corpus, and had a vocabulary size of 32k.
The model was trained for 15 hours on an A100 40GB GPU.
It achieves the following results on the evaluation set:
Loss: 2.446Perplexity: 11.59
Even though this model has 42 Million parameters it beats the 7x larger 279 Million parameter [xlm - roberta - base](https://huggingface.co/FacebookAI/xlm - roberta - base) multilingual model on Amharic Sentiment Classification and Named Entity Recognition tasks.
đģ Usage Examples
Basic Usage
You can use this model directly with a pipeline for masked language modeling:
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='rasyosef/roberta-medium-amharic')
>>> unmasker("á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° <mask> á°ááĨá¯ááĸ")
[{'score': 0.7755730152130127,
'token': 137,
'token_str': 'áááĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° áááĩ á°ááĨá¯ááĸ'},
{'score': 0.09340856224298477,
'token': 346,
'token_str': 'á ááĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° á ááĩ á°ááĨá¯ááĸ'},
{'score': 0.08586721867322922,
'token': 217,
'token_str': 'áááŗáĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° áááŗáĩ á°ááĨá¯ááĸ'},
{'score': 0.011987944133579731,
'token': 733,
'token_str': 'á ááŗáĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° á ááŗáĩ á°ááĨá¯ááĸ'},
{'score': 0.010042797774076462,
'token': 1392,
'token_str': 'áááą',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° áááą á°ááĨá¯ááĸ'}]
đ Documentation
Finetuning
This model was finetuned and evaluated on the following Amharic NLP tasks:
- Sentiment Classification
- Dataset: [amharic - sentiment](https://huggingface.co/datasets/rasyosef/amharic - sentiment)
- Code: https://github.com/rasyosef/amharic - sentiment - classification
- Named Entity Recognition
- Dataset: [amharic - named - entity - recognition](https://huggingface.co/datasets/rasyosef/amharic - named - entity - recognition)
- Code: https://github.com/rasyosef/amharic - named - entity - recognition
Finetuned Model Performance
The reported F1 scores are macro averages.
| Model |
Size (# params) |
Perplexity |
Sentiment (F1) |
Named Entity Recognition (F1) |
| roberta - base - amharic |
110M |
8.08 |
0.88 |
0.78 |
| roberta - medium - amharic |
42.2M |
11.59 |
0.84 |
0.75 |
| bert - medium - amharic |
40.5M |
13.74 |
0.83 |
0.68 |
| bert - small - amharic |
27.8M |
15.96 |
0.83 |
0.68 |
| bert - mini - amharic |
10.7M |
22.42 |
0.81 |
0.64 |
| bert - tiny - amharic |
4.18M |
71.52 |
0.79 |
0.54 |
| xlm - roberta - base |
279M |
|
0.83 |
0.73 |
| afro - xlmr - base |
278M |
|
0.83 |
0.75 |
| afro - xlmr - large |
560M |
|
0.86 |
0.76 |
| am - roberta |
443M |
|
0.82 |
0.69 |
đĻ Model Information
| Property |
Details |
| Library Name |
transformers |
| Datasets |
oscar, mc4, rasyosef/amharic - sentences - corpus |
| Language |
am |
| Metrics |
perplexity |
| Pipeline Tag |
fill - mask |
đ¨ Widget Examples
- Example 1:
- Text: á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° á°ááĨá¯ááĸ
- Example 2:
- Text: áŖáááĩ á ááĩáĩ áááŗáĩ á¨á ááŽáŗ áááĢáĩ á¨áĻá ááĸ á áĨá
á á¨áá¯ááĸ
- Example 3:
- Text: áŦááĢááĢá á¨áŗá áĨáĩá¨áŗá á á ááĩ ááá á¨á°ááá áĩááģá¸áá áá°ááŗá¸áá á°á¨áĩá á¨áááŊá ááŖ á¨áá°áá°á á¨áá¨áĨ áááĒ áá áĩáááĩ á ááŦáá°ááĩ áááĢá áŠáļ áĸá°á¨áá ááŦá áá á¨á°áááá áĨáá
áĩáá´ ááá á áĨá¨á°ááᨠáááĸ
- Example 4:
- Text: á°ááĒááš á ááĩáĩአáĢá¸ááá áĩ á¨áá áĢ áĩáĢ ááĢá¨á áĨá á
áááá áĨáá°á á¨á áááŗá á¨ááĢáĩá°áĢáá ááŦáĩ á ááą áááĸ
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
