đ bert-medium-amharic
This bert-medium-amharic model shares the same architecture as bert-medium. It was pretrained from scratch using the Amharic subsets of the oscar, mc4, and amharic-sentences-corpus datasets, with a total of 290 Million tokens. The tokenizer was also trained from scratch on the same text corpus, having a vocabulary size of 28k.
đ Quick Start
Model Information
| Property |
Details |
| Library Name |
transformers |
| Datasets |
oscar, mc4, rasyosef/amharic-sentences-corpus |
| Language |
am |
| Metrics |
perplexity |
| Pipeline Tag |
fill-mask |
Widget Examples
- Example 1:
- Text: á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° [MASK] á°ááĨá¯ááĸ
- Example 2:
- Text: áŖáááĩ á ááĩáĩ áááŗáĩ á¨á ááŽáŗ áááĢáĩ á¨áĻá [MASK] ááĸ á áĨá
á á¨áá¯ááĸ
- Example 3:
- Text: áŦááĢááĢá á¨áŗá áĨáĩá¨áŗá á á ááĩ ááá á¨á°ááá áĩááģá¸áá áá°ááŗá¸áá á°á¨áĩá á¨áááŊá ááŖ á¨áá°áá°á á¨áá¨áĨ áááĒ áá áĩáááĩ á ááŦáá°ááĩ áááĢá áŠáļ [MASK] áĸá°á¨áá ááŦá áá á¨á°áááá áĨáá
áĩáá´ ááá á áĨá¨á°ááᨠáááĸ
- Example 4:
- Text: á°ááĒááš á ááĩáĩአáĢá¸ááá áĩ á¨áá áĢ áĩáĢ ááĢá¨á [MASK] áĨá á
áááá áĨáá°á á¨á áááŗá á¨ááĢáĩá°áĢáá ááŦáĩ á ááą áááĸ
đģ Usage Examples
Basic Usage
>>> from transformers import pipeline
>>> unmasker = pipeline('fill-mask', model='rasyosef/bert-medium-amharic')
>>> unmasker("á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° [MASK] á°ááĨá¯ááĸ")
[{'score': 0.5135582089424133,
'token': 9345,
'token_str': 'áááĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° áááĩ á°ááĨá¯á áĸ'},
{'score': 0.2923661470413208,
'token': 9617,
'token_str': 'áááŗáĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° áááŗáĩ á°ááĨá¯á áĸ'},
{'score': 0.09527599066495895,
'token': 9913,
'token_str': 'á ááĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° á ááĩ á°ááĨá¯á áĸ'},
{'score': 0.06960058212280273,
'token': 10898,
'token_str': 'á ááŗáĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá° á ááŗáĩ á°ááĨá¯á áĸ'},
{'score': 0.019061630591750145,
'token': 28157,
'token_str': '##áááĩ',
'sequence': 'á¨áááĢá¸á á¨áĸáĩáŽáĩáĢ á¨áᥠáááŊ ááá°áááĩ á°ááĨá¯á áĸ'}]
đ§ Technical Details
This model was finetuned and evaluated on the following Amharic NLP tasks:
Sentiment Classification
- Dataset: amharic-sentiment
- Code: https://github.com/rasyosef/amharic-sentiment-classification
Named Entity Recognition
Finetuned Model Performance
The reported F1 scores are macro averages.
| Model |
Size (# params) |
Perplexity |
Sentiment (F1) |
Named Entity Recognition (F1) |
| bert-medium-amharic |
40.5M |
13.74 |
0.83 |
0.68 |
| bert-small-amharic |
27.8M |
15.96 |
0.83 |
0.68 |
| bert-mini-amharic |
10.7M |
22.42 |
0.81 |
0.64 |
| bert-tiny-amharic |
4.18M |
71.52 |
0.79 |
0.54 |
| xlm-roberta-base |
279M |
|
0.83 |
0.73 |
| am-roberta |
443M |
|
0.82 |
0.69 |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
