đ English to Niger - Kordofanian Languages Translation Model
This project focuses on translating English to Niger - Kordofanian languages. It provides a transformer - based model with specific pre - processing steps and offers various evaluation metrics on test sets.
đ Quick Start
The model is designed to translate English to Niger - Kordofanian languages. You can download the original weights, test set translations, and test set scores from the provided links.
Download Links
⨠Features
- Language Support: Supports translation from English to multiple Niger - Kordofanian languages including
bam_Latn, ewe, fuc, fuv, ibo, kin, etc.
- Pre - processing: Uses normalization and SentencePiece (spm32k, spm32k) for pre - processing.
- Language Token Requirement: A sentence initial language token in the form of
>>id<< (id = valid target language ID) is required.
đ Documentation
Model Details
- Model Type: Transformer
- Source Language: English (
eng)
- Target Languages:
bam_Latn, ewe, fuc, fuv, ibo, kin, lin, lug, nya, run, sag, sna, swh, toi_Latn, tso, umb, wol, xho, yor, zul
Pre - processing
The model uses normalization and SentencePiece (spm32k, spm32k) for pre - processing.
System Info
| Property |
Details |
| hf_name |
eng - nic |
| source_languages |
eng |
| target_languages |
nic |
| opus_readme_url |
https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-nic/README.md |
| original_repo |
Tatoeba - Challenge |
| tags |
['translation'] |
| languages |
['en', 'sn', 'rw', 'wo', 'ig', 'sg', 'ee', 'zu', 'lg', 'ts', 'ln', 'ny', 'yo', 'rn', 'xh', 'nic'] |
| src_constituents |
{'eng'} |
| tgt_constituents |
{'bam_Latn', 'sna', 'kin', 'wol', 'ibo', 'swh', 'sag', 'ewe', 'zul', 'fuc', 'lug', 'tso', 'lin', 'nya', 'yor', 'run', 'xho', 'fuv', 'toi_Latn', 'umb'} |
| src_multilingual |
False |
| tgt_multilingual |
True |
| prepro |
normalization + SentencePiece (spm32k, spm32k) |
| url_model |
https://object.pouta.csc.fi/Tatoeba-MT-models/eng-nic/opus-2020-07-27.zip |
| url_test_set |
https://object.pouta.csc.fi/Tatoeba-MT-models/eng-nic/opus-2020-07-27.test.txt |
| src_alpha3 |
eng |
| tgt_alpha3 |
nic |
| short_pair |
en - nic |
| chrF2_score |
0.42700000000000005 |
| bleu |
11.1 |
| brevity_penalty |
1.0 |
| ref_len |
10625.0 |
| src_name |
English |
| tgt_name |
Niger - Kordofanian languages |
| train_date |
2020 - 07 - 27 |
| src_alpha2 |
en |
| tgt_alpha2 |
nic |
| prefer_old |
False |
| long_pair |
eng - nic |
| helsinki_git_sha |
480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535 |
| transformers_git_sha |
2207e5d8cb224e954a7cba69fa4ac2309e9ff30b |
| port_machine |
brutasse |
| port_time |
2020 - 08 - 21 - 14:41 |
đ License
This project is licensed under the Apache - 2.0 license.
đ Benchmarks
| Testset |
BLEU |
chr - F |
| Tatoeba - test.eng - bam.eng.bam |
6.2 |
0.029 |
| Tatoeba - test.eng - ewe.eng.ewe |
4.5 |
0.258 |
| Tatoeba - test.eng - ful.eng.ful |
0.5 |
0.073 |
| Tatoeba - test.eng - ibo.eng.ibo |
3.9 |
0.267 |
| Tatoeba - test.eng - kin.eng.kin |
6.4 |
0.475 |
| Tatoeba - test.eng - lin.eng.lin |
1.2 |
0.308 |
| Tatoeba - test.eng - lug.eng.lug |
3.9 |
0.405 |
| Tatoeba - test.eng.multi |
11.1 |
0.427 |
| Tatoeba - test.eng - nya.eng.nya |
14.0 |
0.622 |
| Tatoeba - test.eng - run.eng.run |
13.6 |
0.477 |
| Tatoeba - test.eng - sag.eng.sag |
5.5 |
0.199 |
| Tatoeba - test.eng - sna.eng.sna |
19.6 |
0.557 |
| Tatoeba - test.eng - swa.eng.swa |
1.8 |
0.163 |
| Tatoeba - test.eng - toi.eng.toi |
8.3 |
0.231 |
| Tatoeba - test.eng - tso.eng.tso |
50.0 |
0.789 |
| Tatoeba - test.eng - umb.eng.umb |
7.8 |
0.342 |
| Tatoeba - test.eng - wol.eng.wol |
6.7 |
0.143 |
| Tatoeba - test.eng - xho.eng.xho |
26.4 |
0.620 |
| Tatoeba - test.eng - yor.eng.yor |
15.5 |
0.342 |
| Tatoeba - test.eng - zul.eng.zul |
35.9 |
0.750 |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
