đ English to Bantu Languages Translation Model
This project focuses on translating English to various Bantu languages. It provides a transformer - based model with specific pre - processing steps and offers detailed benchmarks.
đ Quick Start
The model is designed for translating English to Bantu languages. To use it, you need to follow the pre - processing steps and use the appropriate language tokens.
⨠Features
- Multi - target Translation: Capable of translating English into multiple Bantu languages, including kin, lin, lug, nya, run, sna, swh, toi_Latn, tso, umb, xho, zul.
- Pre - processing: Utilizes normalization and SentencePiece (spm32k, spm32k) for pre - processing.
- Language Token Requirement: A sentence initial language token in the form of
>>id<< (id = valid target language ID) is required.
đĻ Installation
There is no specific installation command provided in the original document, so this section is skipped.
đģ Usage Examples
There is no code example provided in the original document, so this section is skipped.
đ Documentation
eng - bnt
- Source Group: English
- Target Group: Bantu languages
- OPUS Readme: eng - bnt
- Model: Transformer
- Source Language(s): eng
- Target Language(s): kin lin lug nya run sna swh toi_Latn tso umb xho zul
- Pre - processing: normalization + SentencePiece (spm32k, spm32k)
- Language Token Requirement: A sentence initial language token in the form of
>>id<< (id = valid target language ID) is required.
- Download Original Weights: opus - 2020 - 07 - 26.zip
- Test Set Translations: opus - 2020 - 07 - 26.test.txt
- Test Set Scores: opus - 2020 - 07 - 26.eval.txt
Benchmarks
| Testset |
BLEU |
chr - F |
| Tatoeba - test.eng - kin.eng.kin |
12.5 |
0.519 |
| Tatoeba - test.eng - lin.eng.lin |
1.1 |
0.277 |
| Tatoeba - test.eng - lug.eng.lug |
4.8 |
0.415 |
| Tatoeba - test.eng.multi |
12.1 |
0.449 |
| Tatoeba - test.eng - nya.eng.nya |
22.1 |
0.616 |
| Tatoeba - test.eng - run.eng.run |
13.2 |
0.492 |
| Tatoeba - test.eng - sna.eng.sna |
32.1 |
0.669 |
| Tatoeba - test.eng - swa.eng.swa |
1.7 |
0.180 |
| Tatoeba - test.eng - toi.eng.toi |
10.7 |
0.266 |
| Tatoeba - test.eng - tso.eng.tso |
26.9 |
0.631 |
| Tatoeba - test.eng - umb.eng.umb |
5.2 |
0.295 |
| Tatoeba - test.eng - xho.eng.xho |
22.6 |
0.615 |
| Tatoeba - test.eng - zul.eng.zul |
41.1 |
0.769 |
System Info
| Property |
Details |
| hf_name |
eng - bnt |
| source_languages |
eng |
| target_languages |
bnt |
| opus_readme_url |
https://github.com/Helsinki-NLP/Tatoeba-Challenge/tree/master/models/eng-bnt/README.md |
| original_repo |
Tatoeba - Challenge |
| tags |
['translation'] |
| languages |
['en', 'sn', 'zu', 'rw', 'lg', 'ts', 'ln', 'ny', 'xh', 'rn', 'bnt'] |
| src_constituents |
{'eng'} |
| tgt_constituents |
{'sna', 'zul', 'kin', 'lug', 'tso', 'lin', 'nya', 'xho', 'swh', 'run', 'toi_Latn', 'umb'} |
| src_multilingual |
False |
| tgt_multilingual |
True |
| prepro |
normalization + SentencePiece (spm32k, spm32k) |
| url_model |
https://object.pouta.csc.fi/Tatoeba-MT-models/eng-bnt/opus-2020-07-26.zip |
| url_test_set |
https://object.pouta.csc.fi/Tatoeba-MT-models/eng-bnt/opus-2020-07-26.test.txt |
| src_alpha3 |
eng |
| tgt_alpha3 |
bnt |
| short_pair |
en - bnt |
| chrF2_score |
0.449 |
| bleu |
12.1 |
| brevity_penalty |
1.0 |
| ref_len |
9989.0 |
| src_name |
English |
| tgt_name |
Bantu languages |
| train_date |
2020 - 07 - 26 |
| src_alpha2 |
en |
| tgt_alpha2 |
bnt |
| prefer_old |
False |
| long_pair |
eng - bnt |
| helsinki_git_sha |
480fcbe0ee1bf4774bcbe6226ad9f58e63f6c535 |
| transformers_git_sha |
2207e5d8cb224e954a7cba69fa4ac2309e9ff30b |
| port_machine |
brutasse |
| port_time |
2020 - 08 - 21 - 14:41 |
đ License
The project is licensed under the Apache - 2.0 license.
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
