Opus Mt Tc Big Cat Oci Spa En

🚀 opus-mt-tc-big-cat_oci_spa-en

This is a neural machine translation model designed to translate text from Catalan, Occitan, and Spanish (cat+oci+spa) into English (en). It offers a practical solution for multi - language translation needs.

🚀 Quick Start

The opus-mt-tc-big-cat_oci_spa-en model is a part of the OPUS-MT project. This project aims to make neural machine translation models accessible for numerous languages globally. All models are initially trained using the Marian NMT framework, an efficient NMT implementation in pure C++. These models are then converted to pyTorch via the transformers library by huggingface. The training data is sourced from OPUS, and the training pipelines follow the procedures of OPUS-MT-train.

• Publications: OPUS-MT – Building open translation services for the World and The Tatoeba Translation Challenge – Realistic Data Sets for Low Resource and Multilingual MT (Please, cite if you use this model.)

@inproceedings{tiedemann-thottingal-2020-opus,
    title = "{OPUS}-{MT} {--} Building open translation services for the World",
    author = {Tiedemann, J{\"o}rg  and Thottingal, Santhosh},
    booktitle = "Proceedings of the 22nd Annual Conference of the European Association for Machine Translation",
    month = nov,
    year = "2020",
    address = "Lisboa, Portugal",
    publisher = "European Association for Machine Translation",
    url = "https://aclanthology.org/2020.eamt-1.61",
    pages = "479--480",
}

@inproceedings{tiedemann-2020-tatoeba,
    title = "The Tatoeba Translation Challenge {--} Realistic Data Sets for Low Resource and Multilingual {MT}",
    author = {Tiedemann, J{\"o}rg},
    booktitle = "Proceedings of the Fifth Conference on Machine Translation",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2020.wmt-1.139",
    pages = "1174--1182",
}

✨ Features

• Multilingual Support: Capable of translating from Catalan, Occitan, and Spanish to English.
• Open - Source Project: Part of the OPUS - MT initiative, promoting open - access to machine translation technology.
• Efficient Training: Trained with the Marian NMT framework and converted to pyTorch for wider usability.

📦 Installation

The model can be used within the Python environment by installing the necessary libraries. You need to have transformers installed. You can install it using pip:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import MarianMTModel, MarianTokenizer

src_text = [
    "¿Puedo hacerte una pregunta?",
    "Toca algo de música."
]

model_name = "pytorch-models/opus-mt-tc-big-cat_oci_spa-en"
tokenizer = MarianTokenizer.from_pretrained(model_name)
model = MarianMTModel.from_pretrained(model_name)
translated = model.generate(**tokenizer(src_text, return_tensors="pt", padding=True))

for t in translated:
    print( tokenizer.decode(t, skip_special_tokens=True) )

# expected output:
#     Can I ask you a question?
#     He plays some music.

Advanced Usage

You can also use OPUS - MT models with the transformers pipelines:

from transformers import pipeline
pipe = pipeline("translation", model="Helsinki-NLP/opus-mt-tc-big-cat_oci_spa-en")
print(pipe("¿Puedo hacerte una pregunta?"))

# expected output: Can I ask you a question?

📚 Documentation

Model Info

Benchmarks

• Test set translations: opusTCv20210807+bt_transformer-big_2022-03-13.test.txt
• Test set scores: opusTCv20210807+bt_transformer-big_2022-03-13.eval.txt
• Benchmark results: benchmark_results.txt
• Benchmark output: benchmark_translations.zip

🔧 Technical Details

Model Conversion Info

• Transformers version: 4.16.2
• OPUS - MT git hash: 3405783
• Port time: Wed Apr 13 18:30:38 EEST 2022
• Port machine: LM0 - 400 - 22516.local

📄 License

This model is released under the cc - by - 4.0 license.

Acknowledgements

The development of this model is supported by multiple projects and organizations. It is part of the European Language Grid as pilot project 2866. It also receives funding from the FoTran project, funded by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 771113), and the MeMAD project, funded by the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No 780069. We are thankful for the computational resources and IT infrastructure provided by CSC -- IT Center for Science, Finland.