Mbart Large 50 En Cbk Finetuned

🚀 English - Zamboanga Chavacano Translation Model

This is a fine - tuned translation model that aims to translate English to Zamboanga Chavacano, leveraging transfer learning from related languages.

🚀 Quick Start

This model is a fine - tuned version of the mbart-large-50-many-to-many-mmt model. It is trained on English to Zamboanga Chavacano sentences from a custom corpus. Zamboanga Chavacano, a Spanish creole, is the most widely spoken Chavacano variety in the Philippines. The model uses transfer learning to take advantage of the relatedness between Chavacano and Spanish, Tagalog, and other Romance languages.

✨ Features

• Translation Capability: Translates English sentences to Zamboanga Chavacano.
• Leverage of Related Languages: Utilizes transfer learning from related languages to improve translation quality.

📦 Installation

Since this model uses the transformers library, you can install it using the following command:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "facebook/mbart-large-50-many-to-many-mmt"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

input_text = "Your English sentence here"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
outputs = model.generate(input_ids)
translation = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(translation)

Advanced Usage

# Training parameters
batch_size = 16
num_train_epochs = 4
learning_rate=2e-4
gradient_accumulation_steps=4
weight_decay=0.01
warmup_steps=500
adam_epsilon=1e-8
max_grad_norm=1.0
fp16=True

# You can use these parameters to fine - tune the model further

📚 Documentation

Dataset Details

The training set contains 5203 sentences, the validation set contains 650 sentences, and the test set contains 651 sentences. Each sentence has 2 - 20 words.

Model Performance

The BLEU score of this model on the validation set is 36.37 and 37.74 on the test set.

Additional Info

📄 License

This model is licensed under the apache - 2.0 license.