Zhtw En

🚀 zhtw-en

This model is designed to translate Traditional Chinese sentences into English, with a focus on understanding Taiwanese-style Traditional Chinese and producing more accurate English translations.

🚀 Quick Start

Prerequisites

Before using this model, ensure you have the transformers library installed. You can install it using the following command:

pip install transformers

Usage Example

from transformers import pipeline

model_checkpoint = "agentlans/zhtw-en"
translator = pipeline("translation", model=model_checkpoint)

# From Chinese Wikipedia's article of the day
translator("《阿奇大戰鐵血戰士》是2015年4至7月黑馬漫畫和阿奇漫畫在美國發行的四期限量連環漫畫圖書，由亞歷克斯·德坎皮創作，費爾南多·魯伊斯繪圖，屬跨公司跨界作品。")[0]['translation_text']

# Output
# Acer's Iron Blood Fighter is a four-year series of comic books published in the United States by Black Horse and Ah Chi comics from April to July of that year. The book was created by Alexander d'Campie and painted by Philnanto Ruiz. It is a cross-firm work.

# Compare with my own gold standard translation:
# "Archie vs. Predator" is a limited four-issue comic book series published by Black Horse and Archie Comics in the United States from April to July 2015. It was created by Alex de Campi and drawn by Fernando Ruiz. It's a crossover work.

✨ Features

• Accurate Translation: Focuses on understanding Taiwanese-style Traditional Chinese and producing more accurate English translations.
• Fine-tuned Model: Fine-tuned on the zetavg/coct-en-zh-tw-translations-twp-300k dataset based on Helsinki-NLP/opus-mt-zh-en.

📦 Installation

The model can be used through the transformers library. Install the library using the following command:

pip install transformers

📚 Documentation

Intended Uses & Limitations

Intended Use Cases

• Translating single sentences from Chinese to English.
• Applications requiring understanding of the Chinese language as spoken in Taiwan.

Limitations

• Designed for single-sentence translation, so it may not perform well on longer texts without pre-processing.
• Sometimes hallucinates or omits information, especially with short or long inputs. Further fine-tuning will address this.

Training and Evaluation Data

This model was trained and evaluated on the Corpus of Contemporary Taiwanese Mandarin (COCT) translations dataset.

• Training Data: 80% of the COCT dataset
• Validation Data: 20% of the COCT dataset

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

• Learning Rate: 5e-05
• Train Batch Size: 8
• Eval Batch Size: 8
• Seed: 42
• Optimizer: adamw_torch with betas=(0.9,0.999) and epsilon=1e-08
• LR Scheduler Type: linear
• Number of Epochs: 3.0

Training Results

Framework Versions

• Transformers 4.48.1
• Pytorch 2.3.0+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0

📄 License

This model is licensed under cc-by-4.0.