Mt Dspec Legislation En Cy

🚀 mt-dspec-legislation-en-cy

A language translation model specialized in English - Welsh translation for the legislation domain.

🚀 Quick Start

This is a language translation model designed for translating between English and Welsh, specifically tailored to the legislation domain.

✨ Features

• Specialized in the legislation domain for English - Welsh translation.
• Trained using a custom DVC pipeline with Marian NMT.
• Utilizes datasets from multiple sources including UK Government Legislation data, OPUS - cy - en, Cofnod Y Cynulliad, and Cofion Techiaith Cymru.

📦 Installation

Ensure you have the prerequisite python libraries installed:

# The constraint imposed on the transformers version below
# is due to the following issue:
#    https://github.com/huggingface/transformers/issues/26271
pip install sentencepiece "transformers>4.26.1<=4.30.2"

💻 Usage Examples

Basic Usage

import trnasformers
model_id = "techiaith/mt-spec-health-en-cy"
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
model = transformers.AutoModelForSeq2SeqLM.from_pretrained(model_id)
translate = transformers.pipeline("translation", model=model, tokenizer=tokenizer)
translated = translate(
  "The Curriculum and Assessment (Wales) Act 2021 (the Act) "
  "established the Curriculum for Wales and replaced the general "
  "curriculum used up until that point."
)
print(translated["translation_text"])

📚 Documentation

This model was trained using a custom DVC pipeline employing Marian NMT. The datasets were prepared from the following sources:

• UK Government Legislation data
• OPUS - cy - en
• Cofnod Y Cynulliad
• Cofion Techiaith Cymru

The data was split into train, validation, and test sets. The test set containing legislation - specific segments was randomly selected from TMX files originating from the Cofion Techiaith Cymru website, which have been pre - classified as pertaining to the specific domain, and data files scraped from the UK Government Legislation website.

After extracting the test set, the aggregation of the remaining data was then split into 10 training and validation sets and fed into 10 marian training sessions.

🔧 Technical Details

Evaluation scores were produced using the python libraries SacreBLEU and torchmetrics.

Model Information Table

Model Evaluation Results

📄 License

This project is licensed under the Apache - 2.0 license.