AI Translator Eng To 9ja

🚀 AI-translator-eng-to-9ja

这个模型是一个拥有4.18亿参数的翻译模型，专为将英语翻译成约鲁巴语、伊博语和豪萨语而构建。它在包含150万条句子（每种语言50万条）的数据集上进行训练，能够为这些语言提供高质量的翻译。构建该模型的初衷是打造一个系统，让使用伊博语、豪萨语和约鲁巴语与大语言模型（LLMs）进行交流变得更加容易。

🚀 快速开始

要使用此模型进行翻译任务，你可以从Hugging Face的transformers库中加载它：

from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

import huggingface_hub
huggingface_hub.login()


# Load the fine-tuned model
model = M2M100ForConditionalGeneration.from_pretrained("HelpMumHQ/AI-translator-eng-to-9ja")
tokenizer = M2M100Tokenizer.from_pretrained("HelpMumHQ/AI-translator-eng-to-9ja")

# translate English to Igbo
eng_text="Healthcare is an important field in virtually every society because it directly affects the well-being and quality of life of individuals. It encompasses a wide range of services and professions, including preventive care, diagnosis, treatment, and management of diseases and conditions."
tokenizer.src_lang = "en"
tokenizer.tgt_lang = "ig"
encoded_eng = tokenizer(eng_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_eng, forced_bos_token_id=tokenizer.get_lang_id("ig"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)



# translate English to yoruba
eng_text="Healthcare is an important field in virtually every society because it directly affects the well-being and quality of life of individuals. It encompasses a wide range of services and professions, including preventive care, diagnosis, treatment, and management of diseases and conditions. Effective healthcare systems aim to improve health outcomes, reduce the incidence of illness, and ensure that individuals have access to necessary medical services."
tokenizer.src_lang = "en"
tokenizer.tgt_lang = "yo"
encoded_eng = tokenizer(eng_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_eng, forced_bos_token_id=tokenizer.get_lang_id("yo"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

# translate English to Hausa
eng_text="Healthcare is an important field in virtually every society because it directly affects the well-being and quality of life of individuals. It encompasses a wide range of services and professions, including preventive care, diagnosis, treatment, and management of diseases and conditions. Effective healthcare systems aim to improve health outcomes, reduce the incidence of illness, and ensure that individuals have access to necessary medical services."
tokenizer.src_lang = "en"
tokenizer.tgt_lang = "ha"
encoded_eng = tokenizer(eng_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_eng, forced_bos_token_id=tokenizer.get_lang_id("ha"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

✨ 主要特性

• 支持的语言：
  • 源语言：英语
  • 目标语言：约鲁巴语、伊博语、豪萨语

📦 安装指南

文档未提及安装相关内容，跳过该章节。

💻 使用示例

基础用法

# 上述使用此模型进行翻译任务的代码示例
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer

import huggingface_hub
huggingface_hub.login()


# Load the fine-tuned model
model = M2M100ForConditionalGeneration.from_pretrained("HelpMumHQ/AI-translator-eng-to-9ja")
tokenizer = M2M100Tokenizer.from_pretrained("HelpMumHQ/AI-translator-eng-to-9ja")

# translate English to Igbo
eng_text="Healthcare is an important field in virtually every society because it directly affects the well-being and quality of life of individuals. It encompasses a wide range of services and professions, including preventive care, diagnosis, treatment, and management of diseases and conditions."
tokenizer.src_lang = "en"
tokenizer.tgt_lang = "ig"
encoded_eng = tokenizer(eng_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_eng, forced_bos_token_id=tokenizer.get_lang_id("ig"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)

# 其他语言翻译代码同理

📚 详细文档

支持的语言代码

• 英语：en
• 约鲁巴语：yo
• 伊博语：ig
• 豪萨语：ha

训练数据集

训练数据集包含150万对翻译数据，这些数据来自开源平行语料库和针对约鲁巴语、伊博语和豪萨语精心策划的数据集的组合。

局限性

• 虽然该模型在英语到约鲁巴语、伊博语和豪萨语的翻译中表现良好，但性能可能会因文本的复杂性和领域而异。
• 对于极长的句子或有歧义的上下文，翻译质量可能会下降。

训练超参数

训练过程中使用了以下超参数：

• 学习率：2e - 05
• 训练批次大小：64
• 评估批次大小：64
• 随机种子：42
• 优化器：Adam（β1 = 0.9，β2 = 0.999，ε = 1e - 08）
• 学习率调度类型：线性
• 训练轮数：3

框架版本

• Transformers 4.44.2
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1

🔧 技术细节

文档未提供足够详细的技术实现细节，跳过该章节。

📄 许可证

该模型采用MIT许可证。