Gemma 2 2b Jpn It Translate

🚀 gemma-2-2b-jpn-it-translate 翻譯模型

gemma-2-2b-jpn-it-translate 是基於 Google 發佈的 google/gemma-2-2b-jpn-it 模型，針對翻譯任務進行微調的模型。雖然它只有 20 億（2B）個參數，但在某些領域，其翻譯質量可接近一年前 70 億（7B）參數的模型。而且，該模型文件大小相對較小，約為 5GB，因此能夠實現快速執行。

🚀 快速開始

gemma-2-2b-jpn-it-translate 模型在翻譯任務中表現出色，能夠為用戶提供高質量的翻譯服務。以下是使用該模型進行翻譯的基本步驟：

1. 安裝必要的庫：使用 pip install -U transformers 命令安裝所需的庫。
2. 加載模型和分詞器：使用 AutoModelForCausalLM 和 AutoTokenizer 加載模型和分詞器。
3. 設置系統提示和指令：根據翻譯任務的需求，設置系統提示和指令。
4. 進行翻譯：將待翻譯的文本輸入到模型中，獲取翻譯結果。

✨ 主要特性

• 高質量翻譯：儘管參數數量為 20 億（2B），但在某些領域能提供接近一年前 70 億（7B）模型的翻譯質量。
• 快速執行：文件大小約 5GB，相對較小，可實現高速運行。
• 支持無限長文本翻譯：旨在實現對無限長文本的高速翻譯。
• 簡化操作：使用 apply_chat_template 避免手動編寫易出錯的提示模板。

📚 詳細文檔

模型說明

此模型是對 Google 發佈的日語專用模型「gemma-2-2b-jpn-it」進行微調後的產物。其目標是實現高速且能處理無限長文本的翻譯。具體而言，先輸入相當於系統提示的文本（日語/英語）後，模型會對用戶後續輸入的文本進行翻譯並輸出（日語/英語）。此外，由於使用了 apply_chat_template，無需手動編寫容易出錯的提示模板。

需要注意的是，該模型是按句子進行翻譯訓練的，因此一次性輸入包含換行的長文本會導致翻譯質量下降。翻譯長文本時，請先將其按句子進行預處理，再輸入模型。

一括翻譯用樣本 Colab 腳本

擁有 Google 賬號的用戶，點擊以下鏈接中的「Open In Colab」按鈕，即可免費進行測試： gemma_2_2b_jpn_it_tranlate_batch_translation_sample.ipynb

💻 使用示例

日英翻譯用示例代碼

如果您擁有配備 GPU 的計算機，可以參考以下腳本進行操作；若沒有，可嘗試使用 gguf 版模型。請注意，以下示例腳本中對句子進行分割的部分並非完整實現，請根據實際需求進行修改。

import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def get_torch_dtype():
    if torch.cuda.is_available():
        device = torch.device("cuda")
        prop = torch.cuda.get_device_properties(device)
        # Ampere (Compute Capability 8.0 above), for example L4 support bfloat16, but T4 not support.
        if prop.major >= 8:
            return torch.bfloat16
    return torch.float16


model_name = "webbigdata/gemma-2-2b-jpn-it-translate"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=get_torch_dtype(),
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.unk_token

system_prompt = "You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.\n\n"
instruct = """Translate Japanese to English.\nWhen translating, please use the following hints:\n[writing_style: casual]"""

# 文章を區切る関數
def split_sentences(text):
    sentences = []
    last = 0
    # 句點で文を分割
    for match in re.finditer(r'[。！？…]', text):
        end = match.end()
        # 句點の直後に続く改行を含める
        while end < len(text) and text[end] == '\n':
            end += 1
        sentence = text[last:end]
        sentences.append(sentence)
        last = end
    # 殘りのテキストを追加
    if last < len(text):
        remaining = text[last:]
        sentences.append(remaining)
    # 各文內の改行を適切に分割
    final_sentences = []
    for s in sentences:
      if '\n' in s:
          parts = s.split('\n')
          for i, part in enumerate(parts):
              if part:
                  # 最後の部分でなければ改行を追加
                  if i < len(parts) - 1:
                      final_sentences.append(part + '\n')
                  else:
                      final_sentences.append(part)
              # 改行自體を保持
              if i < len(parts) - 1:
                  final_sentences.append('\n')
      else:
          final_sentences.append(s)
    return final_sentences

# 翻訳処理を行う関數
def translate_sentence(sentence, previous_context):
      # 過去のコンテキストと新しい文を配列に格納
      if sentence.strip() == '':
          return sentence

      messages = previous_context + [
          {"role": "user", "content": sentence}
      ]

      # apply_chat_templateを使用してプロンプトを生成
      inputs = tokenizer.apply_chat_template(
          messages,
          tokenize=True,
          add_generation_prompt=True,
          return_tensors="pt",
      ).to("cuda")

      translation = ""
      with torch.no_grad():
          generated_ids = model.generate(
              input_ids=inputs,
              num_beams=3, max_new_tokens=1200, do_sample=True, temperature=0.5, top_p=0.3
          )
          full_output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
          translation = full_output.split('\nmodel\n')[-1].strip()
      return translation

from collections import deque

# メイン処理
def main(text):
    sentences = split_sentences(text)
    translated_sentences = []

    # Initialize context with system prompt
    context = deque([
        {"role": "user", "content": system_prompt + instruct},
        {"role": "assistant", "content": "OK"}
    ], maxlen=6)  # Maximum 10 elements (5 user, 5 assistant)

    for i, sentence in enumerate(sentences):
        # For the first sentence, use the full context including system prompt
        if i == 0:
            translation_context = list(context)
        else:
            # For subsequent sentences, exclude the system prompt
            translation_context = list(context)[2:]

        translated_sentence = translate_sentence(sentence, translation_context)
        translated_sentences.append(translated_sentence)

        # Add new interactions to the context
        if sentence.strip() != '':
            context.append({"role": "user", "content": sentence})
        else:
            context.append({"role": "user", "content": sentence})

        if translated_sentence.strip() != '':
            context.append({"role": "assistant", "content": translated_sentence})
        else:
            context.append({"role": "assistant", "content": translated_sentence})

    return translated_sentences


text = """こんにちは。私は田中です。今日はとても良い天気ですね。朝ごはんはパンとコーヒーを食べました。そのあとに散歩に行きました。公園にはたくさんの人がいました。子供たちは遊んでいました。
犬を連れている人もいました。私はベンチに座って本を読みました。風がとても気持ちよかったです。その後、友達とカフェに行きました。
カフェではコーヒーを飲みながらおしゃべりをしました。友達は最近引っ越したばかりだと言いました。新しい家の寫真を見せてくれました。
とてもきれいな家でした。時間が経つのがあっという間でした。夕方になり、私は家に帰りました。夕食にはカレーを作りました。カレーはとても美味しかったです。今日一日、とても楽しかったです。"""
translated = main(text)
print(translated)

英日翻譯用示例代碼

import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def get_torch_dtype():
    if torch.cuda.is_available():
        device = torch.device("cuda")
        prop = torch.cuda.get_device_properties(device)
        # Ampere (Compute Capability 8.0 above), for example L4 support bfloat16, but T4 not support.
        if prop.major >= 8:
            return torch.bfloat16
    return torch.float16

model_name = "gemma-2-2b-jpn-it-translate"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=get_torch_dtype(),
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.unk_token

system_prompt = "You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.\n\n"
instruct = """Translate English to Japanese.\nWhen translating, please use the following hints:\n[writing_style: business]"""

# Function to split English sentences
def split_sentences(text):
    sentences = []
    # Split by newlines, periods, exclamation marks, question marks, or two or more consecutive spaces
    pattern = r'(?:\r?\n|\.|\!|\?|(?:\s{2,}))'
    splits = re.split(pattern, text)

    for split in splits:
        split = split.strip()
        if split:
            sentences.append(split)

    return sentences

# Function to translate a sentence
def translate_sentence(sentence, previous_context):
    if sentence.strip() == '':
        return sentence

    messages = previous_context + [
        {"role": "user", "content": sentence}
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
    ).to("cuda")

    translation = ""
    with torch.no_grad():
        generated_ids = model.generate(
            input_ids=inputs,
            num_beams=3, max_new_tokens=1200, do_sample=True, temperature=0.5, top_p=0.3
        )
        full_output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
        translation = full_output.split('\nmodel\n')[-1].strip()
    return translation

from collections import deque

# Main processing function
def main(text):
    sentences = split_sentences(text)
    translated_sentences = []

    context = deque([
        {"role": "user", "content": system_prompt + instruct},
        {"role": "assistant", "content": "OK"}
    ], maxlen=6)

    for i, sentence in enumerate(sentences):
        if i == 0:
            translation_context = list(context)
        else:
            translation_context = list(context)[2:]

        translated_sentence = translate_sentence(sentence, translation_context)
        translated_sentences.append(translated_sentence)

        if sentence.strip() != '':
            context.append({"role": "user", "content": sentence})
        else:
            context.append({"role": "user", "content": sentence})

        if translated_sentence.strip() != '':
            context.append({"role": "assistant", "content": translated_sentence})
        else:
            context.append({"role": "assistant", "content": translated_sentence})

    return translated_sentences

# Sample English text for translation (business context)
text = """Dear valued clients and partners,

I hope this email finds you well. I am writing to provide you with an important update regarding our company's recent developments and future plans.

Firstly, I am pleased to announce that our Q3 financial results have exceeded expectations, with a 15% increase in revenue compared to the same period last year. This success is largely attributed to the launch of our new product line and the expansion of our services into emerging markets.

In light of this growth, we are planning to implement several strategic initiatives in the coming months:

1. Expansion of our R&D department: We will be investing significantly in research and development to maintain our competitive edge in the market.

2. Sustainability efforts: We are committed to reducing our carbon footprint by 30% over the next five years. This includes transitioning to renewable energy sources and implementing eco-friendly practices across all our operations.

3. Digital transformation: We will be upgrading our IT infrastructure to enhance efficiency and provide better service to our clients.

Additionally, we are excited to announce our upcoming annual conference, which will be held virtually this year due to ongoing global health concerns. The conference will take place on November 15-16, 2024, and will feature keynote speeches from industry leaders, interactive workshops, and networking opportunities.

We value your continued support and partnership. If you have any questions or would like further information about any of these initiatives, please don't hesitate to reach out to your account manager or contact our customer support team.

Thank you for your trust in our company. We look forward to achieving new milestones together.

Best regards,
John Smith
CEO, XYZ Corporation"""

翻譯結果

['貴社にご愛顧いただき、誠にありがとうございます。', 'このメールがご健在であることを心よりお祈り申し上げます。',
'弊社の最近の進展と今後の計畫について、重要なお知らせをご提供いたします。',
'まず、第3四半期の収益が予想を上回ったことをお知らせいたします。昨年の同時期と比較して、売上高が15％増加しました。',
'この成功は、新製品ラインの発売と、新興市場へのサービスの拡大が大きく貢獻しています。',
'この成長を踏まえ、今後の數ヶ月にわたって、いくつかの戦略的イニシアティブを実施する予定です。',
'1', 'R＆D部門の拡大：市場での競爭力を維持するために、大幅に研究開発に投資する予定です。',
'2', 'サステナビリティの取り組み：次の5年間で、炭素排出量を30%削減することを目指しています。',
'これは、再生可能エネルギー源への移行と、すべての事業活動における環境にやさしい実踐の導入を含むものです。',
'3', 'デジタルトランスフォーメーション: 私たちのITインフラを強化し、効率を向上させ、より良いサービスを提供する',
'さらに、今年は新型コロナウイルス感染症の懸念が続くため、オンラインで開催されますが、毎年恆例の年次カンファレンスをお知らせいたします。',
'カンファレンスは2024年11月15日～16日に開催され、業界のリーダーによるキーノートスピーチ、インタラクティブワークショップ、ネットワークングの機會が盛りだくさんです。',
'引き続きご支援とご協力を賜りますようお願い申し上げます。',
'これらのイニシアチブについてご質問がある場合や、さらに詳しい情報をご希望の場合は、ご擔當マネジャーにご連絡するか、弊社のカスタマーサポートチームにご連絡ください。',
'弊社の信頼を賜り、誠にありがとうございます。',
'共に新たな目標を達成できることを楽しみにしています。',
'ご清栄のこととお慶び申し上げます。',
'ジョン・スミス', 'XYZ株式會社のCEO']

🔧 技術細節

基準測試結果

📄 許可證

致謝

• 開發者：[dahara1@webbigdata]
• 支持語言 (NLP)：[英語、日語]
• 基礎模型 [可選]：gemma-2-2b-jpn-it

BibTeX 引用

@misc{dahara2024imatrix,
  author       = {dahara1@webbigdata},
  title        = {gemma-2-2b-jpn-it-translate: A translation task-specific model based on gemma-2-2b-jpn-it},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/webbigdata/gemma-2-2b-jpn-it-translate/}},
  note         = {Accessed: 2024-10-10},
  abstract     = {This model was developed to verify how much Japanese-English and English-Japanese translation performance can be improved with the 2B model.},
}