Gemma 2 2b Jpn It Translate

🚀 gemma-2-2b-jpn-it-translate

gemma-2-2b-jpn-it-translateは、Googleが公開したgoogle/gemma-2-2b-jpn-itを翻訳タスク用にチューニングしたモデルです。パラメーター数は20億（2B）で、分野によっては1年前の70億（7B）モデルに迫るレベルの翻訳品質を提供します。また、ファイルサイズが約5GBと比較的小さいため、高速な実行が可能です。

🚀 クイックスタート

このモデルは、Googleが公開した日本語専用モデル「gemma-2-2b-jpn-it」をファインチューニングしたものです。具体的には、最初にシステムプロンプト相当の文章（日本語/英語）を与えると、以降はユーザーが入力した文章を翻訳した文章（日本語/英語）を出力するようにトレーニングされています。

✨ 主な機能

• 高速、且つ無限長の文章の翻訳を目指しています。
• apply_chat_templateを使用しているため、ミスを誘発しやすいプロンプトテンプレートの手書き作業が不要です。

📦 インストール

pip install -U transformers

💻 使用例

基本的な使用法

日英翻訳用サンプルコード

import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def get_torch_dtype():
    if torch.cuda.is_available():
        device = torch.device("cuda")
        prop = torch.cuda.get_device_properties(device)
        # Ampere (Compute Capability 8.0 above), for example L4 support bfloat16, but T4 not support.
        if prop.major >= 8:
            return torch.bfloat16
    return torch.float16


model_name = "webbigdata/gemma-2-2b-jpn-it-translate"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=get_torch_dtype(),
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.unk_token

system_prompt = "You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.\n\n"
instruct = """Translate Japanese to English.\nWhen translating, please use the following hints:\n[writing_style: casual]"""

# 文章を区切る関数
def split_sentences(text):
    sentences = []
    last = 0
    # 句点で文を分割
    for match in re.finditer(r'[。！？…]', text):
        end = match.end()
        # 句点の直後に続く改行を含める
        while end < len(text) and text[end] == '\n':
            end += 1
        sentence = text[last:end]
        sentences.append(sentence)
        last = end
    # 残りのテキストを追加
    if last < len(text):
        remaining = text[last:]
        sentences.append(remaining)
    # 各文内の改行を適切に分割
    final_sentences = []
    for s in sentences:
      if '\n' in s:
          parts = s.split('\n')
          for i, part in enumerate(parts):
              if part:
                  # 最後の部分でなければ改行を追加
                  if i < len(parts) - 1:
                      final_sentences.append(part + '\n')
                  else:
                      final_sentences.append(part)
              # 改行自体を保持
              if i < len(parts) - 1:
                  final_sentences.append('\n')
      else:
          final_sentences.append(s)
    return final_sentences

# 翻訳処理を行う関数
def translate_sentence(sentence, previous_context):
      # 過去のコンテキストと新しい文を配列に格納
      if sentence.strip() == '':
          return sentence

      messages = previous_context + [
          {"role": "user", "content": sentence}
      ]

      # apply_chat_templateを使用してプロンプトを生成
      inputs = tokenizer.apply_chat_template(
          messages,
          tokenize=True,
          add_generation_prompt=True,
          return_tensors="pt",
      ).to("cuda")

      translation = ""
      with torch.no_grad():
          generated_ids = model.generate(
              input_ids=inputs,
              num_beams=3, max_new_tokens=1200, do_sample=True, temperature=0.5, top_p=0.3
          )
          full_output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
          translation = full_output.split('\nmodel\n')[-1].strip()
      return translation

from collections import deque

# メイン処理
def main(text):
    sentences = split_sentences(text)
    translated_sentences = []

    # Initialize context with system prompt
    context = deque([
        {"role": "user", "content": system_prompt + instruct},
        {"role": "assistant", "content": "OK"}
    ], maxlen=6)  # Maximum 10 elements (5 user, 5 assistant)

    for i, sentence in enumerate(sentences):
        # For the first sentence, use the full context including system prompt
        if i == 0:
            translation_context = list(context)
        else:
            # For subsequent sentences, exclude the system prompt
            translation_context = list(context)[2:]

        translated_sentence = translate_sentence(sentence, translation_context)
        translated_sentences.append(translated_sentence)

        # Add new interactions to the context
        if sentence.strip() != '':
            context.append({"role": "user", "content": sentence})
        else:
            context.append({"role": "user", "content": sentence})

        if translated_sentence.strip() != '':
            context.append({"role": "assistant", "content": translated_sentence})
        else:
            context.append({"role": "assistant", "content": translated_sentence})

    return translated_sentences


text = """こんにちは。私は田中です。今日はとても良い天気ですね。朝ごはんはパンとコーヒーを食べました。そのあとに散歩に行きました。公園にはたくさんの人がいました。子供たちは遊んでいました。
犬を連れている人もいました。私はベンチに座って本を読みました。風がとても気持ちよかったです。その後、友達とカフェに行きました。
カフェではコーヒーを飲みながらおしゃべりをしました。友達は最近引っ越したばかりだと言いました。新しい家の写真を見せてくれました。
とてもきれいな家でした。時間が経つのがあっという間でした。夕方になり、私は家に帰りました。夕食にはカレーを作りました。カレーはとても美味しかったです。今日一日、とても楽しかったです。"""
translated = main(text)
print(translated)

英日翻訳用サンプルコード

import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

def get_torch_dtype():
    if torch.cuda.is_available():
        device = torch.device("cuda")
        prop = torch.cuda.get_device_properties(device)
        # Ampere (Compute Capability 8.0 above), for example L4 support bfloat16, but T4 not support.
        if prop.major >= 8:
            return torch.bfloat16
    return torch.float16

model_name = "gemma-2-2b-jpn-it-translate"
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=get_torch_dtype(),
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.unk_token

system_prompt = "You are a highly skilled professional Japanese-English and English-Japanese translator. Translate the given text accurately, taking into account the context and specific instructions provided. Steps may include hints enclosed in square brackets [] with the key and value separated by a colon:. Only when the subject is specified in the Japanese sentence, the subject will be added when translating into English. If no additional instructions or context are provided, use your expertise to consider what the most appropriate context is and provide a natural translation that aligns with that context. When translating, strive to faithfully reflect the meaning and tone of the original text, pay attention to cultural nuances and differences in language usage, and ensure that the translation is grammatically correct and easy to read. After completing the translation, review it once more to check for errors or unnatural expressions. For technical terms and proper nouns, either leave them in the original language or use appropriate translations as necessary. Take a deep breath, calm down, and start translating.\n\n"
instruct = """Translate English to Japanese.\nWhen translating, please use the following hints:\n[writing_style: business]"""

# Function to split English sentences
def split_sentences(text):
    sentences = []
    # Split by newlines, periods, exclamation marks, question marks, or two or more consecutive spaces
    pattern = r'(?:\r?\n|\.|\!|\?|(?:\s{2,}))'
    splits = re.split(pattern, text)

    for split in splits:
        split = split.strip()
        if split:
            sentences.append(split)

    return sentences

# Function to translate a sentence
def translate_sentence(sentence, previous_context):
    if sentence.strip() == '':
        return sentence

    messages = previous_context + [
        {"role": "user", "content": sentence}
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
    ).to("cuda")

    translation = ""
    with torch.no_grad():
        generated_ids = model.generate(
            input_ids=inputs,
            num_beams=3, max_new_tokens=1200, do_sample=True, temperature=0.5, top_p=0.3
        )
        full_output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
        translation = full_output.split('\nmodel\n')[-1].strip()
    return translation

from collections import deque

# Main processing function
def main(text):
    sentences = split_sentences(text)
    translated_sentences = []

    context = deque([
        {"role": "user", "content": system_prompt + instruct},
        {"role": "assistant", "content": "OK"}
    ], maxlen=6)

    for i, sentence in enumerate(sentences):
        if i == 0:
            translation_context = list(context)
        else:
            translation_context = list(context)[2:]

        translated_sentence = translate_sentence(sentence, translation_context)
        translated_sentences.append(translated_sentence)

        if sentence.strip() != '':
            context.append({"role": "user", "content": sentence})
        else:
            context.append({"role": "user", "content": sentence})

        if translated_sentence.strip() != '':
            context.append({"role": "assistant", "content": translated_sentence})
        else:
            context.append({"role": "assistant", "content": translated_sentence})

    return translated_sentences

# Sample English text for translation (business context)
text = """Dear valued clients and partners,

I hope this email finds you well. I am writing to provide you with an important update regarding our company's recent developments and future plans.

Firstly, I am pleased to announce that our Q3 financial results have exceeded expectations, with a 15% increase in revenue compared to the same period last year. This success is largely attributed to the launch of our new product line and the expansion of our services into emerging markets.

In light of this growth, we are planning to implement several strategic initiatives in the coming months:

1. Expansion of our R&D department: We will be investing significantly in research and development to maintain our competitive edge in the market.

2. Sustainability efforts: We are committed to reducing our carbon footprint by 30% over the next five years. This includes transitioning to renewable energy sources and implementing eco-friendly practices across all our operations.

3. Digital transformation: We will be upgrading our IT infrastructure to enhance efficiency and provide better service to our clients.

Additionally, we are excited to announce our upcoming annual conference, which will be held virtually this year due to ongoing global health concerns. The conference will take place on November 15-16, 2024, and will feature keynote speeches from industry leaders, interactive workshops, and networking opportunities.

We value your continued support and partnership. If you have any questions or would like further information about any of these initiatives, please don't hesitate to reach out to your account manager or contact our customer support team.

Thank you for your trust in our company. We look forward to achieving new milestones together.

Best regards,
John Smith
CEO, XYZ Corporation"""

高度な使用法

文単位で翻訳する事を学習しているため、改行を含む長文を一度に渡すと品質が低下します。長文を翻訳する際は文単位で区切る前処理をしてからモデルに与えてください。

一括翻訳用サンプルColabスクリプト

Googleアカウントをお持ちの方は下記のリンク先で「Open In Colab」ボタンを押すと無料で確かめる事ができます。 gemma_2_2b_jpn_it_tranlate_batch_translation_sample.ipynb

📚 ドキュメント

モデル説明

このモデルは、Googleが公開した日本語専用モデル「gemma-2-2b-jpn-it」をファインチューニングしたものです。特徴として、高速、且つ無限長の文章の翻訳ができる事を目指しました。

注意事項

文単位で翻訳する事を学習しているため、改行を含む長文を一度に渡すと品質が低下します。長文を翻訳する際は文単位で区切る前処理をしてからモデルに与えてください。

🔧 技術詳細

• パラメーター数は20億（2B）です。
• ファイルサイズが約5GBと比較的小さいため、高速な実行が可能です。

📄 ライセンス

原文書にライセンス情報が記載されていないため、このセクションは省略されています。

⚠️ 重要提示

文単位で翻訳する事を学習しているため、改行を含む長文を一度に渡すと品質が低下します。長文を翻訳する際は文単位で区切る前処理をしてからモデルに与えてください。

💡 使用建议

GPU付きのパソコンをお持ちの方はサンプルコードを参考に動作させる事ができます。GPU付きのパソコンをお持ちでない方はgguf版モデルをお試しください。