English To Hinglish Fintuned Lamma 3 8b Instruct

🚀 英文到印地式英語語言翻譯器項目

本項目旨在開發一個高性能的語言翻譯模型，能夠將標準英語翻譯成印地式英語（一種在印度非正式交流中常用的印地語和英語的混合語言）。

🚀 快速開始

本項目包含一個由作者精心策劃和製作的數據集。你可以通過以下鏈接購買該數據集：購買鏈接

✨ 主要特性

• 多語言支持：支持英語和印地式英語的翻譯。
• 基於強大模型：基於unsloth/llama-3-8b-Instruct-bnb-4bit模型進行微調。
• 開源項目：採用apache-2.0許可證，可自由使用和修改。

📦 安裝指南

!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"
!pip install --no-deps xformers trl peft accelerate bitsandbytes

💻 使用示例

基礎用法

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "suyash2739/English_to_Hinglish_fintuned_lamma_3_8b_instruct",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

高級用法

def pipe(text):
  prompt = """Translate the input from English to Hinglish to give the response.

### Input:
{}

### Response:
"""
  inputs = tokenizer(
      [
          prompt.format(text),
      ], return_tensors = "pt").to("cuda")

  outputs = model.generate(**inputs, max_new_tokens = 2048, use_cache = True)
  raw_text = tokenizer.batch_decode(outputs)[0]
  return raw_text.split("### Response:\n")[1].split("<|eot_id|>")[0]

text = "This is a fine-tuned Hinglish translation model using Llama 3." # INPUT
print(pipe(text))
## Yeh ek fine-tuned Hinglish translation model hai jo Llama 3 ka istemal karta hai.

📚 詳細文檔

模型對比

• 英文原文

English = """Finance Minister Nirmala Sitharaman said, "There used to be a poverty index...a human development index and all of them continue, but today what is keenly watched is VIX, the volatility index of the markets." Stability of the government is important for markets to be efficient, she stated. PM Narendra Modi's third term will make markets function with stability, she added."""

• GPT 4輸出

Gpt 4o = """ Finance Minister Nirmala Sitharaman ne kaha, "Pehle ek poverty index hota tha...ek human development index hota tha aur yeh sab ab bhi hain, lekin aaj jo sabse zyada dekha ja raha hai, woh hai VIX, jo markets ka volatility index hai." Unhone kaha ki sarkar ki stability markets ke efficient hone ke liye zaroori hai. PM Narendra Modi ka teesra term markets ko stability ke saath function karne mein madad karega, unhone joda."""

• 本微調模型輸出

LLama model = Finance Minister Nirmala Sitharaman ne kaha, "Pehle ek poverty index hota tha... ek human development index hota tha aur sab kuch ab bhi chal raha hai, lekin aaj jo kaafi zyada dekha ja raha hai, woh VIX hai, jo markets ki volatility ka index hai." Unhone kaha ki markets ke liye sarkar ki stability zaroori hai. PM Narendra Modi ke teesre term se markets stability ke saath function karenge, unhone joda.

模型相關信息

模型訓練加速

這個Llama模型使用Unsloth和Huggingface的TRL庫進行訓練，訓練速度提升了2倍。

作者信息

• 領英：https://www.linkedin.com/in/suyash-ag/
• GitHub：https://github.com/Suyash018

支持作者

模型圖片