RLM Hinglish Translator

🚀 Project Hinglish - A Hinglish to English Language Translator

Project Hinglish aims to develop a high - performance language translation model. This model can translate Hinglish, a blend of Hindi and English commonly used in informal communication in India, into standard English. The model is fine - tuned on gemma - 2b using the PEFT (LoRA) method with a rank of 128, aiming to handle the unique syntactical and lexical characteristics of Hinglish.

🚀 Quick Start

Project Hinglish focuses on creating a powerful language translation model. It targets the conversion of Hinglish, a unique language mix, to English. By leveraging the gemma - 2b model and fine - tuning it with PEFT (LoRA), it can better adapt to Hinglish's special features.

✨ Features

• High - performance Translation: Capable of accurately translating Hinglish to standard English.
• Fine - tuned on gemma - 2b: Utilizes the power of gemma - 2b with efficient fine - tuning methods.
• PEFT (LoRA) Technique: Employs Parameter - efficient Fine Tuning with Low - Rank Adaptation for effective model adaptation.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rudrashah/RLM-hinglish-translator")
model = AutoModelForCausalLM.from_pretrained("rudrashah/RLM-hinglish-translator")

template = "Hinglish:\n{hi_en}\n\nEnglish:\n{en}" #THIS IS MOST IMPORTANT, WITHOUT THIS IT WILL GIVE RANDOM OUTPUT
input_text = tokenizer(template.format(hi_en="aapka name kya hai?",en=""),return_tensors="pt")

output = model.generate(**input_text)
print(tokenizer.decode(output[0]))

You can also refer to the Usage Code Notebook on kaggle for more details.

📚 Documentation

Fine - Tune Method

• Fine - Tuning Approach Using PEFT (LoRA): The fine - tuning process uses Parameter - efficient Fine Tuning (PEFT) techniques, specifically LoRA (Low - Rank Adaptation). LoRA efficiently modifies a pre - trained model by introducing low - rank matrices that adapt the model’s attention and feed - forward layers. This method enables significant model adaptation with minimal parameter updates, preserving the original model's strengths while effectively adapting it to the nuances of Hinglish.
• Dataset: The model is trained on the cmu_hinglish_dog dataset, combined with sentences from the author's daily chats with friends and Uber messages.

Example Output

📄 License

This project is licensed under the MIT license.