Indic Gemma 2b Finetuned Sft Navarasa 2.0

🚀 Indic-gemma-2b-finetuned-sft-Navarasa-2.0

This project presents a fine - tuned model, Indic - gemma - 2b - finetuned - sft - Navarasa - 2.0, which is based on the google/gemma - 2b model. It has been LoRA finetuned on instruction datasets in 15 Indian languages and English, enhancing its multilingual capabilities for various text - generation tasks.

✨ Features

• Multilingual Support: Fine - tuned on datasets from 15 Indian languages (Telugu, Tamil, Malayalam, etc.) and English, enabling it to handle diverse language instructions.
• Efficient Training: Trained using the unsloth library, with inference code provided for faster results. It can also be used with the HuggingFace library.
• Clear Input Format: Defines a clear input text format for easy interaction with the model.

📦 Installation

To install the necessary dependencies, run the following commands:

!pip install -U xformers --index-url https://download.pytorch.org/whl/cu121
!pip install "unsloth[kaggle-new] @git+https://github.com/unslothai/unsloth.git@nightly"

💻 Usage Examples

Basic Usage

The input text format for interacting with the model is as follows:

### Instruction: {instruction}

### Input: {input}

## Response: {response}

Inference With Unsloth

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = False 
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "Telugu-LLM-Labs/Indic-gemma-2b-finetuned-sft-Navarasa-2.0",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    device_map="auto"
)
FastLanguageModel.for_inference(model) # Enable native 2x faster inference

input_prompt = """
### Instruction:
{}

### Input:
{}

### Response:
{}"""

input_text = input_prompt.format(
        "Tranlsate following sentence to Hindi.", # instruction
        "India is a great country.", # input
        "", # output - leave this blank for generation!
    )

inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)

Inference with HuggingFace

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained(
    "Telugu-LLM-Labs/Indic-gemma-2b-finetuned-sft-Navarasa-2.0",
    load_in_4bit = False,
    token = hf_token
)
model.to("cuda")

tokenizer = AutoTokenizer.from_pretrained("Telugu-LLM-Labs/Indic-gemma-2b-finetuned-sft-Navarasa-2.0")

input_prompt = """
### Instruction:
{}

### Input:
{}

### Response:
{}"""

input_text = input_prompt.format(
        "Tranlsate following sentence to Hindi.", # instruction
        "India is a great country.", # input
        "", # output - leave this blank for generation!
    )

inputs = tokenizer([input_text], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 300, use_cache = True)
response = tokenizer.batch_decode(outputs)[0]

📚 Documentation

• Training Details:
  • Samples: The model is trained on approximately 650K instruction samples.
  • GPU: 1 A100, 80GB
  • Time: 45 Hours
  • Platform: E2E Networks
• Datasets:
  • Hindi: [ravithejads/samvaad - hi - filtered](https://huggingface.co/datasets/ravithejads/samvaad - hi - filtered), HydraIndicLM/hindi_alpaca_dolly_67k(sampled)
  • Telugu: [Telugu - LLM - Labs/telugu_alpaca_yahma_cleaned_filtered_romanized](https://huggingface.co/datasets/Telugu - LLM - Labs/telugu_alpaca_yahma_cleaned_filtered_romanized), [Telugu - LLM - Labs/telugu_teknium_GPTeacher_general_instruct_filtered_romanized](https://huggingface.co/datasets/Telugu - LLM - Labs/telugu_teknium_GPTeacher_general_instruct_filtered_romanized)
  • And many more...

Refer to the blog post for sample examples. Please check our [Code Repository](https://github.com/TeluguLLMLabs/Indic - gemma - 7b - Navarasa) for training and inference scripts.

🔧 Technical Details

The model is a collaborative effort by Ravi Theja and Ramsri Goutham. Feel free to DM either of us if you have any questions.

📄 License

• License: Other
• License Name: gemma - terms - of - use
• License Link: https://ai.google.dev/gemma/terms