Orpo Mistral 7B V0.3

🚀 Orpo-Mistral-7B-v0.3

This is an ORPO fine - tuned version of Mistral - 7B - v0.3, which combines traditional supervised fine - tuning and preference alignment into one process, following the ChatML chat template.

🚀 Quick Start

This model is an ORPO fine - tuned version of the mistralai/Mistral-7B-v0.3 on 2.5k subsamples of the mlabonne/orpo-dpo-mix-40k dataset. Thanks to Maxime Labonne for providing this amazing guide on Odds Ratio Policy Optimization (ORPO). ORPO combines the traditional supervised fine - tuning and preference alignment stages into a single process.

This model follows the ChatML chat template!

✨ Features

• ORPO fine - tuned based on Mistral - 7B - v0.3.
• Combines supervised fine - tuning and preference alignment.
• Follows the ChatML chat template.

📦 Installation

The installation process mainly involves installing the transformers library and related dependencies. You can use the following command to install:

pip install transformers torch

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, pipeline

model_id = "MuntasirHossain/Orpo-Mistral-7B-v0.3"
tokenizer = AutoTokenizer.from_pretrained(model_id)

llm = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

def generate(input_text):
  messages = [{"role": "user", "content": input_text}]
  prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
  outputs = llm(prompt, max_new_tokens=512,)
  return outputs[0]["generated_text"][len(prompt):]

generate("Explain quantum tunneling in simple terms.")