Openr1 Qwen 7B SFT Instruct

🚀 OpenR1-Qwen-7B-SFT-Instruct

This model is a fine - tuned version of Qwen/Qwen2.5 - 7B - Instruct, trained on the open - r1/OpenR1 - Math - 220k dataset using TRL, offering enhanced performance in relevant tasks.

🚀 Quick Start

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="ZMC2019/OpenR1-Qwen-7B-SFT-Instruct", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

✨ Features

This model is a fine - tuned version of Qwen/Qwen2.5-7B-Instruct on the open-r1/OpenR1-Math-220k dataset. It has been trained using TRL.

📦 Installation

The installation of relevant dependencies can be inferred from the framework versions used in the training process. For example, you may need to install the following libraries with corresponding versions:

• TRL: 0.16.0.dev0
• Transformers: 4.49.0
• Pytorch: 2.5.1
• Datasets: 3.3.2
• Tokenizers: 0.21.0

You can use pip to install them, for example:

pip install trl==0.16.0.dev0 transformers==4.49.0 torch==2.5.1 datasets==3.3.2 tokenizers==0.21.0

📚 Documentation

Training procedure

This model was trained with SFT.

Framework versions

📄 License

This model is released under the license license.

📚 Citations

Cite TRL as:

@misc{vonwerra2022trl,
	title        = {{TRL: Transformer Reinforcement Learning}},
	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
	year         = 2020,
	journal      = {GitHub repository},
	publisher    = {GitHub},
	howpublished = {\url{https://github.com/huggingface/trl}}
}