Qwen2.5 Math 7B 16k Think

🚀 LUFFY

LUFFY uses the base Qwen2.5-Math-7B model, with modifications to enhance performance and functionality.

🚀 Quick Start

The LUFFY project is based on the base Qwen2.5-Math-7B model, as described in the paper Learning to Reason under Off-Policy Guidance. We've made several key modifications:

• Changed the rope_theta from 10000 to 40000 and extended the context window to 16k.
• Modified the chat_template for the system prompt and added <think>.

You can find the project on GitHub: https://github.com/ElliottYan/LUFFY

📄 License

This project is licensed under the MIT license.

📚 Documentation

Citation

If you find our model, data, or evaluation code useful, please kindly cite our paper:

@misc{luffy,
      title={Learning to Reason under Off-Policy Guidance}, 
      author={Jianhao Yan and Yafu Li and Zican Hu and Zhi Wang and Ganqu Cui and Xiaoye Qu and Yu Cheng and Yue Zhang},
      year={2025},
      eprint={2504.14945},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2504.14945}, 
}