Qwen2.5 Math 7B RoPE 300k

🚀 Qwen2.5-Math-7B-RoPE-300k

This model is a variant of Qwen/Qwen2.5-Math-7B, extending the context from 4k to 32k tokens by increasing the RoPE base frequency to 300k.

📚 Documentation

Model Information

Model Features

This model is a variant of Qwen/Qwen2.5-Math-7B. By increasing the RoPE base frequency to 300k, the model's context has been extended from 4k to 32k tokens.

📄 Citation

If you find this model useful in your work, please cite the original source:

@article{yang2024qwen25mathtechnicalreportmathematical,
  title={Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement}, 
  author={An Yang and Beichen Zhang and Binyuan Hui and Bofei Gao and Bowen Yu and Chengpeng Li and Dayiheng Liu and Jianhong Tu and Jingren Zhou and Junyang Lin and Keming Lu and Mingfeng Xue and Runji Lin and Tianyu Liu and Xingzhang Ren and Zhenru Zhang},
  journal={arXiv preprint arXiv:2409.12122},
  year={2024}
}