RM R1 Qwen2.5 Instruct 14B

🚀 RM-R1: Reward Modeling as Reasoning

RM-R1 is a training framework for the Reasoning Reward Model (ReasRM). It assesses two candidate answers by first "thinking out loud"—generating rubrics or reasoning traces—and then expressing its preference. Compared with previous scalar or vanilla generative reward models, RM-R1 achieves up to +13.8 % absolute accuracy gains on public reward model benchmarks while offering fully interpretable critiques.

🚀 Quick Start

RM-R1 is a novel approach for training a Reasoning Reward Model. It provides a more interpretable and accurate way to evaluate candidate answers.

✨ Features

• Two - stage training:
  1. Distillation of approximately 8.7 K high - quality reasoning traces (Chain - of - Rubrics).
  2. Reinforcement Learning with Verifiable Rewards (RLVR) on approximately 64 K preference pairs.
• Backbones released: 7 B / 14 B / 32 B Qwen - 2.5 - Instruct variants + DeepSeek - distilled checkpoints.

📚 Documentation

Intended uses

• RLHF / RLAIF: A plug - and - play reward function for policy optimisation.
• Automated evaluation: LLM - as - a - judge for open - domain QA, chat, and reasoning.
• Research: Study process supervision, chain - of - thought verification, or rubric generation.

📄 License

The project is released under the MIT license.