Acereason Nemotron 7B

🚀 AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning

We're excited to present AceReason-Nemotron-7B, a math and code reasoning model trained solely via reinforcement learning (RL), commencing from the DeepSeek-R1-Distilled-Qwen-7B. It achieves remarkable results, scoring 69.0% on AIME 2024 (+14.5%), 53.6% on AIME 2025 (+17.4%), 51.8% on LiveCodeBench v5 (+8%), and 44.1% on LiveCodeBench v6 (+7%). Through extensive ablations, we systematically study the RL training process and propose a simple yet effective approach: first conduct RL training on math - only prompts, then perform RL training on code - only prompts. Notably, we discover that math - only RL not only significantly boosts the performance of strong distilled models on math benchmarks but also on code reasoning tasks. Additionally, extended code - only RL further enhances code benchmark performance while causing minimal decline in math results. We find that RL not only elicits the foundational reasoning capabilities acquired during pre - training and supervised fine - tuning (e.g., distillation) but also extends the model's reasoning limits, enabling it to solve previously unsolvable problems.

We share our training recipe and training logs in our technical report.

🚀 Quick Start

AceReason-Nemotron is a powerful math and code reasoning model. You can start using it by following the steps below.

✨ Features

• Trained by RL: Entirely trained through reinforcement learning, starting from the DeepSeek-R1-Distilled-Qwen-7B.
• Impressive Results: Achieves high scores on various math and code benchmarks.
• Effective Training Approach: A simple yet effective two - step RL training approach (math - only then code - only).
• Enhanced Reasoning: Elicits and extends the model's reasoning capabilities.

📦 Installation

The installation mainly involves setting up the necessary Python environment and installing the transformers library.

pip install transformers torch

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = 'nvidia/AceReason-Nemotron-7B'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype="auto", device_map="auto")

prompt = "Jen enters a lottery by picking $4$ distinct numbers from $S=\\{1,2,3,\\cdots,9,10\\}.$ $4$ numbers are randomly chosen from $S.$ She wins a prize if at least two of her numbers were $2$ of the randomly chosen numbers, and wins the grand prize if all four of her numbers were the randomly chosen numbers. The probability of her winning the grand prize given that she won a prize is $\\tfrac{m}{n}$ where $m$ and $n$ are relatively prime positive integers. Find $m+n$."
messages = [{"role": "user", "content": prompt}]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to("cuda")

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=32768,
    temperature=0.6,
    top_p=0.95
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Advanced Usage

When dealing with different types of questions (math or code), you can follow the usage recommendations below.

📚 Documentation

Results

We evaluate our model against competitive reasoning models of comparable size within Qwen2.5 and Llama3.1 model family on AIME 2024, AIME 2025, LiveCodeBench v5 (2024/08/01 - 2025/02/01), and LiveCodeBench v6 (2025/02/01 - 2025/05/01). More evaluation results can be found in our technical report.

Usage Recommendations

⚠️ Important Note

When using the model, please follow the usage recommendations to get the best results.

💡 Usage Tip

1. Don't include a system prompt; instead, place all instructions directly in the user prompt.
2. We recommend using the following instruction for math questions: Please reason step by step, and put your final answer within \boxed{}.
3. We recommend using the following instruction for code questions:

question = "" # code question
starter_code = "" # starter code function header

code_instruction_nostartercode = """Write Python code to solve the problem. Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
code_instruction_hasstartercode = """Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
if starter_code != "":
    question += "\n\n" + "Solve the problem starting with the provided function header.\n\nFunction header:\n" + "```\n" + starter_code + "\n```"
    question += "\n\n" + code_instruction_hasstartercode
else:
    question += "\n\n" + code_instruction_nostartercode

final_prompt = "<｜User｜>" + question + "<｜Assistant｜><think>\n"

5. Our inference engine for evaluation is vLLM==0.7.3 using top - p = 0.95, temperature = 0.6, max_tokens = 32768.
6. We use AceMath scorer for math evaluation and LiveCodeBench official script for code evaluation.

Correspondence to

Yang Chen (yachen@nvidia.com), Zhuolin Yang (zhuoliny@nvidia.com), Zihan Liu (zihanl@nvidia.com), Chankyu Lee (chankyul@nvidia.com), Wei Ping (wping@nvidia.com)

📄 License

Your use of this model is governed by the NVIDIA Open Model License.

🔧 Technical Details

We share our training recipe and training logs in our technical report. Through extensive ablations, we systematically study the RL training process and propose a two - step RL training approach.

📖 Citation

@article{acereason2025,
  title={AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning},
  author={Chen, Yang and Yang, Zhuolin and Liu, Zihan and Lee, Chankyu and Shoeybi, Mohammad and Catanzaro, Bryan and Ping, Wei},
  journal={arXiv preprint},
  year={2025}
}