Thinkedit Deepseek Llama3 8b

🚀 ThinkEdit-deepseek-llama3-8b

ThinkEdit is a lightweight weight - editing method that boosts the performance of reasoning - augmented models by mitigating overly short thinking.

🚀 Quick Start

This section provides a high - level overview of the ThinkEdit models and their significance.

✨ Features

• Lightweight Editing: Identifies about 2% of "short reasoning" attention heads and edits only about 0.1% of total parameters.
• Performance Boost: Removes the "short reasoning" direction from the output, enhancing performance, especially on cases with short reasoning traces.

📚 Documentation

Repository Information

• Repository for: ThinkEdit - deepseek - llama3 - 8b
  • (We also release ThinkEdit versions for ThinkEdit - deepseek - qwen - 1.5b and ThinkEdit - deepseek - qwen - 14b.)
• Authors: Chung - En Sun, Ge Yan, Tsui - Wei Weng
• Paper: ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models
• Code: https://github.com/Trustworthy - ML - Lab/ThinkEdit

Introduction

Reasoning - augmented models sometimes fail by generating overly short, abstract chain - of - thought (CoT) reasoning, hurting their accuracy.

ThinkEdit is a lightweight weight - editing method that:

• Identifies ~2% of "short reasoning" attention heads
• Edits only ~0.1% of total parameters
• Removes the "short reasoning" direction from their output
• Boosts performance, especially on cases with short reasoning traces

Full Performance Results

1. Overall Accuracy

2. Accuracy on Short Reasoning Cases (Top 5% / 10% / 20%)

3. Reasoning Lengths (Top 5% / 10% / 20% Shortest Responses)

Usage

The usage of ThinkEdit models is exactly the same as the original deepseek - distilled models.

Citation

@misc{sun2025thinkedit,
      title={ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Models},
      author={Chung-En Sun and Ge Yan and Tsui-Wei Weng},
      year={2025},
      eprint={2503.22048},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2503.22048},
}

📄 License

This project is licensed under the MIT license.