PARD Llama 3.2 1B

🚀 PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation

PARD is a high - performance speculative decoding method that enables low - cost adaptation of autoregressive draft models into parallel draft models, offering significant speedup and efficiency in LLM inference.

🚀 Quick Start

Please visit PARD repo for more information on how to start using this project.

✨ Features

PARD offers the following key features:

• Low-Cost Training: PARD adapts AR (autoregressive) draft models into parallel draft models with minimal overhead. It achieves an average inference speedup of 1.78× compared to pure AR draft models. By introducing a conditional drop - token strategy, it improves training efficiency by up to 3× while maintaining the same level of accuracy.
• Generalizability: Due to its target - independent design, a single PARD draft model can accelerate an entire family of target models. In contrast, target - dependent approaches like Medusa and EAGLE require retraining or tuning for each new target. Thus, PARD significantly reduces both deployment complexity and adaptation cost.
• High Performance: When integrated into an optimized inference framework called Transformers+, PARD delivers up to a 4.08× speedup, with LLaMA3.1 8B reaching a state - of - the - art 311.5 tokens per second. When integrated into vLLM, PARD delivers up to 3.06× speedup, outperforming other speculative decoding methods in vLLM by 1.51×.

📚 Documentation

Introduction

PARD is a high - performance speculative decoding method that also enables low - cost adaptation of autoregressive draft models into parallel draft models.

Model Weights

📄 License

This project is licensed under the MIT license.

📖 Citation

@article{an2025pard,
  title={PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation},
  author={An, Zihao and Bai, Huajun and Liu, Ziqiong and Li, Dong and Barsoum, Emad},
  journal={arXiv preprint arXiv:2504.18583},
  year={2025}
}