# Lightweight DPO Optimization

StableLM Zephyr 3B is a 3-billion-parameter instruction-tuned model trained on public datasets, synthetic datasets, and Direct Preference Optimization (DPO), delivering excellent performance.