🚀 Arcee-Blitz (24B)
Arcee-Blitz (24B) is a new Mistral-based 24B model distilled from DeepSeek. It is designed to be both fast and efficient, and can be regarded as a practical “workhorse” model that can handle a variety of tasks without the overhead of larger architectures.
🚀 Quick Start
This section provides a brief introduction to the model and its key features. For more detailed information, please refer to the subsequent sections.
✨ Features
- Fast and Efficient: Arcee-Blitz (24B) is designed to be both fast and efficient, making it a practical “workhorse” model for a variety of tasks.
- Improved World Knowledge: The model shows large improvements in performance on MMLU-Pro compared to the original Mistral-Small-3, indicating a significant increase in world knowledge.
- Data Contamination Checking: The training data and pipeline were carefully examined to avoid contamination, and the model is open to further community validation and testing.
📦 Installation
Quantizations of Arcee-Blitz (24B) are available in different formats:
- GGUF quants: Available here
- AWQ quants: Available here
📚 Documentation
Model Details
| Property |
Details |
| Architecture Base |
Mistral-Small-24B-Instruct-2501 |
| Parameter Count |
24B |
| Distillation Data |
Merged Virtuoso pipeline with Mistral architecture, hotstarting the training with over 3B tokens of pretraining distillation from DeepSeek-V3 logits |
| Fine-Tuning and Post-Training |
After capturing core logits, additional fine-tuning and distillation steps were performed to enhance overall performance. |
| License |
Apache-2.0 |
Improving World Knowledge
Arcee-Blitz demonstrates significant improvements in performance on MMLU-Pro compared to the original Mistral-Small-3, reflecting a substantial increase in world knowledge.
Data contamination checking
The training data and pipeline were carefully examined to avoid contamination. While confident in the validity of the model's performance gains, the developers remain open to further community validation and testing, which is one of the key reasons for releasing these models as open-source.
Benchmark Comparison
| Benchmark |
mistral‑small‑3 |
arcee‑blitz |
| MixEval |
81.6% |
85.1% |
| GPQADiamond |
42.4% |
43.1% |
| BigCodeBench Complete |
44.4% |
45.5% |
| BigCodeBench Instruct |
34.7% |
35.9% |
| BigCodeBench Complete-hard |
16.2% |
19.6% |
| BigCodeBench Instruct-hard |
15.5% |
15.5% |
| IFEval |
77.44 |
80.60 |
| BBH |
64.46 |
65.00 |
| GPQA |
33.90 |
36.70 |
| MMLU Pro |
44.70 |
60.20 |
| MuSR |
40.90 |
50.00 |
| Math Level 5 |
12.00 |
38.60 |
Limitations
- Context Length: 32k Tokens (may vary depending on the final tokenizer settings and system resources).
- Knowledge Cut-off: Training data may not reflect the latest events or developments beyond June 2024.
Ethical Considerations
- Content Generation Risks: Like any language model, Arcee-Blitz can generate potentially harmful or biased content if prompted in certain ways.
📄 License
Arcee-Blitz (24B) is released under the Apache-2.0 License. You are free to use, modify, and distribute this model in both commercial and non-commercial applications, subject to the terms and conditions of the license.
If you have questions or would like to share your experiences using Arcee-Blitz (24B), please connect with us on social media. We're excited to see what you build—and how this model helps you innovate!
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
