Lamarck 14B V0.7

🚀 Lamarck 14B v0.7

A generalist merge with emphasis on multi - step reasoning, prose, and multi - language ability.

🚀 Quick Start

This README provides detailed information about Lamarck 14B v0.7, a powerful model with excellent performance in multi - step reasoning, prose, and multi - language tasks.

✨ Features

• High - performance: With no benchmark regressions and mostly gains over the previous release, this version of Lamarck has broken the 41.0 average maximum for 14B parameter models.
• Well - rounded ability: Emphasizes multi - step reasoning, prose, and multi - language ability, striving to be a well - rounded model in the 14B parameter model class.
• Custom toolchain: Produced by a custom toolchain to automate a process of LoRAs and layer - targeting merges.

📚 Documentation

Base Models

Other Information

Model Production Process

Lamarck is produced by a custom toolchain to automate a process of LoRAs and layer - targeting merges:

• Extracted LoRA adapters from special - purpose merges
• Custom base models and model_stocks: Use LoRAs from [huihui - ai/Qwen2.5-14B-Instruct - abliterated - v2](https://huggingface.co/huihui - ai/Qwen2.5-14B-Instruct - abliterated - v2) to minimize IFEVAL loss often seen in model_stock merges.
• Separate branches: Have separate branches for aggressive breadcrumbs and conservative DELLA merges.
• Highly targeted gradients: Apply highly targeted weight/density gradients for every 2 - 4 layers at each stage.
• Finalization: Finalize through SLERP+TIES merges recombining the breadcrumbs and DELLA branches to taste.

Model Ancestry and Influence

Lamarck's performance comes from an ancestry that goes back through careful merges to select finetuning work, upcycled and combined.

• Early layers: Emphasize [arcee - ai/Virtuoso - Small](https://huggingface.co/arcee - ai/Virtuoso - Small), [sthenno - com/miscii - 14b - 1225](https://huggingface.co/sthenno - com/miscii - 14b - 1225), and [VAGOsolutions/SauerkrautLM - v2 - 14b - DPO](https://huggingface.co/VAGOsolutions/SauerkrautLM - v2 - 14b - DPO) for extra BBH.
• Later layers: Add synergistic influence from [deepseek - ai/DeepSeek - R1 - Distill - Qwen - 14B](https://huggingface.co/deepseek - ai/DeepSeek - R1 - Distill - Qwen - 14B), [Krystalan/DRT - o1 - 14B](https://huggingface.co/Krystalan/DRT - o1 - 14B), [EVA - UNIT - 01/EVA - Qwen2.5 - 14B - v0.2](https://huggingface.co/EVA - UNIT - 01/EVA - Qwen2.5 - 14B - v0.2), and [CultriX/Qwen2.5 - 14B - Wernicke](https://huggingface.co/CultriX/Qwen2.5 - 14B - Wernicke).

Prose and Translation Abilities

Its prose and translation abilities are boosted by repeated re - emphasis of [Krystalan/DRT - o1 - 14B](https://huggingface.co/Krystalan/DRT - o1 - 14B) and [underwoods/medius - erebus - magnum - 14b](https://huggingface.co/underwoods/medius - erebus - magnum - 14b). Other models found in [sometimesanotion/Qwenvergence - 14B - v3 - Prose](https://huggingface/sometimesanotion/Qwenvergence - 14B - v3 - Prose) also have an impact on prose quality and a surprising synergy of reasoning.

Acknowledgments

Kudos to @arcee - ai, @deepseek - ai, @Krystalan, @underwoods, @VAGOSolutions, @CultriX, @sthenno - com, and @rombodawg whose models had the most influence. [Vimarckoso v3](https://huggingface.co/sometimesanotion/Qwen2.5 - 14B - Vimarckoso - v3) has the model card which documents its extended lineage.

📄 License

This project is licensed under the apache - 2.0 license.

⚠️ Important Note

With no benchmark regressions, mostly gains over the previous release, this version of Lamarck has broken the 41.0 average maximum for 14B parameter models. Those providing feedback, thank you!

💡 Usage Tip

Lamarck 14B v0.7 has excellent performance in multi - step reasoning, prose, and multi - language tasks. You can use it according to your specific needs.