đ Model Merge Project
This project focuses on merging pre - trained language models, aiming to combine the strengths of multiple models to achieve better performance in image - text - to - text tasks.
đ Quick Start
This is a merge of pre - trained language models created using mergekit.
⨠Features
- Model Merging: Combines multiple pre - trained Gemma3 - 12B models.
- Specific Merge Method: Utilizes the DARE TIES merge method.
- Customizable Configuration: Allows users to adjust parameters such as density and weight for each model in the merge.
đĻ Installation
No specific installation steps are provided in the original document, so this section is skipped.
đģ Usage Examples
No code examples are provided in the original document, so this section is skipped.
đ Documentation
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using [soob3123/amoral - gemma3 - 12B - v2](https://huggingface.co/soob3123/amoral - gemma3 - 12B - v2) as a base.
Models Merged
The following models were included in the merge:
- [TheDrummer/Fallen - Gemma3 - 12B - v1](https://huggingface.co/TheDrummer/Fallen - Gemma3 - 12B - v1)
- [ReadyArt/The - Omega - Directive - Gemma3 - 12B - v1.0](https://huggingface.co/ReadyArt/The - Omega - Directive - Gemma3 - 12B - v1.0)
- IlyaGusev/saiga_gemma3_12b
- [huihui - ai/gemma - 3 - 12b - it - abliterated](https://huggingface.co/huihui - ai/gemma - 3 - 12b - it - abliterated)
- [soob3123/amoral - gemma3 - 12B - v2 - qat](https://huggingface.co/soob3123/amoral - gemma3 - 12B - v2 - qat)
Configuration
The following YAML configuration was used to produce this model:
models:
- model: soob3123/amoral-gemma3-12B-v2
- model: IlyaGusev/saiga_gemma3_12b
parameters:
density: 0.5
weight: 0.5
- model: TheDrummer/Fallen-Gemma3-12B-v1
parameters:
density: 0.5
weight: 0.5
- model: soob3123/amoral-gemma3-12B-v2-qat
parameters:
density: 0.5
weight: 0.5
- model: huihui-ai/gemma-3-12b-it-abliterated
parameters:
density: 0.5
weight: 0.5
- model: ReadyArt/The-Omega-Directive-Gemma3-12B-v1.0
parameters:
density: 0.5
weight: 0.5
merge_method: dare_ties
base_model: soob3123/amoral-gemma3-12B-v2
parameters:
normalize: true
dtype: bfloat16
đ§ Technical Details
No specific technical details (more than 50 - word descriptions) are provided in the original document, so this section is skipped.
đ License
The model is under the gemma license.
Information Table
| Property |
Details |
| Base Models |
TheDrummer/Fallen - Gemma3 - 12B - v1, soob3123/amoral - gemma3 - 12B - v2, ReadyArt/The - Omega - Directive - Gemma3 - 12B - v1.0, IlyaGusev/saiga_gemma3_12b, huihui - ai/gemma - 3 - 12b - it - abliterated, soob3123/amoral - gemma3 - 12B - v2 - qat |
| Library Name |
transformers |
| Tags |
mergekit, merge |
| License |
gemma |
| Pipeline Tag |
image - text - to - text |
%20--%3e%3cdefs%3e%3cstyle%3e%20.st0%20{%20fill:%20%23061b40;%20}%20.st1%20{%20fill:%20%23306af1;%20}%20.st2%20{%20fill:%20%235ce5cf;%20}%20%3c/style%3e%3c/defs%3e%3cg%3e%3cpath%20class='st0'%20d='M55,10.5h9v3h-9v9h12V7.5h-12v3ZM64,19.5h-6v-3h6v3Z'/%3e%3cpolygon%20class='st0'%20points='69%2016.5%2078%2016.5%2078%2019.5%2069%2019.5%2069%2022.5%2081%2022.5%2081%2013.5%2072%2013.5%2072%2010.5%2081%2010.5%2081%207.5%2069%207.5%2069%2016.5'/%3e%3cpolygon%20class='st0'%20points='95%2010.5%2095%207.5%2083%207.5%2083%2022.5%2095%2022.5%2095%2019.5%2086%2019.5%2086%2016.5%2095%2016.5%2095%2013.5%2086%2013.5%2086%2010.5%2095%2010.5'/%3e%3cpath%20class='st0'%20d='M40,1.5v21h11.6l1.4-1.4v-7.6h0c0,0-1.4-1.5-1.4-1.5l1.4-1.4V2.9l-1.4-1.4h-11.6ZM50,19.5h-7v-6h7v6ZM50,10.5h-7v-6h7v6Z'/%3e%3c/g%3e%3cpath%20class='st1'%20d='M23.1,24L14.7,4.8l-4.9,11.2h3.8l-1.8,4H3.3L12.1,0H2C.9,0,0,.9,0,2v20c0,1.1.9,2,2,2h21.1Z'/%3e%3cpath%20class='st2'%20d='M34,0h-16.8l10.6,24h6.2c1.1,0,2-.9,2-2V2C36,.9,35.1,0,34,0ZM32.5,20h-4V4h4v16Z'/%3e%3c/svg%3e)
