Paxinium

🚀 Language Model Merge Project

This project involves merging pre - trained language models, offering enhanced performance and capabilities through the combination of multiple models.

🚀 Quick Start

This is a merge of pre - trained language models created using mergekit.

📚 Documentation

✨ Features

• Merges multiple pre - trained language models.
• Utilizes the Model Stock merge method.

📦 Installation

The library used for this project is transformers. You can install it using the following command:

pip install transformers

📋 Model Information

🔧 Technical Details

Merge Details

Merge Method

This model was merged using the Model Stock merge method using [Delta - Vector/Francois - PE - V2 - Huali - 12B](https://huggingface.co/Delta - Vector/Francois - PE - V2 - Huali - 12B) as a base.

Models Merged

The following models were included in the merge:

• [Delta - Vector/Rei - V3 - KTO - 12B](https://huggingface.co/Delta - Vector/Rei - V3 - KTO - 12B)
• [yamatazen/EtherealAurora - 12B - v2](https://huggingface.co/yamatazen/EtherealAurora - 12B - v2)
• [redrix/GodSlayer - 12B - ABYSS](https://huggingface.co/redrix/GodSlayer - 12B - ABYSS)
• [yamatazen/BlueLight - 12B](https://huggingface.co/yamatazen/BlueLight - 12B)

Configuration

The following YAML configuration was used to produce this model:

base_model: Delta-Vector/Francois-PE-V2-Huali-12B
models:
  - model: Delta-Vector/Rei-V3-KTO-12B
  - model: redrix/GodSlayer-12B-ABYSS
  - model: yamatazen/BlueLight-12B
  - model: yamatazen/EtherealAurora-12B-v2  
merge_method: model_stock
dtype: bfloat16
parameters:
  normalize: true