GodSlayer-12B-ABYSS Open-Source Language Model - Reducing Bias for Diverse, Stable and Coherent Responses

Godslayer 12B ABYSS

Developed by redrix

GodSlayer-12B-ABYSS is a 12B-parameter model obtained by merging multiple pre-trained language models through mergekit, designed to reduce positivity bias while maintaining stability and coherence, and to enhance realism and diversified response capabilities.

Large Language Model

Transformers

#Multi-model Fusion #Realistic Response #Role-play Optimization

Downloads 78

Release Time : 4/25/2025

Model Overview

This model focuses on chat, role-play, and creative writing tasks, optimizing response quality and diversity by merging multiple high-quality models.

Model Features

Diversified Responses

By merging multiple models, it reduces positivity bias and provides more realistic and diversified responses.

Stability and Coherence

Optimized merging methods ensure generated text maintains coherence while achieving high stability.

Role-play Optimization

Particularly suitable for role-play tasks, capable of generating expressive and personalized dialogues.

Model Capabilities

Text generation

Chat dialogue

Role-play

Creative writing

Diversified response generation

Use Cases

Entertainment

Role-playing Games

Used to generate NPC dialogues in games, enhancing player immersion.

Generates natural and personalized character dialogues

Creative Writing Assistant

Helps writers generate creative content or story inspiration.

Provides diversified writing suggestions and content

Dialogue Systems

Chatbot

Builds chatbots capable of natural and diversified conversations.

Generates more authentic and engaging dialogue responses

🚀 GodSlayer-12B-ABYSS

This is a merge of pre-trained language models created using mergekit. The model aims to maintain stability and coherence, while counteracting positivity-bias and enhancing realism and diverse responses.

🚀 Quick Start

This section briefly introduces the model and its creation method. The GodSlayer-12B-ABYSS model is a merged pre - trained language model, created with the help of the mergekit tool.

✨ Features

Stability and Coherence: Remains fairly stable and coherent during text generation.
Bias Counteraction: Counteracts positivity - bias, providing more balanced responses.
Diverse Responses: Improves the realism and diversity of generated responses.

📚 Documentation

Merge Details

Merge Method

This model was merged using the NuSLERP merge method, with IntervitensInc/Mistral - Nemo - Base - 2407 - chatml as the base model.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configurations were used to produce this model:

# P1:
models:
  - model: PocketDoc/Dans-PersonalityEngine-V1.1.0-12b
    parameters:
      weight:
        - filter: self_attn
          value: 0.2
        - filter: mlp
          value: 0.2
        - value: 0.2
      density: 0.6
  - model: ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
    parameters:
      weight:
        - filter: self_attn
          value: 0.15
        - filter: mlp
          value: 0.15
        - value: 0.15
      density: 0.55
  - model: HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407
    parameters:
      weight:
        - filter: self_attn
          value: 0.1
        - filter: mlp
          value: 0.1
        - value: 0.1
      density: 0.5
  - model: LatitudeGames/Wayfarer-12B
    parameters:
      weight:
        - filter: self_attn
          value: 0.25
        - filter: mlp
          value: 0.25
        - value: 0.25
      density: 0.65
base_model: TheDrummer/UnslopNemo-12B-v4
merge_method: della_linear
dtype: bfloat16
chat_template: "chatml"
tokenizer_source: union
parameters:
  normalize: true
  int8_mask: true
  epsilon: 0.1
  lambda: 1

# P2:
models:
  - model: rAIfle/Questionable-MN-bf16
    parameters:
      weight:
        - filter: self_attn
          value: 0.2
        - filter: mlp
          value: 0.2
        - value: 0.2
      density: 0.6
  - model: DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS
    parameters:
      weight:
        - filter: self_attn
          value: 0.3
        - filter: mlp
          value: 0.3
        - value: 0.3
      density: 0.7
  - model: allura-org/MN-12b-RP-Ink
    parameters:
      weight:
        - filter: self_attn
          value: 0.35
        - filter: mlp
          value: 0.35
        - value: 0.35
      density: 0.75
  - model: romaingrx/red-teamer-mistral-nemo
    parameters:
      weight:
        - filter: self_attn
          value: 0.25
        - filter: mlp
          value: 0.25
        - value: 0.25
      density: 0.65
base_model: TheDrummer/UnslopNemo-12B-v4
merge_method: della_linear
dtype: bfloat16
chat_template: "chatml"
tokenizer_source: union
parameters:
  normalize: true
  int8_mask: true
  epsilon: 0.1
  lambda: 1

# Final:
models:
  - model: P1
    parameters:
      weight: 0.5
  - model: P2
    parameters:
      weight: 0.5
base_model: IntervitensInc/Mistral-Nemo-Base-2407-chatml
merge_method: nuslerp
dtype: bfloat16
chat_template: "chatml"
tokenizer:
  source: union
parameters:
  normalize: true
  int8_mask: true

🔧 Technical Details

The model is merged based on multiple pre - trained language models. The NuSLERP and DELLA - linear merge methods are used to combine different models, aiming to balance the characteristics of each model to achieve the desired stability, bias - counteraction, and response diversity. The configuration parameters in the YAML files control the weights and densities of each model during the merging process, which is crucial for the final performance of the merged model.

📄 Information Table

Property	Details
Base Model	LatitudeGames/Wayfarer-12B, ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2, PocketDoc/Dans-PersonalityEngine-V1.1.0-12b, HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407, TheDrummer/UnslopNemo-12B-v4, romaingrx/red-teamer-mistral-nemo, DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS, rAIfle/Questionable-MN-bf16, allura-org/MN-12b-RP-Ink, IntervitensInc/Mistral-Nemo-Base-2407-chatml
Library Name	transformers
Tags	mergekit, merge, 12b, chat, roleplay, creative - writing, DELLA - linear
Merge Method	NuSLERP, DELLA - linear
Data Type	bfloat16
Chat Template	chatml
Tokenizer Source	union

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご