L3-Aethora-15B Open-source Large Language Model - Optimize Conversation Interaction, Reduce Rejections for More Compliant Communication

L3 Aethora 15B

Developed by Steelskull

L3-Aethora-15B is a large language model based on the Llama3 architecture, optimized for dialogue interaction through special adjustment methods, suppressing rejection tendencies to focus on producing more compliant and facilitative conversations.

Large Language Model

Transformers

#High-compliance dialogue #Low perplexity optimization #Creative response

Downloads 2,512

Release Time : 6/5/2024

Model Overview

This model adjusts responses via the abilteration method, suppressing rejection capabilities to focus on generating more compliant and facilitative dialogue interactions. It then underwent modified DUS (Deep Upgrade Synthesis) merging through passthrough merge, creating AbL3In-15b, and was trained on the Aether-Lite-V1.2 dataset.

Model Features

Optimized dialogue interaction

Adjusts model responses via the abilteration method, suppressing rejection capabilities to focus on producing more compliant and facilitative dialogue interactions.

Efficient merging method

Uses passthrough merge for modified DUS (Deep Upgrade Synthesis) merging, improving efficiency and reducing perplexity.

High-quality training data

Trained on the Aether-Lite-V1.2 dataset, containing approximately 82,000 high-quality samples balancing creativity and intelligence.

Model Capabilities

Text generation

Dialogue interaction

Creative writing

Intelligent Q&A

Use Cases

Dialogue systems

Intelligent customer service

Used to build more natural and compliant customer service dialogue systems.

Provides smoother, less rejection-prone conversation experiences.

Creative writing

Story generation

Used to generate creative stories or content.

Produces creative and coherent text.

🚀 L3-Aethora-15B

L3-Aethora-15B is a model presented by The Skullery, crafted with specific training methods and datasets to offer efficient and high - quality dialogue interactions.

🚀 Quick Start

This section doesn't provide direct quick - start steps. However, for those interested in using L3 - Aethora - 15B, you can access the model and relevant dataset through the following links:

Model: The model details and related information can be explored further on its Hugging Face page.
Dataset: Aether - Lite - V1.2

✨ Features

Customized Response Adjustment: L3 - Aethora - 15B was crafted through using the abilteration method to adjust model responses. The model's refusal is inhibited, focusing on yielding more compliant and facilitative dialogue interactions.
Efficient Merge Method: It underwent a modified DUS (Depth Up Scale) merge (originally used by @Elinas) by using passthrough merge to create a 15b model, with specific adjustments (zeroing) to 'o_proj' and 'down_proj', enhancing its efficiency and reducing perplexity.
High - Quality Training: Trained for 4 epochs using Rslora & DORA training methods on the Aether - Lite - V1.2 dataset, which contains ~82000 high - quality samples, designed to strike a fine balance between creativity, slop, and intelligence at about a 60/40 split.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

General Information

Creator: Steelskull
Dataset: Aether - Lite - V1.2
Training Setup: Trained on 4 x A100 for 15 hours using RsLora and DORA.

Dataset Summary (Filtered)

Dataset	Processed Count	Removed Count
mrfakename/Pure - Dove - ShareGPT	3707	150
mrfakename/Capybara - ShareGPT	13412	2594
jondurbin/airoboros - 3.2	54517	4192
PJMixers/grimulkan_theory - of - mind - ShareGPT	533	6
grimulkan/PIPPA - augmented - dedup	869	46
grimulkan/LimaRP - augmented	790	14
PJMixers/grimulkan_physical - reasoning - ShareGPT	895	4
MinervaAI/Aesir - Preview	994	6
Doctor - Shotgun/no - robots - sharegpt	9911	89

Deduplication Stats

Starting row count: 85628
Final row count: 81960
Rows removed: 3668

Quants

🔧 Technical Details

Model Crafting: L3 - Aethora - 15B was initially adjusted using the abilteration method to modify model responses, inhibiting refusal and promoting more compliant dialogue. Then, a modified DUS merge was applied, with specific adjustments to 'o_proj' and 'down_proj' to create AbL3In - 15b.
Training: AbL3In - 15b was trained for 4 epochs on the Aether - Lite - V1.2 dataset using Rslora & DORA training methods.

📄 License

The model is under the llama3 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご