L3-8B-Lunar-Stheno Open-Source Language Model - Fixing Issues, Enhancing Context Awareness and Text Generation Abilities

L3 8B Lunar Stheno

Developed by HiroseKoichi

L3-8B-Lunar-Stheno is a model merged from L3-8B-Lunaris-v1 and L3-8B-Stheno-v3.2, addressing issues of overly long responses and insufficient initiative while enhancing context awareness and text generation capabilities.

Large Language Model

Transformers

#Context Awareness Enhancement #Text Generation Optimization #Role-Play Specialized

Downloads 44

Release Time : 7/2/2024

Model Overview

This model optimizes the performance of L3-8B-Lunaris-v1 and L3-8B-Stheno-v3.2 through the SLERP merging method, resolving issues of verbose responses and lack of initiative while maintaining role-play capabilities.

Model Features

Optimized Response Length

Resolved the issue of overly long responses in L3-8B-Lunaris-v1, generating more concise content.

Enhanced Initiative

Fixed the tendency of L3-8B-Lunaris-v1 to avoid direct actions; the model now prefers executing tasks.

Improved Context Awareness

Excels in both role-play and non-role-play scenarios with significantly enhanced context awareness.

Model Capabilities

Text Generation

Role-Play

Context Awareness

Use Cases

Role-Play

Game NPC Dialogue

Used to generate dialogue for game NPCs, providing a more natural interactive experience.

Generates more concise and contextually appropriate content.

Text Generation

Creative Writing

Assists users in creative writing by generating coherent and imaginative text.

Produces more concise content that aligns with user needs.

🚀 L3-8B-Lunar-Stheno

L3-8B-Lunar-Stheno is a model that addresses the issues of its predecessors. L3-8B-Lunaris-v1 shows significant improvements over L3-8B-Stheno-v3.2 in situational awareness and prose, yet it has some drawbacks. This merge resolves those problems, and the results are quite impressive.

🚀 Quick Start

The description above gives a general idea of the model's background and improvements. To use this model, you can refer to the quantization formats and details below.

✨ Features

Improved Performance: Fixes the issues of long response length, inaction, and reduced performance outside role - play present in previous versions.
Simple Merge: Created by replacing the weights of L3-8B-Stheno-v3.2 with those of L3-8B-Lunaris-v1 using a SLERP merge without blending the models.

📦 Installation

This README does not provide specific installation steps. You can use the quantization formats and related links to obtain the model and install it according to the general methods of the relevant frameworks.

💻 Usage Examples

This README does not contain code examples for using the model.

📚 Documentation

Quantization Formats

GGUF

Static:
- https://huggingface.co/HiroseKoichi/L3-8B-Lunar-Stheno-GGUF
- https://huggingface.co/mradermacher/L3-8B-Lunar-Stheno-GGUF
Imatrix:
- https://huggingface.co/mradermacher/L3-8B-Lunar-Stheno-i1-GGUF

Details

Property	Details
License	llama3
Instruct Format	llama-3
Context Size	8K

Models Used

Merge Config

models:
    - model: Sao10K/L3-8B-Stheno-v3.2
    - model: Sao10K/L3-8B-Lunaris-v1
merge_method: slerp
base_model: Sao10K/L3-8B-Stheno-v3.2
parameters:
  t:
    - filter: self_attn
      value: 0
    - filter: mlp
      value: 1
    - value: 0
dtype: bfloat16

🔧 Technical Details

The model is created by replacing the weights of L3-8B-Stheno-v3.2 with those of L3-8B-Lunaris-v1 using a SLERP merge. This method helps to fix the problems of the previous version, such as long response length, inaction, and reduced performance outside role - play.

📄 License

The model is under the llama3 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご