30B-Lazarus Open-Source Language Model - Superimpose features to enhance the effect and avoid feature dilution. It's the top practical choice!

30B Lazarus

Developed by CalderaAI

The 30B-Lazarus Model is an experimental language model that applies LoRA and model fusion, aiming to enhance model characteristics through superposition while avoiding feature dilution effects.

Large Language Model

Transformers

Other#Multimodal Chain of Thought #Long Text Generation #Instruction Mix Compatibility

Downloads 136

Release Time : 5/25/2023

Model Overview

This model enhances language generation capabilities by superimposing multiple LoRA components (such as Super Chain of Thought, Story Generation II, GPT4 Alpaca, etc.) on a non-standard HuggingFace format Camel base model. It is particularly suitable for chat and story creation tasks.

Model Features

LoRA Superposition Enhancement

Enhances model characteristics by superimposing multiple LoRA components (e.g., Super Chain of Thought, Story Generation II, GPT4 Alpaca, etc.), avoiding the limitations of a single model.

Multi-Instruction Format Compatibility

Supports both Alpaca and Deer instruction formats, adapting to different usage scenarios.

High Flexibility

Can significantly alter generation results by adjusting preset parameters (e.g., temperature), suitable for diverse text generation needs.

Model Capabilities

Text generation

Chat dialogue

Story creation

Instruction following

Use Cases

Chat and Dialogue

Intelligent Chatbot

Used to build highly interactive chatbots that support complex dialogue scenarios.

Generates natural and fluent dialogues, adapting to various conversation styles.

Content Creation

Story Generation

Used to generate coherent and creative story content.

Produces logical and engaging story segments.

🚀 30B-Lazarus

This project combines multiple language models and LoRAs to create a unique model, aiming to add desired features without diluting the model's effective behavior.

🚀 Quick Start

Instruct and Setup Suggestions

Alpaca instruct is primary, Vicuna instruct format may work.
If using KoboldAI or Text - Generation - WebUI, recommend switching between Godlike and Storywriter presets and adjusting output length + instructions in memory.
Other presets as well as custom settings can yield highly different results, especially Temperature.
If poking it with a stick doesn't work try poking harder.

✨ Features

Composition

[] = applied as LoRA to a composite model | () = combined as composite models

[SuperCOT([gtp4xalpaca(manticorechatpygalpha + vicunaunlocked)]+[StoryV2(kaiokendev - SuperHOT - LoRA - prototype30b - 8192)])]

Model Goals

This model is the result of an experimental use of LoRAs on language models and model merges that are not the base HuggingFace - format LLaMA model they were intended for. The desired outcome is to additively apply desired features without paradoxically watering down a model's effective behavior.

Potential Limitations

LoRAs applied on top of each other may intercompete.

Subjective Results

Very promising. Further experimental tests and objective tests are required.

📚 Documentation

Language Models and LoRAs Used Credits

manticore - 30b - chat - pyg - alpha [Epoch0.4] by openaccess - ai - collective
- https://huggingface.co/openaccess - ai - collective/manticore - 30b - chat - pyg - alpha
SuperCOT - LoRA [30B] by kaiokendev
- https://huggingface.co/kaiokendev/SuperCOT - LoRA
Storytelling - LLaMa - LoRA [30B, Version 2] by GamerUnTouch
- https://huggingface.co/GamerUntouch/Storytelling - LLaMa - LoRAs
SuperHOT Prototype [30b 8k ctx] by kaiokendev
- https://huggingface.co/kaiokendev/SuperHOT - LoRA - prototype
ChanSung's GPT4 - Alpaca - LoRA
- https://huggingface.co/chansung/gpt4 - alpaca - lora - 30b
Neko - Institute - of - Science's Vicuna Unlocked LoRA (Checkpoint 46080)
- https://huggingface.co/Neko - Institute - of - Science/VicUnLocked - 30b - LoRA

Also thanks to Meta for LLaMA. Each model and LoRA was hand picked and considered for what it could contribute to this ensemble. Thanks to each and every one of you for your incredible work developing some of the best things to come out of this community.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご