Starcannon Unleashed 12B V1.0

🚀 Starcannon-Unleashed-12B-v1.0-GGUF

This is a merged pre - trained language model, aiming to combine the best features of different models to provide high - quality responses.

🚀 Quick Start

This model can be used with both ChatML and Mistral formats. You can test it out and choose the one you like best. For recommended settings, refer to the Settings section.

✨ Features

• Combined Strengths: Merged from Starcannon v3 and NemoMix Unleashed, it combines the in - character responses of Starcannon v3 and the long - context stability of NemoMix Unleashed.
• Multiple Quantization Options: Available in GGUF and EXL2 quantizations, providing flexibility for different usage scenarios.

📦 Installation

The model is available in different quantized versions on Hugging Face:

GGUF

• VongolaChouko/Starcannon-Unleashed-12B-v1.0-GGUF
• mradermacher/Starcannon-Unleashed-12B-v1.0-GGUF
• bartowski/Starcannon-Unleashed-12B-v1.0-GGUF

EXL2

• 8bpw
• 6bpw
• 4bpw

💻 Usage Examples

Instruct

Both ChatML and Mistral should work fine. The author personally tested this using ChatML and found the model's responses better in this format. You can test it out and observe which one you like best.

Settings

It is recommended to use these settings: Starcannon-Unleashed-12B-v1.0-ST-Formatting-2024-10-29.json

⚠️ Important Note

Open Silly Tavern and use "Master Import", which can be found under "A" tab — Advanced Formatting. Replace the "INSERT WORLD HERE" placeholders with the world/universe in which your character belongs to. If not applicable, just remove that part.

⚠️ Important Note

Check your User Settings and set "Example Messages Behavior" to "Never include examples", in order to prevent the Examples of Dialogue from getting sent two times in the context. People reported that if not set, this results in <|im_end|> tokens being outputted. Refer to this post for more info.

Temperature 1.15 - 1.25 is good, but lower should also work well, as long as you also tweak the Min P and XTC to ensure the model won't choke. Play around with it to see what suits your taste.

This is a modified version of MarinaraSpaghetti's Mistral - Small - Correct.json, transformed into ChatML. You can find the original version here: MarinaraSpaghetti/SillyTavern-Settings

Tips

💡 Usage Tip

• Examples of Dialogue and First Message are very important. The model will copy the style you wrote in these sections. So for example, if you want short outputs, make Examples of Dialogue and First Message short, and if you want longer outputs, make sure your examples have full paragraphs, composed of several sentences.
• If your Examples of Dialogue and First Message are short/concise but the model still rambles, lower Temperature in small increments, but keep Min P and XTC as is first. Test the result and adjust them to your liking. If it still rambles make XTC Threshold higher.
• Utilize Author's Note In - chat @ Depth 2 as System if you want the instruction to have greater impact on the next response. If you want something exciting and spontaneous, you can try out this note I used when I tested out the model: "Scenario: Spontaneous. {{char}} has full autonomy to do anything they wish and progress the interaction in any way they like."

📚 Documentation

Introduction

The author is very excited to share the result of countless hours of work. This is the first time the author attempted to merge a model and is not an expert in coding. After rigorous testing, the results are satisfactory. The goal was to create a model that combines the best parts of Starcannon v3 and NemoMix Unleashed.

Sample Output

🔧 Technical Details

Merge Details

This is a merge of pre - trained language models created using mergekit.

Merge Method

This model was merged using the della_linear merge method using G:\text - generation - webui\models\MarinaraSpaghetti_NemoMix - Unleashed - 12B as a base.

Models Merged

The following models were included in the merge:

• G:\text - generation - webui\models\Nothingiisreal_MN - 12B - Starcannon - v3

Configuration

The following YAML configuration was used to produce this model:

base_model: G:\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B
dtype: bfloat16
merge_method: della_linear
parameters:
  epsilon: 0.05
  int8_mask: 1.0
  lambda: 1.0
slices:
- sources:
  - layer_range: [0, 40]
    model: G:\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B
    parameters:
      density: 0.65
      weight: 0.4
  - layer_range: [0, 40]
    model: G:\text-generation-webui\models\Nothingiisreal_MN-12B-Starcannon-v3
    parameters:
      density: 0.55
      weight: 0.6

📄 License

This model is licensed under cc - by - nc - 4.0.

Credits

A very huge thank you to MarinaraSpaghetti and Nothing is Real! Their models inspired the author to learn how to create this one.