Toastypigeon Gemma 3 Starshine 12B GGUF

🚀 Quantized Gemma-3-Starshine-12B

This project provides quantized model files in GGUF format, converted from the original ToastyPigeon/Gemma-3-Starshine-12B. The quantization aims to potentially enhance the model's performance, making it more suitable for various applications.

🚀 Quick Start

This repository contains GGUF format model files converted from ToastyPigeon/Gemma-3-Starshine-12B. The conversion was performed by ArtusDev using llama.cpp, specifically utilizing the imatrix quantization option for potentially improved performance.

✨ Features

🌠G3 Starshine 12B🌠

This was Merge A / A1 in the testing set.

A creative writing model based on a merge of fine - tunes on Gemma 3 12B IT and Gemma 3 12B PT. This is the Story Focused merge. This version works better for storytelling and scenarios, as the prose is more novel - like and it has a tendency to impersonate the user character.

See the Alternate RP Focused version as well.

This is a merge of two G3 models, one trained on instruct and one trained on base:

• allura-org/Gemma-3-Glitter-12B - Itself a merge of a storywriting and RP train (both also by ToastyPigeon), on instruct
• ToastyPigeon/Gemma-3-Confetti-12B - Experimental application of the Glitter data using base instead of instruct, additionally includes some adventure data in the form of SpringDragon.

The result is a lovely blend of Glitter's ability to follow instructions and Confetti's free - spirit prose, effectively 'loosening up' much of the hesitancy that was left in Glitter.

Vision works (as well as any vision works with this model right now) if you pair a GGUF of this with an appropriate mmproj file; the author intends to fix the missing vision tower + make this properly multimodal in the near future.

Thank you to jebcarter for the idea to make this. The author loves how it turned out!

Instruct Format

Uses Gemma2/3 instruct, but has been trained to recognize an optional system role.

⚠️ Important Note

While it won't immediately balk at the system role, results may be better without it.

<start_of_turn>system
{optional system turn with prompt}<end_of_turn>
<start_of_turn>user
{User messages; can also put sysprompt here to use the built-in g3 training}<end_of_turn>
<start_of_turn>model
{model response}<end_of_turn>

Merge Configuration

Yeah, the author actually tried several things and surprisingly this one worked best.

models:
  - model: ToastyPigeon/Gemma-3-Confetti-12B
    parameters:
      weight: 0.5
  - model: allura-org/Gemma-3-Glitter-12B
    parameters:
      weight: 0.5
merge_method: linear
tokenizer_source: allura-org/Gemma-3-Glitter-12B

📄 License

The model is under the gemma license.