Jamet 8B L3 MK.V Blackroot

🚀 JAMETSSS

This is an RP and Storytelling model based on specific merging and training methods, offering unique text - generation capabilities.

🚀 Quick Start

This model is available in different GGUF formats:

• GGUF made available by Backyard AI
• ANOTHER GGUF made available by mradermacher

✨ Features

• Different Base and Methods: It uses a different base model and methods. Unlike model stock merge, this approach avoids issues like repeating sentences or words at 0 temperature, similar to the Halu Blackroot issue. This model may be more similar to the Anjir model.
• Multiple Variations: There are about 9 variations of this model. After testing all of them at Q4_K_M, variation number 7 is released.
• Based on Multiple Models: This model is based on this model, which is based on the UltimateAnjir model. It shares the same creative, cheerful, and positive tendencies and is then merged with Llama 3 Instruct.
• DPO Training: DPO is used to reduce cheerfulness, emojis, and positivity. A QLora is trained with about 1,000 prompts from Alpaca to generate a dataset, with specific selection and filtering processes.
• Applying Loras and Adapters: The Abomination Lora from Blackroot and the Anjir Adapter (64 Rank version with reduced Alpha) are applied to improve formatting while retaining previous Lora influences.
• Merging with Anjrit: The model is merged with the Anjrit model. Although the Anjrit model struggles with longer contexts, its no - refusals storytelling abilities are utilized.

📚 Documentation

More Details

1. Base Model: This model is based on this model, which is derived from the UltimateAnjir model. It has the same creative, cheerful, and positive characteristics and is then combined with Llama 3 Instruct.
2. DPO Process: To address the issue of excessive cheerfulness, emojis, and positivity (based on the Jamet MK.II Feedback regarding positivity), DPO is applied. A QLora is trained using about 1,000 prompts from Alpaca to generate a dataset. Responses with emojis are identified using regex, emojis are removed, and responses are classified into chosen (without emojis) and rejected (with emojis) groups.
3. Applying Loras: The Abomination Lora from Blackroot is applied to the model.
4. Applying Adapter: The Anjir Adapter (64 Rank version with reduced Alpha) is used to improve formatting while maintaining the influence of previous Loras. This is based on the feedback that Anjir has better formatting than the Halu Blackroot.
5. Merging with Anjrit: The model is merged with the Anjrit model. The Anjrit model has limitations in handling longer contexts, but its no - refusals storytelling abilities are valuable. You can find a brief overview of the Anjrit model on the Anjir model page.

Notes

• Responsibility: The author is not responsible for anything related to the use of this model.
• Model Purpose: This is an RP and Storytelling model.
• Feedback: You can write your feedback in the discussion section to help improve the models.
• Temperature Setting: Similar to previous models, higher temperatures may lead to incoherent results. It is recommended to use a temperature around 0.85 - 1.05. Merging the base with Llama 3 Instruct has helped to some extent.

📄 License

This model uses the llama3 license.