Magnum V2.5 12b Kto

🚀 v2.5 KTO - Text Generation Model

v2.5 KTO is an innovative text - generation model aiming to replicate the high - quality prose of Claude 3 models, especially Sonnet and Opus. It is fine - tuned on top of [anthracite - org/magnum - 12b - v2](https://huggingface.co/anthracite - org/magnum - 12b - v2), offering a unique approach to text generation.


🚀 Quick Start

v2.5 KTO is an experimental release. We are testing a hybrid reinforcement learning strategy of KTO + DPOP. For this, we use rejected data sampled from the original model as "rejected" and data from the original finetuning dataset as "chosen". This experiment was conducted on a limited portion of primarily instruction - following data. In the future, we plan to scale up with a larger KTO dataset for better generalization.

✨ Features

• Prose Quality: Designed to replicate the prose quality of Claude 3 models (Sonnet and Opus).
• Hybrid RL Strategy: Testing a KTO + DPOP hybrid reinforcement learning strategy.
• Multilingual Support: Supports multiple languages including English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, and Japanese.

📄 License

This model is released under the Apache 2.0 license.

💻 Usage Examples

Basic Usage

The model has been Instruct - tuned with the ChatML formatting. A typical input would look like this:

"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""

📚 Documentation

Credits

This model is a result of a team effort by Anthracite. The following datasets were used in the development:

• Stheno dataset (filtered)
• kalomaze/Opus_Instruct_25k
• Nopm/Opus_WritingStruct
• [Gryphe/Sonnet3.5 - SlimOrcaDedupCleaned](https://huggingface.co/datasets/Gryphe/Sonnet3.5 - SlimOrcaDedupCleaned) (A ~16k rows subset)
• kalomaze/Opus_Instruct_3k

Safety

...