Magnum-v2-72B open-source large language model - Replicate the prose quality of Claude 3 and it's free to use!

Magnum V2 72b

Developed by anthracite-org

This model is a large language model fine-tuned based on Qwen-2 72B Instruct, aiming to replicate the prose quality of the Claude 3 series of models and is the seventh version in the series of models.

Large Language Model

Safetensors

Supports Multiple LanguagesOpen Source License:Other #Claude 3-style prose #Multilingual dialogue #72B large parameters

Downloads 302

Release Time : 8/18/2024

Model Overview

This model is committed to replicating the prose quality of the Claude 3 models (especially Sonnet and Opus) and uses the ChatML format for instruction fine-tuning, supporting multiple languages.

Model Features

Claude 3 prose quality replication

Specifically optimized and replicated for the prose quality of the Claude 3 series of models (especially Sonnet and Opus)

Multilingual support

Supports multiple languages such as English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, and Japanese

ChatML format support

Uses the ChatML format for instruction fine-tuning, facilitating conversational interaction

Large-scale training

Uses 8 AMD Instinct™ MI300X accelerators for full-parameter fine-tuning, adopting 16k tokens sample packing

Model Capabilities

Text generation

Multilingual understanding

Conversational interaction

Prose creation

Use Cases

Content creation

High-quality prose generation

Generates high-quality prose content with a Claude 3 style

Approaches the prose quality of the Claude 3 model

Multilingual applications

Multilingual text generation

Supports text generation tasks in 9 languages

🚀 magnum-v2-72b

This is the seventh model in a series aiming to replicate the prose quality of Claude 3 models, fine - tuned on top of Qwen - 2 72B Instruct.

🚀 Quick Start

This is the seventh (Lucky!) in a series of models designed to replicate the prose quality of the Claude 3 models, specifically Sonnet and Opus. This model is fine - tuned on top of Qwen-2 72B Instruct.

image/png

✨ Features

Supports multiple languages including English, French, German, Spanish, Italian, Portuguese, Russian, Chinese, and Japanese.
Designed to replicate the prose quality of Claude 3 models (Sonnet and Opus).
Fine - tuned on top of Qwen - 2 72B Instruct.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

Model has been Instruct tuned with the ChatML formatting. A typical input would look like this:

"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""

📚 Documentation

Credits

This model has been a team effort, and the credits goes to all members of Anthracite.

Training

The training was done for 2 epochs. We used 8x AMD Instinct™ MI300X Accelerators for the full - parameter fine - tuning of the model.

We also trained with a weight decay of 0.01 to help further stabilize the loss trajectory and mitigate catastrophic forgetting, and utilize a peak learning rate of 4e - 6 to prevent the 2nd epoch loss from dropping too significantly (as it is a strong indicator of overfitting). image/png

Sample Packing was done for 16k tokens rather than the 8k tokens used in our previous runs.

Model Information

Property	Details
Model Type	magnum-v2-72b
Base Model	Qwen/Qwen2-72B-Instruct
Datasets	Doctor-Shotgun/C2-Stheno, anthracite-org/kalo-opus-instruct-22k-no-refusal, anthracite-org/nopm_claude_writing_fixed
Pipeline Tag	text-generation

Evaluation Results

Open LLM Leaderboard Evaluation Results Detailed results can be found here

Metric	Value
Avg.	41.15
IFEval (0 - Shot)	75.60
BBH (3 - Shot)	57.85
MATH Lvl 5 (4 - Shot)	31.65
GPQA (0 - shot)	18.12
MuSR (0 - shot)	14.18
MMLU - PRO (5 - shot)	49.51

🔧 Technical Details

No specific technical details beyond what is already covered are provided, so this section is skipped.

📄 License

License Name: tongyi - qianwen
License Link: https://huggingface.co/anthracite-org/magnum-v2-72b/blob/main/LICENSE

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご