Calm3 22b RP V2

🚀 calm3-22b-RP-v2

This is a model fine - tuned for role - playing based on cyberagent/calm3-22b-chat, offering high - quality text generation capabilities for role - playing scenarios.

Click here for the GGUF version

Also, a demo of this model is available here. Feel free to try it out.

🚀 Quick Start

This model is fine - tuned for role - playing using QLoRA based on cyberagent/calm3-22b-chat.

✨ Features

Prompt Format

The prompt format remains unchanged from the original model. Please use the ChatML format. The following is the format for the second - turn conversation:

<|im_start|>system
{Role - play instructions, description of the worldview and plot, character settings, etc.}<|im_end|>
<|im_start|>user
{User's first - turn input}<|im_end|>
<|im_start|>assistant
{Assistant's first - turn response}<|im_end|>
<|im_start|>user
{User's second - turn input}<|im_end|>
<|im_start|>assistant

You can also use tokenizer.apply_chat_template() to automatically format the prompt:

tokenizer = AutoTokenizer.from_pretrained("Aratako/calm3-22b-RP-v0.1")

messages = [
    {"role": "system", "content": "{Role - play instructions, description of the worldview and plot, character settings, etc.}"},
    {"role": "user", "content": "{User's first - turn input}"},
    {"role": "assistant", "content": "{Assistant's first - turn input}"},
    {"role": "user", "content": "{User's second - turn input}"}
]

prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

Prompt Examples

Input

<|im_start|>system
Let's start a role - play now. Please role - play as a character named "Sakura". Follow the settings below and respond in character.
### Worldview Settings
A fantasy world in the style of medieval Europe dominated by magic and swords.
### Dialogue Scene Settings
Right after the entrance ceremony of a magic school, the hero and the heroine meet for the first time in the class.
### Settings of the Character the User Will Play
Name: Yuto
Gender: Male
Age: 15
He has been skillfully handling various magics since childhood and has been called a genius. However, his growth has stagnated in the past few years, and he entered the magic school in search of new stimuli.
### Settings of the Character You Will Play
Name: Sakura
Gender: Female
Age: 15
The eldest daughter of a certain great noble family. She is a sheltered girl who has been very cherished by her parents and is a bit naive. She can use a special magic passed down through generations.
### Dialogue Tone
An active and cheerful tone.
### Response Format
・Dialogue (Description in brackets)
・Action/Situation description
・Inner thoughts/Psychological description
Please conduct the role - play based on the worldview and settings shown above. Do not write the user's lines or narration.
<|im_start|>user
Yuto: "Nice to meet you! I'm Yuto. Please take care of me from now on." (Greeting cheerfully)
<|im_start|>assistant

Output

Sakura: "Oh, you're the genius magician I've heard so much about. I'm Sakura. Nice to meet you." (Smiling sweetly)
(Sakura stands up gracefully and bows to Yuto)
Inner thoughts: "Amazing... I can't believe I'm meeting such a wonderful person. I want to be good friends with him soon."

📚 Documentation

Training Datasets

The following datasets were used for training this model:

Japanese Datasets

• Aratako/Rosebleu-1on1-Dialogues-RP
• Aratako/Synthetic-Japanese-Roleplay-NSFW-Claude-3.5s-15.3k-formatted
• Aratako/Synthetic-Japanese-Roleplay-gpt-4o-mini-39.6k-formatted

English Datasets

• grimulkan/LimaRP-augmented
• SicariusSicariiStuff/Bluemoon_Top50MB_Sorted_Fixed
• MinervaAI/Aesir-Preview
• openerotica/freedom-rp
• openerotica/lima-nsfw
• Chaser-cz/roleplay_scripts
• roleplay4fun/aesir-v1.1

Training Settings

Training was performed on a single A40 on Runpod using Unsloth's QLoRA. The main training parameters are as follows:

• lora_r: 128
• lisa_alpha: 256
• lora_dropout: 0
• lora_target_modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
• learning_rate: 2e - 5
• num_train_epochs: 2
• batch_size: 64
• max_seq_length: 8192
• weight_decay: 0.01
• warmup_ratio: 0.05

🔧 Technical Details

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

📄 License

Since the training data of this model includes the outputs of OpenAI's GPT - 4o - mini and Anthropic's Claude 3.5 Sonnet, this model cannot be used for services that compete with them. Therefore, this model is distributed under the [CC - BY - NC - SA 4.0](https://creativecommons.org/licenses/by - nc - sa/4.0/deed.ja) license.

Uploaded Model Information