Archaeo-32B-KTO Open-Source Model - Free Deployment to Boost Role-Playing and Creative Writing

Archaeo 32B KTO

Developed by Delta-Vector

A fusion model based on Archaeo-32B with reinforcement learning training, optimized for role-playing and creative writing

Large Language Model

Transformers

Other#Role-Playing Optimization #Creative Writing Enhancement #KTO Reinforcement Learning

Downloads 25

Release Time : 5/3/2025

Model Overview

A large language model fine-tuned with the KTO algorithm using the Hamanasu-Magnum and Kunou fusion scheme, particularly suitable for role-playing dialogues and creative writing scenarios

Model Features

Role-Playing Optimization

Specially trained with reinforcement learning for role-playing scenarios, capable of generating dialogues that better align with character settings

Creative Writing Enhancement

Excels in creative writing tasks, capable of generating imaginative text content

KTO Reinforcement Learning

Fine-tuned using the Kahneman-Tversky Optimization algorithm to enhance model output quality

Long-Context Support

Supports context lengths of up to 8192 tokens, ideal for long-form creative writing

Model Capabilities

Role-Playing Dialogue

Creative Writing

Story Generation

Instruction Following

Long-Text Generation

Use Cases

Entertainment

Interactive Role-Playing

Engages in immersive role-playing dialogues with users

Generates natural dialogues that fit character settings

Story Creation Assistance

Assists writers in generating creative content or continuing stories

Produces creative storylines and character developments

Education

Creative Writing Teaching

Serves as a teaching aid for writing, providing creative inspiration

Helps students overcome writer's block and sparks creativity

🚀 Archaeo-32B-KTO: Roleplaying & Creative Writing Model

This is a series of model merges specifically designed for roleplaying and creative writing. It is an RL-trained model based on Archaeo, created by merging Hamanasu - Magnum and Kunou, and trained with Axolotl on 8xH200s.

📋 Metadata

Property	Details
Tags	roleplay, creative-writing, merge, mergekit, kto, rl, finetune
Datasets	Delta-Vector/Tauri-IFeval-Dans-Tulu-KTO, Delta-Vector/Tauri-Opus-Accepted-GPT-Rejected-Opus-Writing-Prompts, Delta-Vector/Tauri-KTO-Instruct-Mix, Delta-Vector/Tauri-Purpura-Arkhaios-CC-KTO
Base Model	Delta-Vector/Archaeo-32B
Pipeline Tag	text-generation
Library Name	transformers

🎨 ASCII Art

                        __~a~_
                        ~~;  ~_
          _                ~  ~_                _
         '_\;__._._._._._._]   ~_._._._._._.__;/_`
         '(/'/'/'/'|'|'|'| (    )|'|'|'|'\'\'\'\)'
         (/ / / /, | | | |(/    \) | | | ,\ \ \ \)
        (/ / / / / | | | ~(/    \) ~ | | \ \ \ \ \)
       (/ / / / /  ~ ~ ~   (/  \)    ~ ~  \ \ \ \ \)
      (/ / / / ~          / (||)|          ~ \ \ \ \)
      ~ / / ~            M  /||\M             ~ \ \ ~
       ~ ~                  /||\                 ~ ~
                           //||\\
                           //||\\
                           //||\\
                           '/||\'        "Archaeopteryx"

🚀 Quick Start

ChatML Formatting

"""<|im_start|>system
system prompt<|im_end|>
<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""

📦 Installation (Axolotl Configuration)

base_model: ./model

plugins:
  - axolotl.integrations.liger.LigerPlugin
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
liger_rope: true
liger_rms_norm: true
liger_layer_norm: true
liger_glu_activation: true
liger_fused_linear_cross_entropy: true
cut_cross_entropy: false

load_in_8bit: false
load_in_4bit: false
strict: false

rl: kto
kto_undesirable_weight: 1.0

datasets:
  - path: Delta-Vector/Tauri-Opus-Accepted-GPT-Rejected-Opus-Writing-Prompts
    split: train
    type: chatml.argilla
  - path: Delta-Vector/Tauri-IFeval-Dans-Tulu-KTO
    split: train
    type: chatml.argilla
  - path: Delta-Vector/Tauri-KTO-Instruct-Mix
    split: train
    type: chatml.argilla
  - path: Delta-Vector/Tauri-Purpura-Arkhaios-CC-KTO
    split: train
    type: chatml.argilla
dataset_prepared_path: last_run_prepared
val_set_size: 0.0
output_dir: ./archaeo-kto-v2
remove_unused_columns: false

#@lora_mlp_kernel: true
#lora_qkv_kernel: true
#lora_o_kernel: true


adapter: lora
lora_model_dir:

sequence_len: 8192
pad_to_sequence_len: false

lora_r: 64
lora_alpha: 32
lora_dropout: 0.0
lora_target_linear: true
lora_fan_in_fan_out:
lora_target_modules:
  - gate_proj
  - down_proj
  - up_proj
  - q_proj
  - v_proj
  - k_proj
  - o_proj

wandb_project: Francois-V2
wandb_entity: 
wandb_watch:
wandb_name: Archaeo-32b-KTO
wandb_log_model:

gradient_accumulation_steps: 4
micro_batch_size: 4
num_epochs: 1
optimizer: paged_ademamix_8bit
lr_scheduler: constant_with_warmup
learning_rate: 5e-6
max_grad_norm: 0.001

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: true

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 100
evals_per_epoch: 4
eval_table_size:
eval_max_new_tokens: 128
saves_per_epoch: 1
debug:
deepspeed: ./deepspeed_configs/zero3_bf16.json
weight_decay: 0.0025
fsdp:
fsdp_config:

💻 Quants

GGUF: https://hf-mirror.com/Delta-Vector/Archaeo-32B-KTO/resolve/main/README.md?download=true
EXL2: Delta-Vector/Archaeo-32B-KTO-exl2

🙏 Credits

Thank you to: Kubernetes-bad, LucyKnada, Kalomaze, Alicat, Intervitens, Samantha Twinkman, Tav, Trappu & The rest of Anthracite.

💡 Support

Support me on Ko-Fi: https://ko-fi.com/deltavector

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご