🚀 wanabi-24B

wanabi-24B is a large Japanese language model fine-tuned specifically for supporting novel writing. It is based on mistralai/Mistral-Small-24B-Base-2501 and continuously trained with Japanese novel-related text data. The model aims to support various processes in novel writing, from idea generation to text generation and contextually appropriate continuation generation.

Current Version

• v0.3 (Released on May 16, 2025)
  • Development Status: The model is still under development and training. Version v0.3 has four times the training volume of v0.1.
  • Distribution Format: Currently, only the GGUF format is provided. There are four types of quantized models: Q6_K, Q5_K_M, Q4_K_M, and IQ4_XS. All models in v0.3 have undergone imatrix quantization using proprietary calibration data.
• v0.2 (Released on May 9, 2025)
  • Development Status: The model is still under development and training. Version v0.2 has 2.5 times more training than v0.1.
• v0.1
  • Development Status: The model is still under development and training. Version v0.1 is the initial release. In the future, we aim to improve performance by releasing version updates at each training checkpoint.
  • Distribution Format: Currently, only the GGUF (Q4_K_M) format is provided. The non - quantized version will be uploaded after all training is completed.
  • Evolution from the Preview Version: Thank you to all users who used and tested the previously released Preview version. Based on your feedback, we have improved the dataset and introduced the following new features and improvements in this version (v0.1).

🚀 Quick Start

It is highly recommended to use this model in conjunction with the dedicated desktop application Project Wannabe. Project Wannabe provides a GUI to fully leverage the capabilities of wanabi-24B and is designed to make the functions described below intuitive to use.

⚠️ Important Note

To use all the features of the currently released wanabi-24B v0.1 (especially the Author's Note, new continuation generation prompts, rating, and dialogue volume specification), you need to update Project Wannabe. By using Project Wannabe, you can utilize the model's features without worrying about the detailed prompt formats described below.

✨ Features

wanabi-24B v0.1 can perform the following main tasks. Features have been added and improved compared to the Preview version.

New Features

1. Author's Note Function (New)
   • Purpose: By describing the upcoming events, actions, mood descriptions, etc., within approximately 1000 characters in the near future, this function guides the generation of subsequent text more precisely. It is different from the plot that shows the overall structure of the novel.
   • Application: It is incorporated into the prompt for the continuation generation (CONT) task.
2. Rating Function (New)
   • Purpose: Specify the rating (general or r18) of the generated content.
   • Application: Rating: {specified value} is appended to the end of the instruction for all tasks (GEN, CONT, IDEA). In Project Wannabe, the value set in the "Detailed Information" tab takes precedence.
3. Dialogue Volume Specification Function (New)
   • Purpose: Select the proportion of dialogue in the generated text from options such as "Not specified", "Low", "Slightly low", "Normal", "Slightly high", and "High". (This feature is not fully implemented in the current v0.1 but is planned for future versions.)
   • Application: When an option other than "Not specified" is selected, # Dialogue Volume: {specified value} is included in the input part (within the reference information block) of the prompt for the text generation (GEN) and continuation generation (CONT) tasks.

Main Features and Improvements

1. Text Generation (GEN)
   • Generate a novel text based on the given instructions, optionally provided metadata (title, keywords, genre, synopsis, settings, plot), dialogue volume, and rating.
   • It is also possible to generate freely without metadata.
2. Continuation Generation (CONT)
   • Generate a continuation of the given text, taking into account the optionally provided metadata, dialogue volume, rating, and the newly added Author's Note.
   • Significant Improvement in Prompt Structure: The prompt structure has been changed from the previous (Reference Information/Text) → Response format to [ (Reference Information/Main Part of Text) → Author's Note (optional) → Last about 3 lines of Text ] → Response to better capture the context and enhance controllability. The order of reference information and the main part of the text can also be changed in the settings of Project Wannabe.
3. Idea Generation (IDEA)
   • Generate a complete novel idea (title, keywords, genre, synopsis, settings, plot) based on part (or none) of the optionally provided metadata and rating.
   • (The dialogue volume does not affect this task.)

🔧 Technical Details

Base Model

• mistralai/Mistral-Small-24B-Base-2501
  • (During training, unsloth/Mistral-Small-24B-Base-2501-bnb-4bit was used)

Training Framework

• LLaMA-Factory

Training Method

• Method: Supervised Fine - Tuning (SFT)
• Quantization/Adapter: QLoRA (4 - bit)
  • lora_rank: 128
  • lora_alpha: 256
  • lora_dropout: 0.05
  • lora_target: all (all linear layers)
• Precision: bf16
• Sequence Length: 32768
• Batch Size: per_device_train_batch_size = 1, gradient_accumulation_steps = 24 (effective batch size 24)
• Optimization:
  • Optimizer: PagedAdamW (8 - bit)
  • Flash Attention 2: Enabled (fa2)
  • Unsloth Gradient Checkpointing: Enabled (use_unsloth_gc: true)
  • Liger Kernel: Enabled (enable_liger_kernel: true)
• Learning Rate:
  • learning_rate: 3.0e - 5
  • lr_scheduler_type: cosine_with_restarts
  • lr_scheduler_kwargs: {"num_cycles": 5}
  • warmup_ratio: 0.03

📚 Documentation

Prompt Format (mistral_small Template)

This model has been trained using the mistral_small chat template format in LLaMA - Factory. We recommend using the same format during inference. If you use Project Wannabe, you don't need to worry about the following format.

Basic Structure: <s>[INST]{Instruction}\n{Input}[/INST]{Output}</s>

• <s>, </s>: Start and end tokens
• [INST], [/INST]: Tags indicating the start and end of user instructions (Do not insert a space between the tag and the content)
• {Instruction}: Task instruction (e.g., "Please generate a novel.") + Rating
• {Input}: Input information required for the task (metadata, part of the text, etc.). There is one newline (\n) between the Instruction and the Input.
• {Output}: Response text to be generated by the model.

Format Examples

1. Text Generation (GEN)

• Example (with Metadata and Dialogue Volume):

<s>[INST]Please generate a novel text based on the following information. Rating: general
# Title:
The Magician on the Star - Falling Night
# Keywords:
Fantasy
Magic Academy
# Dialogue Volume: High[/INST]{Generated Text}</s>

• Example (without Metadata):

<s>[INST]Please generate a novel freely. Rating: general[/INST]{Generated Text}</s>

2. Continuation Generation (CONT)

• Example (with Reference Information and Author's Note):

<s>[INST]Based on the reference information and the text, please generate a novel that is a natural continuation of the last sentence. Rating: general
【Reference Information】

Genre:

School Comedy

Dialogue Volume: Normal

【Text】

While walking in the corridor, I bumped into a classmate who jumped out from a corner. She was holding a large number of prints in her hand. "Whoops, sorry—"

【Author's Note】

The prints fly into the air due to the impact of the collision. The protagonist helps to pick them up in a hurry. The girl is a bit clumsy but popular.

She blushed and said.[/INST]{Generated Continuation Text}</s>

(Note: The order of 【Reference Information】 and 【Text】 can be changed in the settings of Project Wannabe)

• Example (Main Part and End of the Text Only):

<s>[INST]Based on the text, please generate a novel that is a natural continuation of the last sentence. Rating: r18
【Text】

In a dark alley, I was pressed against the wall, and his hot breath hit my ear. There was no way to resist as the difference in strength was obvious. "…Stop…" My weak voice was swallowed by the night.

His lips slowly approached—[/INST]{Generated Continuation Text}</s>

3. Idea Generation (IDEA)

• Example (with Some Metadata):

<s>[INST]Please generate a complete novel idea (title, keywords, genre, synopsis, settings, plot) based on the following information. Rating: general
# Genre:
SF
Cyberpunk[/INST]# Title:
The Lonely Detective in the Cyber City

# Keywords:
Cyberpunk
Detective
Android
Megacorporation
Conspiracy

# Genre:
SF
Cyberpunk
Hard - boiled

# Synopsis:
In a megacity in 2077, it's pouring rain. Jack, a frustrated detective, receives a request to search for a missing android singer. As the investigation progresses, he gets involved in the conspiracy of a megacorporation that controls the city behind the scenes.

# Settings:
The story is set in a future city where highly advanced technology coexists with a society of inequality. People are constantly connected to the network through brain implants. Androids have evolved to be indistinguishable from humans.

# Plot:
Receive the request -> Collect information and interview related people -> Encounter obstacles and attacks -> Discover evidence leading to the core -> Confront the megacorporation -> Shocking ending</s>

• Example (without Metadata):

<s>[INST]Please generate a novel idea (title, keywords, genre, synopsis, settings, plot) freely. Rating: general[/INST]{Generated Set of Ideas}</s>

⚠️ Limitations and Precautions

• Model Under Development: This model is still under development, and v0.1 is the initial release. Performance and stability may improve in future versions. Thank you to all users who cooperated in using the Preview version.
• Bias: Due to the characteristics of the training data, the generated content may be biased towards specific genres, expressions, and developments.
• Inappropriate Content: Since the training data contains diverse texts, the model may generate text that is not suitable for minors or may cause discomfort. Although we try to control it with the rating function, it is not perfect.
• Quality Limits: There are limits to the diversity, consistency, and context - following ability of the generated text. There is a possibility of failure, especially when generating long texts. Please try guiding with the Author's Note function.
• Usage Precautions: This model is provided for research and experimental purposes. Users are responsible for complying with applicable laws and regulations. Using it for illegal purposes or to infringe on the rights of others is strictly prohibited.
• Personal Responsibility: The developers are not responsible for any results caused by the use of this model. Please use it at your own risk.

Future Plans

• Continuous Training: We are currently continuing training with an extended dataset and plan to release version updates at each training checkpoint.
• Consideration of Small - Scale Models: We are also considering training smaller, more lightweight, and user - friendly models (such as those in the 12B class).

(The roadmap is subject to change)

📄 License

This project is licensed under the Apache - 2.0 license (conforming to the base model, tentative - please change as needed).