Wayfarer 12B

🚀 Wayfarer-12B

Wayfarer-12B is an adventure role - play model designed to offer players a challenging and dangerous experience, countering the positivity bias in modern AI models.

🚀 Quick Start

If you want to easily try this model for free, you can do so at https://aidungeon.com. Or, anyone can download the model to run locally. Quantized GGUF weights can be downloaded here.

✨ Features

• Challenging Experience: Wayfarer is specifically trained to provide players with a challenging and dangerous adventure, where failure, death, and restarts are possible, creating real stakes and consequences.
• Countering Positivity Bias: It has a pessimistic emotional sentiment, with frequent failures and no plot armor, countering the inherent positivity bias in current language models.

📦 Installation

The document doesn't provide specific installation steps, so this section is skipped.

💻 Usage Examples

Basic Usage

The Nemo architecture is known for being sensitive to higher temperatures, so the following settings are recommended as a baseline:

"temperature": 0.8,
"repetition_penalty": 1.05,
"min_p": 0.025

Advanced Usage

You can experiment with different settings based on your specific needs.

📚 Documentation

Model details

Wayfarer 12B was trained on top of the Nemo base model using a two - stage SFT approach. The first stage contained 180K chat - formatted instruct data instances, and the second stage consisted of a 50/50 mixture of synthetic 8k context text adventures and roleplay experiences.

How It Was Made

Wayfarer’s text adventure data was generated by simulating playthroughs of published character creator scenarios from AI Dungeon. Five distinct user archetypes played through each scenario, with characters starting with different factions, locations, etc., to generate five unique samples. One language model played the role of narrator, and the other played the user. They were blind to each other’s underlying logic, allowing the user to surprise the narrator with their choices. Each simulation ran for 8k tokens or until the main character died.

Prompt Format

ChatML was used for both finetuning stages.

<|im_start|>system
You're a masterful storyteller and gamemaster. Write in second person present tense (You are), crafting vivid, engaging narratives with authority and confidence.<|im_end|>
<|im_start|>user
> You peer into the darkness.<|im_end|>
<|im_start|>assistant
You have been eaten by a grue.

GAME OVER<|im_end|>

🔧 Technical Details

• Training Approach: Two - stage SFT approach on top of the Nemo base model.
• Data Generation: Through simulating playthroughs of character creator scenarios in AI Dungeon with multiple user archetypes.
• Emotional Sentiment: Pessimistic to counter the positivity bias in current language models.

📄 License

This model is licensed under the Apache-2.0 license.

Limitations

• Tense and Style: Wayfarer was trained exclusively on second - person present tense data in a narrative style. Other styles may work but could produce sub - optimal results.
• Chat Data: It was trained exclusively on single - turn chat data.

Credits

Thanks to Gryphe Padar for collaborating on this finetune with us!