SimpleStories-30M Open-Source Story Generation Model - Free Deployment to Create Personalized Story Content

Simplestories 30M

Developed by SimpleStories

SimpleStories is a micro model family designed for interpretability research, trained on the SimpleStories dataset, focusing on story generation tasks.

Text Generation

Safetensors

EnglishOpen Source License:MIT #Micro Story Generation #Efficient Text Generation #Interpretability Research

Downloads 735

Release Time : 4/22/2025

Model Overview

SimpleStories is a small language model family specifically designed for story generation tasks. The model family includes multiple variants of different sizes, ranging from 1.25M to 35M parameters, suitable for efficient natural language processing and research purposes.

Model Features

Micro Model Design

The model family includes variants ranging from 1.25M to 35M parameters, suitable for research and educational purposes.

Efficient Tokenizer

Uses a custom WordPiece tokenizer with a vocabulary size of only 4096, achieving efficient processing without compromising generation quality.

Interpretability Research

Specifically designed for interpretability research, the model is small in size but fully functional, facilitating analysis of internal mechanisms.

Diverse Model Variants

Offers five different model variants to meet various computational resource and performance requirements.

Model Capabilities

Story Generation

Text Continuation

Creative Writing Assistance

Use Cases

Education & Research

Language Model Interpretability Research

Due to its small size, the model is suitable for studying the internal mechanisms and learning patterns of language models.

Teaching Demonstrations

Ideal for demonstrating the basic principles and workings of language models in classroom settings.

Creative Writing

Story Generation

Generates coherent short stories based on brief prompts.

Evaluation metrics for generation quality show that the model can produce coherent and creative stories.

Writing Inspiration

Provides creative inspiration and story development directions for writers.

🚀 SimpleStories Model Family

The SimpleStories models are a tiny model family designed for interpretability research. They are trained on the SimpleStories dataset, offering a practical solution for exploring model interpretability.

🚀 Quick Start

The SimpleStories models provide an easy - to - use interface for text generation. Here's a basic guide on how to get started.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, LlamaForCausalLM


MODEL_SIZE = "30M"
model_path = "SimpleStories/SimpleStories-{}".format(MODEL_SIZE)

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = LlamaForCausalLM.from_pretrained(model_path)
model.to("cuda")
model.eval()

prompt = "The curious cat looked at the"

inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
input_ids = inputs.input_ids.to("cuda")

eos_token_id = 1

with torch.no_grad():
    output_ids = model.generate(
        input_ids=input_ids,
        max_new_tokens=400,
        temperature=0.7,
        do_sample=True,
        eos_token_id=eos_token_id
)

output_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(f"\nGenerated text:\n{output_text}")

📚 Documentation

Model Variants

The SimpleStories model family offers several variants with different parameter counts and architectural configurations.

Model Name	n_params	n_layers	d_model	n_heads	n_ctx	d_vocab
SimpleStories-35M	35 million	12	512	8	512	4096
SimpleStories-30M	30 million	10	512	8	512	4096
SimpleStories-11M	11 million	6	384	6	512	4096
SimpleStories-5M	5 million	6	256	4	512	4096
SimpleStories-1.25M	1.25 million	4	128	4	512	4096

Performance Comparison

The generation quality of the models is evaluated through specific metrics. The following figure shows a comparison of different models.

Tokenizer

We use a custom WordPiece tokenizer with a vocabulary size of 4096. Through morphological analysis and coverage gain analysis on the dataset, we built this small - sized tokenizer without sacrificing generation quality.

Dataset

The SimpleStories dataset consists of short stories generated by state - of - the - art language models. It has the following features:

Story annotation with high - level concepts: theme, topic, style, etc.
Higher semantic and syntactic diversity through seeded story generation
Generated by 2024 models
Several pre - computed NLP metrics to assist in filtering
ASCII - only guarantee for the English dataset

You can read the dataset paper on arXiv.

Training

The training and evaluation scripts can be found at https://github.com/danbraunai/simple_stories_train

📄 License

This project is licensed under the MIT License.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご