Storyseeker

🚀 StorySeeker

This model is a fine - tuned version of RoBERTa - base, designed to predict whether a text contains a story. It offers high accuracy and can be easily used with the provided Colab demo.

🚀 Quick Start

You can view a demonstration of how to load our annotations, fetch the texts, load our fine - tuned model from Hugging Face, and run predictions. If you use the Colab link, you don't need to download anything or set up anything on your local machine; everything will run in your internet browser.

Colab: link

Github: link

✨ Features

• Story Prediction: This model can be used to predict whether a text contains or does not contain a story.
• Fine - Tuned on Specific Dataset: It is a fine - tuned version of [roberta - base](https://huggingface.co/roberta - base) on the [🔭StorySeeker](https://github.com/maria - antoniak/storyseeker) dataset.
• High Evaluation Metrics: Achieves a loss of 0.4343 and an accuracy of 0.8416 on the evaluation set.

📦 Installation

No specific installation steps are provided in the original document.

📚 Documentation

Model description

This model can be used to predict whether a text contains or does not contain a story.

For our definition of "story" please refer to our [codebook](https://github.com/maria - antoniak/storyseeker).

Intended uses & limitations

This model is intended for researchers interested in measuring storytelling in online communities, though it can be applied to other kinds of datasets (see generalization results in our preprint).

Training and evaluation data

The model was fine - tuned on the training split of the [🔭StorySeeker](https://github.com/maria - antoniak/storyseeker) dataset, which contains 301 Reddit posts and comments annotated with story and event spans. This model was fine - tuned using binary document labels (the document contains a story or does not contain a story).

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 16
• eval_batch_size: 20
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 20
• num_epochs: 3

Training results

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.2

📄 License

This project is licensed under the MIT license.

📚 Citation

If you use our data, codebook, or models, please cite the following preprint:

[Where do people tell stories online? Story Detection Across Online Communities](https://github.com/maria - antoniak/storyseeker/blob/main/2024_where_are_stories_preprint.pdf)
Maria Antoniak, Joel Mire, Maarten Sap, Elliott Ash, Andrew Piper