Fairseq - Dense - 13B - Janeway Open-source Model - Freely Generate Exciting Science Fiction and Fantasy Texts

Fairseq Dense 13B Janeway

Developed by KoboldAI

A fine-tuned version based on Fairseq's Mixture of Experts dense model, primarily used for sci-fi and fantasy text generation

Large Language Model

Transformers

EnglishOpen Source License:MIT #Sci-Fi Text Generation #Mixture of Experts Architecture #Ebook Style Fine-tuning

Downloads 1,287

Release Time : 4/6/2022

Model Overview

This model is a large-scale language model fine-tuned on Fairseq's Mixture of Experts dense architecture, specializing in sci-fi and fantasy text generation tasks

Model Features

Sci-Fi/Fantasy Text Optimization

Specially fine-tuned for sci-fi and fantasy texts, generating content that better aligns with genre characteristics

Data Annotation Format

Training data uses specific format tagging like [Genre:<genre1>,<genre2>], helping control generated content style

Large-Scale Parameters

13B parameter scale provides strong language understanding and generation capabilities

Model Capabilities

Text Generation

Stylized Writing

Sci-Fi/Fantasy Content Creation

Use Cases

Creative Writing

Sci-Fi Novel Creation

Generating coherent text passages with sci-fi style

Can produce natural language text conforming to sci-fi settings

Fantasy Story Generation

Generating fantasy-themed storylines based on prompts

Produces coherent narratives with fantasy elements

Dialogue Systems

Character Dialogue Simulation

Simulating dialogue styles of characters in sci-fi works

Generates natural dialogues consistent with character settings

🚀 Fairseq-dense 13B - Janeway

A finetuned model based on Fairseq's MoE dense model, offering text generation capabilities.

🚀 Quick Start

You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:

>>> from transformers import pipeline
>>> generator = pipeline('text-generation', model='KoboldAI/fairseq-dense-13B-Janeway')
>>> generator("Welcome Captain Janeway, I apologize for the delay.", do_sample=True, min_length=50)
[{'generated_text': 'Welcome Captain Janeway, I apologize for the delay."\nIt's all right," Janeway said. "I'm certain that you're doing your best to keep me informed of what\'s going on."'}]

✨ Features

Fairseq-dense 13B-Janeway is a finetune created using Fairseq's MoE dense model.

📦 Installation

No installation steps are provided in the original README, so this section is skipped.

💻 Usage Examples

Basic Usage

>>> from transformers import pipeline
>>> generator = pipeline('text-generation', model='KoboldAI/fairseq-dense-13B-Janeway')
>>> generator("Welcome Captain Janeway, I apologize for the delay.", do_sample=True, min_length=50)
[{'generated_text': 'Welcome Captain Janeway, I apologize for the delay."\nIt's all right," Janeway said. "I'm certain that you're doing your best to keep me informed of what\'s going on."'}]

📚 Documentation

Model Details

Property	Details
Model Type	A finetune created using Fairseq's MoE dense model
Training Data	The training data contains around 2210 ebooks, mostly in the sci-fi and fantasy genres. The dataset is identical as dataset used by GPT-Neo-2.7B-Janeway. Some parts of the dataset have been prepended using the following text: `[Genre: <genre1>,<genre2>]`

Limitations and Biases

⚠️ Important Note

Based on known problems with NLP technology, potential relevant factors include bias (gender, profession, race and religion).

BibTeX entry and citation info

Artetxe et al. (2021): Efficient Large Scale Language Modeling with Mixtures of Experts

📄 License

This project is licensed under the MIT license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご