Rut5 Base Headline Generation

🚀 ruT5-base_headline_generation

A T5 Base model for Russian news headline generation. It's fine-tuned for short news texts and also performs well on longer articles, generating abstractive headlines with an average of 6 - 11 words.

🚀 Quick Start

The ruT5-base_headline_generation model is designed for generating headlines for Russian news articles. Here's a basic example of how to use it:

from transformers import AutoTokenizer, T5ForConditionalGeneration

model_name = "wanderer-msk/ruT5-base_headline_generation"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

news_text = """Земляне продолжают осваивать Марс.
Колонисты уже посадили на красной планете 42 яблони."""

model_input = tokenizer(
    news_text,
    truncation=True,
    max_length=1024,
    return_tensors="pt"
)
model_output = model.generate(model_input["input_ids"])
news_headline = tokenizer.decode(
    model_output.squeeze(),
    skip_special_tokens=True
)

print(news_headline)

✨ Features

• Optimized for Short Texts: The model is fine-tuned to achieve the best performance on short news texts (128 words or less).
• Good Performance on Long Articles: It also has decent metrics on longer news articles.
• Abstractive Headline Generation: Generates abstractive headlines that on average include 6 - 11 words.

📦 Installation

To use this model, you need to install the transformers library. You can install it using pip:

pip install transformers

📚 Documentation

Model Details

The model is a T5 Base model adapted for news headline generation in Russian. Base Model: ai-forever/ruT5-base

Training Details

• Training Data: 247,000 Russian news articles
• Training Procedure: The model was trained for 6 epochs. All details and hyperparameters can be found in this Google Colab notebook

Testing Metrics

• Rouge1: 40.24
• Rouge2: 23.05
• RougeL: 37.57