Rugpt3medium_sum_gazeta Open-source Model - Free Generation of Russian Abstract Summaries, Trained on the Gazeta Dataset

Home

Rugpt3medium Sum Gazeta

Developed by IlyaGusev

Russian abstractive summarization model based on rugpt3medium_based_on_gpt2, specifically trained on the Gazeta dataset

Text Generation

Transformers

OtherOpen Source License:Apache-2.0 #Russian abstract summarization #Causal language model #Gazeta dataset

Downloads 1,228

Release Time : 3/2/2022

Model Overview

This is a Russian causal language model specifically designed for generating abstract summaries of news articles. Fine-tuned from the rugpt3medium model based on GPT-2 architecture, it performs exceptionally well on the Gazeta dataset.

Model Features

Russian language optimization

Specially trained and optimized for Russian text, performing well on Russian summarization tasks

Gazeta dataset adaptation

Fine-tuned using the Gazeta news dataset, making it particularly suitable for generating summaries of news articles

Abstractive summarization capability

Capable of generating abstract summaries of articles, not just extracting key sentences

Model Capabilities

Russian text comprehension

News article summarization

Long text processing (up to 600 tokens)

Use Cases

News media

Automated news summarization

Automatically generates concise summaries for news articles

Achieved 24.1 R-1-f score on Gazeta test set

Content aggregation

Multi-source news summarization

Generates uniformly formatted summaries for Russian news from different sources

🚀 RuGPT3MediumSumGazeta

This is a model for abstractive summarization in Russian, leveraging the power of rugpt3medium_based_on_gpt2 to provide efficient and accurate summaries.

🚀 Quick Start

✨ Features

Abstractive Summarization: Tailored for Russian text summarization.
Based on Rugpt3medium: Built upon the robust rugpt3medium_based_on_gpt2 model.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "IlyaGusev/rugpt3medium_sum_gazeta"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

article_text = "..."

text_tokens = tokenizer(
    article_text,
    max_length=600,
    add_special_tokens=False, 
    padding=False,
    truncation=True
)["input_ids"]
input_ids = text_tokens + [tokenizer.sep_token_id]
input_ids = torch.LongTensor([input_ids])

output_ids = model.generate(
    input_ids=input_ids,
    no_repeat_ngram_size=4
)

summary = tokenizer.decode(output_ids[0], skip_special_tokens=False)
summary = summary.split(tokenizer.sep_token)[1]
summary = summary.split(tokenizer.eos_token)[0]
print(summary)

📚 Documentation

Model description

This is the model for abstractive summarization for Russian based on rugpt3medium_based_on_gpt2.

Intended uses & limitations

How to use

Colab: link

🔧 Technical Details

Training data

Dataset: Gazeta

Training procedure

Training script: train.py
Config: gpt_training_config.json

Eval results

Train dataset	Test dataset	Source max_length	Target max_length	no_repeat_ngram_size	num_beams
Gazeta v1 train	Gazeta v1 test	600	200	4	5
Gazeta v1 train	Gazeta v2 test	600	200	4	5

Model	R-1-f	R-2-f	R-L-f	chrF	METEOR	BLEU	Avg char length
mbart_ru_sum_gazeta (Gazeta v1 test)	32.4	14.3	28.0	39.7	26.4	12.1	371
rut5_base_sum_gazeta (Gazeta v1 test)	32.2	14.4	28.1	39.8	25.7	12.3	330
rugpt3medium_sum_gazeta (Gazeta v1 test)	26.2	7.7	21.7	33.8	18.2	4.3	244
mbart_ru_sum_gazeta (Gazeta v2 test)	28.7	11.1	24.4	37.3	22.7	9.4	373
rut5_base_sum_gazeta (Gazeta v2 test)	28.6	11.1	24.5	37.2	22.0	9.4	331
rugpt3medium_sum_gazeta (Gazeta v2 test)	24.1	6.5	19.8	32.1	16.3	3.6	242

Evaluation script: evaluate.py

Flags: --language ru --tokenize-after --lower

📄 License

This model is licensed under the apache-2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご