t5-small-Abstractive-Summarizer Open-source Text Summarization Model - Generate High-quality Abstractive Summaries for Free

T5 Small Abstractive Summarizer

Developed by MK-5

A text summarization model based on the T5-small architecture, fine-tuned on the multi_news dataset, excelling in generating abstractive summaries

Text Generation

Transformers

Open Source License:Apache-2.0 #Multi-document Summarization #T5 Fine-tuning #News Summarization Generation

Downloads 80

Release Time : 9/6/2024

Model Overview

This model is a sequence-to-sequence model based on the T5-small architecture, specifically optimized for multi-document summarization tasks, capable of generating concise summaries from multiple input documents.

Model Features

Multi-document Summarization Capability

Optimized specifically for the multi_news dataset, capable of processing multiple related documents and generating unified summaries

Abstractive Summarization

Not only extracts key sentences but also generates new summarizing statements

Lightweight Model

Based on the T5-small architecture, maintaining good performance while having a smaller model size

Model Capabilities

Text Summarization Generation

Multi-document Content Integration

Abstractive Content Summarization

Use Cases

News Aggregation

Multi-source News Summarization

Integrates reports of the same event from different media sources into concise summaries

Achieved a Rouge1 score of 15.7 on the multi_news validation set

Research Assistance

Literature Review Assistance

Helps researchers quickly grasp the core content of multiple related papers

🚀 t5-small-Abstractive-Summarizer

This is a fine - tuned model based on t5-small for abstractive summarization on the multi_news dataset. It provides valuable results in text summarization tasks, offering a reliable solution for sequence - to - sequence language modeling.

🚀 Quick Start

This model is a fine - tuned version of t5-small on the multi_news dataset. It achieves the following results on the evaluation set:

Loss: 2.7737
Rouge1: 15.7032
Rouge2: 5.2433
Rougel: 12.282
Rougelsum: 14.0946

✨ Features

Fine - tuned on multi_news: Trained on the multi_news dataset, enabling it to handle abstractive summarization tasks effectively.
Multiple evaluation metrics: Evaluated using metrics like Rouge1, Rouge2, Rougel, and Rougelsum, providing a comprehensive assessment of its performance.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

📚 Documentation

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

🔧 Technical Details

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.00056
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
lr_scheduler_type: linear
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.118	1.0	113	2.7677	15.1343	4.7712	11.8812	13.386
2.7857	2.0	226	2.7609	15.7641	4.8705	12.0955	13.9779
2.6158	3.0	339	2.7494	15.1515	4.4523	11.7147	13.4181
2.4962	4.0	452	2.7743	15.344	5.1073	12.1574	13.7917
2.4304	5.0	565	2.7737	15.7032	5.2433	12.282	14.0946

Framework versions

Transformers 4.44.2
Pytorch 2.4.0+cu121
Datasets 2.21.0
Tokenizers 0.19.1

📄 License

This project is licensed under the Apache - 2.0 license.

📄 Information Table

Property	Details
Library Name	transformers
Model Type	t5 - small - Abstractive - Summarizer
Base Model	t5 - small
Tags	summarization, generated_from_trainer
Datasets	multi_news
Metrics	rouge
Training Hyperparameters	learning_rate: 0.00056, train_batch_size: 8, eval_batch_size: 8, seed: 42, optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08, lr_scheduler_type: linear, num_epochs: 5
Framework Versions	Transformers 4.44.2, Pytorch 2.4.0+cu121, Datasets 2.21.0, Tokenizers 0.19.1

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご