flan-t5-large-stacked-samsum-1024 Open-source Summary Generation Model - Enhance the Accuracy of Information Extraction

Flan T5 Large Stacked Samsum 1024

Developed by stacked-summaries

A summary generation model fine-tuned on the stacked-samsum-1024 dataset based on google/flan-t5-large, using stacked summarization method to improve information extraction capability

Text Generation

Transformers

EnglishOpen Source License:Apache-2.0 #Stacked Summary Generation #Multi-Concept Segmentation #Task-Oriented Pretraining

Downloads 16

Release Time : 12/6/2022

Model Overview

This model is trained with the stacked summarization method, enabling better identification and extraction of key information from text, especially suitable for dialogue summarization tasks. The model uses [NEXT_CONCEPT] tokens to separate different concepts for easier information segmentation.

Model Features

Stacked Summarization Training

Trained with stacked summarization method, enabling the model to better identify and separate key concepts in text

Concept Segmentation Tokens

Uses [NEXT_CONCEPT] tokens to automatically segment different concepts in the output summary

Efficient Information Extraction

Focuses on extracting and condensing key information from dialogue text rather than simply mimicking summarization styles

Model Capabilities

Dialogue summarization generation

Multi-concept information extraction

Text condensation

Use Cases

Dialogue Processing

Customer Service Dialogue Summarization

Automatically generates concise summaries of customer service dialogues, highlighting key issues and solutions

ROUGE-1 score 47.6682, ROUGE-L score 39.7678

Meeting Minutes Condensation

Condenses lengthy meeting minutes into key decision points and action items

🚀 flan-t5-large-stacked-samsum-1024

This model is a fine - tuned version of google/flan-t5-large on the stacked-summaries/stacked-samsum-1024 dataset. It aims to enhance text summarization capabilities.

🚀 Quick Start

This model is a fine - tuned version of google/flan-t5-large on the stacked-summaries/stacked-samsum-1024 dataset.

It achieves the following results on the evaluation set:

Loss: 2.1846
Rouge1: 57.9637
Rouge2: 28.7446
Rougel: 44.3826
Rougelsum: 54.0399
Gen Len: 122.77

✨ Features

This model card presents a model trained on a stacked dataset that aims to improve summarization by testing the benefits of "task - oriented pretraining". The model is designed to learn how to effectively condense and distill information from text by stacking summaries and separating them into independent concepts. In this way, the model can learn to identify essential information without simply mimicking the style of the dataset summaries.

The token used to identify a new concept in the summary is [NEXT_CONCEPT]. You can split an output summary based on this token to see how it split the input text information: summary_text.split("[NEXT_CONCEPT]") etc.

📚 Documentation

Intended uses & limitations

max input/output is 1024 tokens
this is mostly a test because samsum is not exactly the best dataset for general - purpose summarization

Training and evaluation data

See the dataset card linked on this page for info

🔧 Technical Details

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 4
seed: 24915
distributed_type: multi - GPU
gradient_accumulation_steps: 32
total_train_batch_size: 256
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e - 08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.02
num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.1195	0.17	20	2.0635	57.8829	28.7887	44.4256	54.1299	121.8
0.1084	0.35	40	2.1178	58.0416	28.6487	44.3905	54.1557	122.893
0.1019	0.52	60	2.1576	57.816	28.7069	44.4242	53.9598	120.524
0.0975	0.7	80	2.1821	57.9597	28.8178	44.4854	54.068	121.793
0.0947	0.87	100	2.1846	57.9637	28.7446	44.3826	54.0399	122.77

Framework versions

Transformers 4.26.0.dev0
Pytorch 1.13.0+cu117
Datasets 2.6.1
Tokenizers 0.13.1

📄 License

This model is licensed under the Apache - 2.0 license.

📦 Information Table

Property	Details
Model Type	Fine - tuned version of google/flan - t5 - large
Training Data	stacked - summaries/stacked - samsum - 1024
Metrics	Rouge (ROUGE - 1, ROUGE - 2, ROUGE - L, ROUGE - LSUM), Loss, Gen Len
Base Model	google/flan - t5 - large

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご