Bart-base-xsum Open-source Abstract Generation Model - Free Deployment, Precise Extraction of Concise Summaries

Bart Base Xsum

Developed by morenolq

A summarization model fine-tuned on the xsum dataset based on facebook/bart-base, excelling in generating concise and accurate summaries.

Text Generation

Transformers

Open Source License:Apache-2.0 #News Summarization #High ROUGE Score #BART Fine-tuning

Downloads 15

Release Time : 9/18/2022

Model Overview

This model is a pre-trained model based on the BART architecture, specifically fine-tuned for summarization tasks, particularly suitable for generating brief summaries of news articles.

Model Features

Efficient Summarization

Performs excellently on the XSUM dataset, capable of generating concise and accurate news summaries.

Multi-Dataset Adaptation

Not only performs well on the XSUM dataset but also adapts to summarization tasks with different styles like CNN/DailyMail.

Stable Training Process

After 10 training epochs, validation metrics show stable improvement, demonstrating good training convergence.

Model Capabilities

Text Summarization

News Content Extraction

Long Text Compression

Use Cases

News Media

News Summarization

Automatically generates concise summaries for long news articles

Achieves a ROUGE-1 score of 38.6513 on the XSUM test set

Content Analysis

Document Content Extraction

Extracts key information from long documents

Achieves a ROUGE-1 score of 14.9833 on the CNN/DailyMail test set

🚀 bart-base-xsum

This is a fine - tuned model based on facebook/bart - base for text summarization tasks. It has been trained on the xsum dataset and shows good performance on evaluation metrics.

🚀 Quick Start

This model is a fine - tuned version of [facebook/bart - base](https://huggingface.co/facebook/bart - base) on xsum dataset. It achieves the following results on the evaluation set:

Loss: 0.8051
R1: 0.5643
R2: 0.3017
Rl: 0.5427
Rlsum: 0.5427

Training: The model has been trained using the script provided in the following repository https://github.com/MorenoLaQuatra/transformers - tasks - templates

📚 Documentation

Model Information

Property	Details
Model Type	Fine - tuned version of `facebook/bart - base` for summarization
Training Data	`xsum` dataset

Training and Evaluation Metrics

The model has been evaluated on multiple datasets with various metrics:

xsum (test split)

Metric	Value
ROUGE - 1	38.6513
ROUGE - 2	17.7585
ROUGE - L	32.2033
ROUGE - LSUM	32.2064
Loss	1.822434902191162
gen_len	19.7028

xsum (validation split)

Metric	Value
ROUGE - 1	38.7415
ROUGE - 2	17.8295
ROUGE - L	32.2861
ROUGE - LSUM	32.2763
Loss	1.8132821321487427
gen_len	19.7116

cnn_dailymail (test split)

Metric	Value
ROUGE - 1	14.9833
ROUGE - 2	3.2788
ROUGE - L	11.0507
ROUGE - LSUM	13.3138
Loss	5.7526116371154785
gen_len	18.3794

Training Procedure

Training Hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e - 05
train_batch_size: 32
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9, 0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 500
num_epochs: 10

Training Results

Training Loss	Epoch	Step	Validation Loss	R1	R2	Rl	Rlsum
0.8983	1.0	6377	0.8145	0.5443	0.2724	0.5212	0.5211
0.8211	2.0	12754	0.7940	0.5519	0.2831	0.5295	0.5295
0.7701	3.0	19131	0.7839	0.5569	0.2896	0.5347	0.5348
0.7046	4.0	25508	0.7792	0.5615	0.2956	0.5394	0.5393
0.6837	5.0	31885	0.7806	0.5631	0.2993	0.5416	0.5416
0.6412	6.0	38262	0.7816	0.5643	0.301	0.5427	0.5426
0.6113	7.0	44639	0.7881	0.5645	0.3017	0.5428	0.5428
0.5855	8.0	51016	0.7921	0.5651	0.303	0.5433	0.5432
0.5636	9.0	57393	0.7972	0.5649	0.3032	0.5433	0.5433
0.5482	10.0	63770	0.7996	0.565	0.3036	0.5436	0.5435

Framework Versions

Transformers 4.22.1
Pytorch 1.11.0+cu113
Datasets 2.0.0
Tokenizers 0.11.6

📄 License

This model is released under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご