Billsum Tiny Summarization Open-Source Text Summarization Model - Precise Extraction of Core Document Content

Home

Billsum Tiny Summarization

Developed by jotamunz

Text summarization model fine-tuned on google/t5-efficient-tiny, trained on the billsum dataset

Text Generation

Transformers

Open Source License:Apache-2.0 #Legal Text Summarization #Efficient Lightweight #T5 Fine-tuning

Downloads 932

Release Time : 9/30/2023

Model Overview

This model is specifically designed for automatic summarization of legal bill texts, capable of compressing lengthy legal documents into concise summaries.

Model Features

Efficient and Lightweight

Based on the T5-efficient-tiny architecture, the model is small in size and suitable for resource-constrained environments.

Legal Text Optimization

Fine-tuned specifically on the billsum legal bill dataset, making it ideal for legal document summarization tasks.

Fast Generation

Average generation length is only 19 tokens, suitable for scenarios requiring quick summaries.

Model Capabilities

Legal Text Summarization

Text Compression

Key Information Extraction

Use Cases

Legal Document Processing

Bill Summary Generation

Automatically generates concise summaries of lengthy legal bills.

Rouge1 score 0.1503

Legal Research Assistance

Helps legal researchers quickly understand the content of bills.

🚀 billsum_tiny_summarization

This model is a fine - tuned version of [google/t5 - efficient - tiny](https://huggingface.co/google/t5 - efficient - tiny) on the billsum dataset, designed for text summarization tasks.

🚀 Quick Start

This model is a fine - tuned version of [google/t5 - efficient - tiny](https://huggingface.co/google/t5 - efficient - tiny) on the billsum dataset. It achieves the following results on the evaluation set:

Loss: 3.5889
Rouge1: 0.1503
Rouge2: 0.0412
Rougel: 0.1244
Rougelsum: 0.1244
Gen Len: 19.0

✨ Features

Based on the pre - trained model google/t5 - efficient - tiny, it is fine - tuned on the billsum dataset for summarization tasks.
Provides multiple evaluation metrics such as Rouge1, Rouge2, etc.

📦 Installation

No installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

No code examples are provided in the original document, so this section is skipped.

📚 Documentation

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

🔧 Technical Details

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e - 05
train_batch_size: 16
eval_batch_size: 16
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
lr_scheduler_type: linear
num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	62	4.2835	0.1413	0.0323	0.1125	0.1124	19.0
No log	2.0	124	3.7275	0.1507	0.0408	0.1263	0.1264	19.0
No log	3.0	186	3.6154	0.1499	0.0407	0.1244	0.1244	19.0
No log	4.0	248	3.5889	0.1503	0.0412	0.1244	0.1244	19.0

Framework versions

Transformers 4.33.3
Pytorch 2.0.1+cu118
Datasets 2.14.5
Tokenizers 0.13.3

📄 License

This model is licensed under the Apache - 2.0 license.

Property	Details
Model Type	Fine - tuned model based on google/t5 - efficient - tiny for summarization
Training Data	billsum
Metrics	rouge

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご