T5 Small Finetuned Summarization Xsum

🚀 t5-small-finetuned-summarization-xsum

该模型基于t5-small在xsum数据集上微调而来，具备快速、轻量的特点，能高效完成文本摘要任务。

🚀 快速开始

此模型是 t5-small 在 xsum 数据集上的微调版本，速度极快且轻量，能在不到 1 秒的时间内完成整个文本的摘要，在低资源使用场景下表现出色。

模型演示

点击此处查看模型演示

该模型在评估集上取得了以下结果：

• 损失值：2.2425
• Rouge1：31.3222
• Rouge2：10.0614
• Rougel：25.0513
• Rougelsum：25.0446
• 生成长度：18.8044

✨ 主要特性

• 轻量且速度极快，无论在 GPU 还是 CPU 上，都能在不到 1 秒的时间内完成文本摘要。若使用 optimum，速度可能更快。

📦 安装指南

使用该模型前，请确保安装必要的库：

pip install transformers accelerate optimum onnxruntime onnx

💻 使用示例

基础用法

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline

model_id = "Rahmat82/t5-small-finetuned-summarization-xsum"

model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
summarizer = pipeline("summarization", model=model, tokenizer=tokenizer)

text_to_summarize = """
The koala is regarded as the epitome of cuddliness. However, animal lovers
will be saddened to hear that this lovable marsupial has been moved to the
endangered species list. The Australian Koala Foundation estimates there are
somewhere between 43,000-100,000 koalas left in the wild. Their numbers have
been dwindling rapidly due to disease, loss of habitat, bushfires, being hit
by cars, and other threats. Stuart Blanch from the World Wildlife Fund in
Australia said: "Koalas have gone from no listing to vulnerable to endangered
within a decade. That is a shockingly fast decline." He added that koalas risk
"sliding toward extinction" 
"""

print(summarizer(text_to_summarize)[0]["summary_text"])

高级用法

使用 optimum/onnxruntime 实现超快速摘要：

#!pip install -q transformers accelerate optimum onnxruntime onnx

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM
from optimum.pipelines import pipeline
import accelerate

model_name = "Rahmat82/t5-small-finetuned-summarization-xsum"

model = ORTModelForSeq2SeqLM.from_pretrained(model_name, export=True)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
summarizer = pipeline("summarization", model=model, tokenizer=tokenizer,
                                       device_map="auto", batch_size=12)

text_to_summarize = """
The koala is regarded as the epitome of cuddliness. However, animal lovers
will be saddened to hear that this lovable marsupial has been moved to the
endangered species list. The Australian Koala Foundation estimates there are
somewhere between 43,000-100,000 koalas left in the wild. Their numbers have
been dwindling rapidly due to disease, loss of habitat, bushfires, being hit
by cars, and other threats. Stuart Blanch from the World Wildlife Fund in
Australia said: "Koalas have gone from no listing to vulnerable to endangered
within a decade. That is a shockingly fast decline." He added that koalas risk
"sliding toward extinction" 
"""

print(summarizer(text_to_summarize)[0]["summary_text"])

📚 详细文档

训练超参数

训练过程中使用了以下超参数：

• 学习率：0.0002
• 训练批次大小：28
• 评估批次大小：28
• 随机种子：42
• 优化器：Adam（β=(0.9, 0.999)，ε=1e-08）
• 学习率调度器类型：线性
• 训练轮数：2
• 混合精度训练：Native AMP

训练结果

框架版本

• Transformers 4.37.0
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1

📄 许可证

本模型使用 Apache-2.0 许可证。