T5 Small Finetuned Summarization Xsum

🚀 t5-small-finetuned-summarization-xsum

該模型基於t5-small在xsum數據集上微調而來，具備快速、輕量的特點，能高效完成文本摘要任務。

🚀 快速開始

此模型是 t5-small 在 xsum 數據集上的微調版本，速度極快且輕量，能在不到 1 秒的時間內完成整個文本的摘要，在低資源使用場景下表現出色。

模型演示

點擊此處查看模型演示

該模型在評估集上取得了以下結果：

• 損失值：2.2425
• Rouge1：31.3222
• Rouge2：10.0614
• Rougel：25.0513
• Rougelsum：25.0446
• 生成長度：18.8044

✨ 主要特性

• 輕量且速度極快，無論在 GPU 還是 CPU 上，都能在不到 1 秒的時間內完成文本摘要。若使用 optimum，速度可能更快。

📦 安裝指南

使用該模型前，請確保安裝必要的庫：

pip install transformers accelerate optimum onnxruntime onnx

💻 使用示例

基礎用法

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, pipeline

model_id = "Rahmat82/t5-small-finetuned-summarization-xsum"

model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
summarizer = pipeline("summarization", model=model, tokenizer=tokenizer)

text_to_summarize = """
The koala is regarded as the epitome of cuddliness. However, animal lovers
will be saddened to hear that this lovable marsupial has been moved to the
endangered species list. The Australian Koala Foundation estimates there are
somewhere between 43,000-100,000 koalas left in the wild. Their numbers have
been dwindling rapidly due to disease, loss of habitat, bushfires, being hit
by cars, and other threats. Stuart Blanch from the World Wildlife Fund in
Australia said: "Koalas have gone from no listing to vulnerable to endangered
within a decade. That is a shockingly fast decline." He added that koalas risk
"sliding toward extinction" 
"""

print(summarizer(text_to_summarize)[0]["summary_text"])

高級用法

使用 optimum/onnxruntime 實現超快速摘要：

#!pip install -q transformers accelerate optimum onnxruntime onnx

from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSeq2SeqLM
from optimum.pipelines import pipeline
import accelerate

model_name = "Rahmat82/t5-small-finetuned-summarization-xsum"

model = ORTModelForSeq2SeqLM.from_pretrained(model_name, export=True)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True)
summarizer = pipeline("summarization", model=model, tokenizer=tokenizer,
                                       device_map="auto", batch_size=12)

text_to_summarize = """
The koala is regarded as the epitome of cuddliness. However, animal lovers
will be saddened to hear that this lovable marsupial has been moved to the
endangered species list. The Australian Koala Foundation estimates there are
somewhere between 43,000-100,000 koalas left in the wild. Their numbers have
been dwindling rapidly due to disease, loss of habitat, bushfires, being hit
by cars, and other threats. Stuart Blanch from the World Wildlife Fund in
Australia said: "Koalas have gone from no listing to vulnerable to endangered
within a decade. That is a shockingly fast decline." He added that koalas risk
"sliding toward extinction" 
"""

print(summarizer(text_to_summarize)[0]["summary_text"])

📚 詳細文檔

訓練超參數

訓練過程中使用了以下超參數：

• 學習率：0.0002
• 訓練批次大小：28
• 評估批次大小：28
• 隨機種子：42
• 優化器：Adam（β=(0.9, 0.999)，ε=1e-08）
• 學習率調度器類型：線性
• 訓練輪數：2
• 混合精度訓練：Native AMP

訓練結果

框架版本

• Transformers 4.37.0
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1

📄 許可證

本模型使用 Apache-2.0 許可證。