Pegasus X Large Book Summary

🚀 pszemraj/pegasus-x-large-book-summary

該模型基於Transformer架構，專為長文檔摘要任務設計，能有效處理長序列文本，在多個數據集上取得了良好的ROUGE指標成績，可用於地震、學術論文、講座等多種文本的摘要生成。

🚀 快速開始

本模型可用於多種文本的摘要生成任務，如地震相關文本、學術論文、講座轉錄文本等。以下是使用示例：

# 假設使用Hugging Face的transformers庫
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# 加載模型和分詞器
tokenizer = AutoTokenizer.from_pretrained("pszemraj/pegasus-x-large-book-summary")
model = AutoModelForSeq2SeqLM.from_pretrained("pszemraj/pegasus-x-large-book-summary")

# 示例文本
text = "large earthquakes along a given fault segment do not occur at random intervals because it takes time to accumulate the strain energy for the rupture. The rates at which tectonic plates move and accumulate strain at their boundaries are approximately uniform. Therefore, in first approximation, one may expect that large ruptures of the same fault segment will occur at approximately constant time intervals. If subsequent main shocks have different amounts of slip across the fault, then the recurrence time may vary, and the basic idea of periodic mainshocks must be modified. For great plate boundary ruptures the length and slip often vary by a factor of 2. Along the southern segment of the San Andreas fault the recurrence interval is 145 years with variations of several decades. The smaller the standard deviation of the average recurrence interval, the more specific could be the long term prediction of a future mainshock."

# 對文本進行分詞
inputs = tokenizer(text, return_tensors="pt")

# 生成摘要
summary_ids = model.generate(inputs["input_ids"], num_beams=2, early_stopping=True, length_penalty=0.1)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print(summary)

✨ 主要特性

• 處理長序列文本：能夠處理長度較長的文本序列，如支持長度達到4096的序列，相比傳統基於Transformer的模型，在處理長序列時計算成本更低。
• 高效注意力機制：採用塊稀疏注意力（block sparse attention）代替普通注意力機制，有效降低了計算複雜度。
• 多任務適用性：適用於多種自然語言處理任務，特別是長文檔摘要任務。

📚 詳細文檔

模型參數

數據集與指標

模型在多個數據集上進行了測試，以下是部分數據集的測試結果：

samsum數據集

launch/gov_report數據集

billsum數據集

📄 許可證

本模型使用的許可證包括：

• Apache - 2.0
• BSD - 3 - Clause