Mlong T5 Large Sumstew

🚀 mLong-T5-large-sumstew

mLong-T5-large-sumstew 是一個多語言、長文本（支持最多 16k 輸入標記）的抽象摘要模型。該模型在 sumstew 數據集上進行訓練，能夠為給定的輸入文檔生成標題和摘要。

🚀 快速開始

✨ 主要特性

• 多語言支持：支持英語、德語、法語、意大利語和西班牙語等多種語言。
• 長文本處理：能夠處理最多 16k 輸入標記的長文本。
• 生成標題和摘要：可以為輸入文檔同時生成標題和摘要。

📦 安裝指南

文檔未提供安裝步驟，跳過該章節。

💻 使用示例

基礎用法

使用 pipeline 進行摘要生成，這種方式簡單易用：

from transformers import pipeline

summarizer = pipeline("summarization", "joemgu/mlong-t5-large-sumstew")
text = "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, 'and what is the use of a book,' thought Alice 'without pictures or conversations?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her. There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, 'Oh dear! Oh dear! I shall be late!' (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge. In another moment down went Alice after it, never once considering how in the world she was to get out again."
summary = summarizer(text)[0]["summary_text"]
print(summary)

輸出結果：

Title: Alice and the White Rabbit Summary: Alice is a bored and curious girl who follows a White Rabbit with a watch into a rabbit-hole. She enters a strange world where she has many adventures and meets many peculiar creatures.

高級用法

使用 .from_pretrained 方法進行更精細的控制：

from transformers import LongT5ForConditionalGeneration, T5Tokenizer

checkpoint = "joemgu/mlong-t5-large-sumstew"

gen_kwargs = {
    "max_length": 1024,
    "do_sample": False,
    "num_beams": 4, # higher = better, but uses more memory
    "use_cache": True, # Set to False if running out of memory, but will be MUCH slower
    "early_stopping": True,
    "num_return_sequences": 1,
    "repetition_penalty": 3.5,
    "encoder_repetition_penalty": 2.0,
    "length_penalty": 1.0, # higher = longer summaries
    "encoder_no_repeat_ngram_size": 4,
    "no_repeat_ngram_size": 6,
}

model = LongT5ForConditionalGeneration.from_pretrained(checkpoint)
tokenizer = T5Tokenizer.from_pretrained(checkpoint)

prefix = "Write a title and summarize: "
input_document = "Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, 'and what is the use of a book,' thought Alice 'without pictures or conversations?' So she was considering in her own mind (as well as she could, for the hot day made her feel very sleepy and stupid), whether the pleasure of making a daisy-chain would be worth the trouble of getting up and picking the daisies, when suddenly a White Rabbit with pink eyes ran close by her. There was nothing so very remarkable in that; nor did Alice think it so very much out of the way to hear the Rabbit say to itself, 'Oh dear! Oh dear! I shall be late!' (when she thought it over afterwards, it occurred to her that she ought to have wondered at this, but at the time it all seemed quite natural); but when the Rabbit actually took a watch out of its waistcoat-pocket, and looked at it, and then hurried on, Alice started to her feet, for it flashed across her mind that she had never before seen a rabbit with either a waistcoat-pocket, or a watch to take out of it, and burning with curiosity, she ran across the field after it, and fortunately was just in time to see it pop down a large rabbit-hole under the hedge. In another moment down went Alice after it, never once considering how in the world she was to get out again."

inputs = tokenizer(prefix + input_document, return_tensors="pt", max_length=16384, truncation=True, add_special_tokens=True)

outputs = model.generate(**inputs, **gen_kwargs)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

在輸入文檔前添加前綴，根據不同的前綴，輸出結果會有所不同：

• 若前綴為 "Summarize: " + INPUT_TEXT，輸出為 "Summary: SUMMARY OF INPUT_TEXT"。
• 若前綴為 "Write a title and summarize: " + INPUT_TEXT，輸出為 "Title: TITLE OF INPUT_TEXT Summary: SUMMARY OF INPUT_TEXT"。

📚 詳細文檔

🔧 技術細節

文檔未提供具體的技術實現細節，跳過該章節。

📄 許可證

本項目採用 Apache-2.0 許可證。