Pegasus X Thai Sum

🚀 Pegasus X Thai Text Summarization Model

This repository presents a fine - tuned Pegasus X model for summarizing Thai text. The model is built upon the Pegasus X architecture, offering an effective solution for text summarization tasks in the Thai language.

🚀 Quick Start

To get started with this model, you first need to install the necessary library.

📦 Installation

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import PegasusXForConditionalGeneration, AutoTokenizer

model = PegasusXForConditionalGeneration.from_pretrained("satjawat/pegasus-x-thai-sum")
tokenizer = AutoTokenizer.from_pretrained("satjawat/pegasus-x-thai-sum")

new_input_string = "ข้อความ"
new_input_ids = tokenizer(new_input_string.lower(), return_tensors="pt").input_ids
summary_ids = model.generate(new_input_ids, max_length=50, num_beams=6, length_penalty=2.0, early_stopping=True)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print("Input:", new_input_string)
print("Generated Summary:", summary)

📚 Documentation

Training Hyperparameters

The following hyperparameters were employed during the model training:

• accumulation_steps: 2
• num_epochs: 20
• num_beams: 6
• learning_rate: lr = 5e - 5
• optimizer: AdamW with betas=(0.9, 0.999) and epsilon = 1e - 08
• activation_function: gelu
• add_bias_logits: True
• normalize_embedding: True
• add_final_layer_norm: False
• normalize_before: False

Score

The model was evaluated using the test dataset of ThaiSum, which includes a total of 11,000 articles. The evaluation scores are as follows:

• Rouge1: 0.490279
• Rouge2: 0.289839
• Rougel: 0.489334

Resource Funding

We would like to express our gratitude to the NSTDA Supercomputer center (ThaiSC) and the National e - Science Infrastructure Consortium for their support in providing computer facilities.

Citation

If you use "satjawat/pegasus - x - thai - sum" in your project or publication, please cite the model as follows:

ปรีชานนท์ ชาติไทย และ สัจจวัจน์ ส่งเสริม. (2567),
การสรุปข้อความข่าวภาษาไทยด้วยโครงข่ายประสาทเทียม (Thai News Text Summarization Using Neural Network),
วิทยาศาสตรบัณฑิต (วทบ.):ขอนแก่น, มหาวิทยาลัยขอนแก่น