BERT2GPT Indonesian Summarization Open-Source Model - Efficiently Generate Indonesian Text Summaries

Bert2gpt Indonesian Summarization

Developed by cahya

An encoder-decoder model for Indonesian text summarization fine-tuned based on BERT-base and GPT2-small, suitable for generating summaries of Indonesian texts.

Text Generation

Transformers

OtherOpen Source License:Apache-2.0 #Indonesian summarization #BERT2GPT architecture #News summarization

Downloads 197

Release Time : 3/2/2022

Model Overview

This model is a text summarization model with an encoder-decoder structure, using BERT as the encoder and GPT2 as the decoder, specifically fine-tuned for Indonesian texts to generate high-quality summaries.

Model Features

Indonesian optimization

Specifically fine-tuned for Indonesian texts, enabling better understanding and generation of Indonesian summaries.

Encoder-decoder structure

Combines BERT's powerful encoding capabilities with GPT2's fluent generation abilities to achieve high-quality text summarization.

Pre-trained model fine-tuning

Fine-tuned based on pre-trained BERT and GPT2 models, fully leveraging large-scale pre-trained language understanding capabilities.

Model Capabilities

Text summarization

Indonesian text processing

Text generation

Use Cases

News summarization

News article summarization

Automatically generates concise summaries of news articles to help readers quickly grasp the main content.

Produces concise and accurate news summaries

Document processing

Long document summarization

Generates key-point summaries for long documents to improve reading efficiency.

Extracts core content of documents

🚀 Indonesian BERT2BERT Summarization Model

A finetuned EncoderDecoder model using BERT-base and GPT2-small for Indonesian text summarization.

🚀 Quick Start

This model is designed to perform text summarization for Indonesian texts. It is based on the EncoderDecoder architecture, leveraging BERT-base and GPT2-small.

✨ Features

Finetuned for Indonesian: Specifically tailored for Indonesian text summarization.
Based on Popular Models: Utilizes cahya/bert-base-indonesian-1.5G and cahya/gpt2-small-indonesian-522M.

📦 Installation

There is no specific installation step provided in the original README. If you want to use this model, you need to have the transformers library installed. You can install it using the following command:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import BertTokenizer, EncoderDecoderModel

tokenizer = BertTokenizer.from_pretrained("cahya/bert2gpt-indonesian-summarization")
tokenizer.bos_token = tokenizer.cls_token
tokenizer.eos_token = tokenizer.sep_token
model = EncoderDecoderModel.from_pretrained("cahya/bert2gpt-indonesian-summarization")

Advanced Usage

from transformers import BertTokenizer, EncoderDecoderModel

tokenizer = BertTokenizer.from_pretrained("cahya/bert2gpt-indonesian-summarization")
tokenizer.bos_token = tokenizer.cls_token
tokenizer.eos_token = tokenizer.sep_token
model = EncoderDecoderModel.from_pretrained("cahya/bert2gpt-indonesian-summarization")

# 
ARTICLE_TO_SUMMARIZE = ""

# generate summary
input_ids = tokenizer.encode(ARTICLE_TO_SUMMARIZE, return_tensors='pt')
summary_ids = model.generate(input_ids,
            min_length=20,
            max_length=80, 
            num_beams=10,
            repetition_penalty=2.5, 
            length_penalty=1.0, 
            early_stopping=True,
            no_repeat_ngram_size=2,
            use_cache=True,
            do_sample = True,
            temperature = 0.8,
            top_k = 50,
            top_p = 0.95)

summary_text = tokenizer.decode(summary_ids[0], skip_special_tokens=True)
print(summary_text)

📚 Documentation

Finetuning Corpus

The bert2gpt-indonesian-summarization model is based on cahya/bert-base-indonesian-1.5G and cahya/gpt2-small-indonesian-522M by cahya, finetuned using the id_liputan6 dataset.

📄 License

This project is licensed under the Apache 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご