Bert2Bert_shared-spanish-finetuned-summarization Open-source Model - Free Automatic Summarization of Spanish Texts

Bert2bert Shared Spanish Finetuned Summarization

Developed by mrm8488

This is a text summarization model fine-tuned on the Spanish BERT model (BETO), specifically designed for automatic summarization of Spanish texts.

Text Generation

Transformers

Spanish#Spanish text summarization #News text processing #BERT2BERT architecture

Downloads 3,185

Release Time : 3/2/2022

Model Overview

The model adopts the BERT2BERT architecture, based on the dccuchile/bert-base-spanish-wwm-cased pre-trained model, fine-tuned on the MLSUM Spanish summarization dataset, capable of generating high-quality Spanish text summaries.

Model Features

Spanish language optimization

Fine-tuned on the Spanish BERT model (BETO), specifically optimized for Spanish text summarization tasks

Large-scale dataset training

Trained on the MLSUM Spanish summarization dataset, containing a large number of news and summary paired samples

Shared-weight architecture

Adopts the BERT2BERT shared-weight architecture, where the encoder and decoder use the same pre-trained model

Model Capabilities

Spanish text summarization generation

Automatic news content summarization

Long text compression

Use Cases

News media

News summarization generation

Automatically generates concise summaries of Spanish news articles

Rouge2 F-score reaches 8.7

Content management

Document summarization

Generates key point summaries for long Spanish documents

🚀 Spanish BERT2BERT (BETO) fine-tuned on MLSUM ES for summarization

This project presents a Spanish BERT2BERT (BETO) model fine - tuned on the MLSUM ES dataset for text summarization. It offers an effective solution for summarizing Spanish news articles, leveraging a large - scale multilingual dataset.

📚 Documentation

Model

The base model used is dccuchile/bert-base-spanish-wwm-cased (BERT Checkpoint). This pre - trained model serves as the foundation for the fine - tuning process on the summarization task.

Dataset

MLSUM is the first large - scale MultiLingual SUMmarization dataset. It is sourced from online newspapers and contains over 1.5 million article/summary pairs in five different languages: French, German, Spanish, Russian, and Turkish. When combined with English newspapers from the popular CNN/Daily mail dataset, it forms a large - scale multilingual dataset that can open up new research directions for the text summarization community. Cross - lingual comparative analyses based on state - of - the - art systems have been reported, highlighting existing biases that justify the use of a multilingual dataset.

MLSUM es

Results

The following table shows the performance metrics of the model on the test set:

Property	Details
Test - Rouge2 - mid - precision	9.6
Test - Rouge2 - mid - recall	8.4
Test - Rouge2 - mid - fmeasure	8.7
Test - Rouge1	26.24
Test - Rouge2	8.9
Test - RougeL	21.01
Test - RougeLsum	21.02

💻 Usage Examples

Basic Usage

import torch
from transformers import BertTokenizerFast, EncoderDecoderModel
device = 'cuda' if torch.cuda.is_available() else 'cpu'
ckpt = 'mrm8488/bert2bert_shared-spanish-finetuned-summarization'
tokenizer = BertTokenizerFast.from_pretrained(ckpt)
model = EncoderDecoderModel.from_pretrained(ckpt).to(device)

def generate_summary(text):

    inputs = tokenizer([text], padding="max_length", truncation=True, max_length=512, return_tensors="pt")
    input_ids = inputs.input_ids.to(device)
    attention_mask = inputs.attention_mask.to(device)
    output = model.generate(input_ids, attention_mask=attention_mask)
    return tokenizer.decode(output[0], skip_special_tokens=True)
    
text = "Your text here..."
generate_summary(text)

Created by Manuel Romero/@mrm8488 with the support of Narrativa

Made with ♥ in Spain

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご