Amazon MultiLingual Review Summarization With Google Mt5 Small

🚀 Amazon MultiLingual Review Summarization with google mT5 Small

This model is designed for multilingual product review summarization, specifically fine - tuned from google/mt5-small on a multi - lingual Amazon reviews dataset. It offers efficient summarization for English and German reviews, with promising evaluation results.

🚀 Quick Start

This model is a fine - tuned version of google/mt5-small on an Multi Lingual Amazon Reviews dataset. It achieves the following results on the evaluation set:

• Loss: 2.9368
• Model Preparation Time: 0.0038
• Rouge1: 16.1955
• Rouge2: 8.1292
• Rougel: 15.9218
• Rougelsum: 15.9516

✨ Features

• Multilingual Support: Capable of summarizing English and German product reviews.
• Fine - Tuned: Based on the google/mt5-small model, fine - tuned for better performance on Amazon reviews.
• Evaluated Metrics: Provides multiple evaluation metrics such as Rouge scores and loss values.

📚 Documentation

Model description

google/mt5-small

Intended uses & limitations

Multilingual Product Review Summarization. Supported Languages: English and German

Training and evaluation data

The original multi - lingual Amazon product reviews dataset available on HuggingFace is defunct. So, we use the version available at Kaggle. The original dataset supports 6 languages: English, German, French, Spanish, Japanese, and Chamorro. Each language has 20,000 training samples, 5,000 validation samples, and 5,000 testing samples. We upload this dataset to HuggingFace hub at [srvmishra832/multilingual - amazon - reviews - 6 - languages](https://huggingface.co/datasets/srvmishra832/multilingual - amazon - reviews - 6 - languages) Here, we only select the English and German language reviews for the pc and electronics product categories. We use the review titles as summaries, and to prevent the model from generating very small summaries, we filter out those examples with extremely short review titles. Finally, we downsample the resulting dataset so that training is feasible on the Google colab T4 GPU in a reasonable amount of time. The final downsampled and concatenated dataset contains 8,000 training samples, 452 validation samples, and 422 test samples.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e - 05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon = 1e - 08 and optimizer_args = No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Framework versions

• Transformers 4.50.0
• Pytorch 2.6.0+cu124
• Datasets 3.4.1
• Tokenizers 0.21.1

📄 License

This project is licensed under the Apache - 2.0 license.