T5 Base Standard Bahasa Cased

🚀 t5-base-standard-bahasa-cased

A pre-trained T5 base standard language model for Malay, designed to handle a variety of language tasks.

🚀 Quick Start

This t5-base-standard-bahasa-cased model is a pre-trained T5 base standard language model tailored for Malay. It has been trained on multiple tasks to offer diverse language capabilities.

✨ Features

The model was pre-trained on the following tasks:

1. Language masking task on Malay news, Malay Wikipedia, Malay Academia.edu, Malay parliament, and translated The Pile.
2. News title prediction on Malay news.
3. Next sentence prediction on Malay news, Malay Wikipedia, Malay Academia.edu, Malay parliament, and translated The Pile.
4. Translated QA Natural.
5. Text Similarity task on translated SNLI and translated MNLI.
6. EN-MS translation.
7. MS-EN translation.
8. Abstractive Summarization.
9. Knowledge Graph triples generation.
10. Paraphrase.

The preparation steps can be reproduced at https://github.com/huseinzol05/malaya/tree/master/pretrained-model/t5/prepare.

📚 Documentation

Pretraining details

• This model was trained using the Google T5 repository https://github.com/google-research/text-to-text-transfer-transformer on a v3 - 8 TPU.
• All steps can be reproduced from https://github.com/huseinzol05/Malaya/tree/master/pretrained-model/t5.

Supported prefix

1. soalan: {string}: Trained using Natural QA.
2. ringkasan: {string}: For abstractive summarization.
3. tajuk: {string}: For abstractive title.
4. parafrasa: {string}: For abstractive paraphrase.
5. terjemah Inggeris ke Melayu: {string}: For EN - MS translation.
6. terjemah Melayu ke Inggeris: {string}: For MS - EN translation.
7. grafik pengetahuan: {string}: For Malay text to English Knowledge Graph triples format.
8. ayat1: {string1} ayat2: {string2}: Semantic similarity.