Indo Roberta Indonli

🚀 Indo-roberta-indonli

Indo-roberta-indonli is a natural language inference classifier based on the Indo-roberta model. It addresses the need for accurate natural language inference in the Indonesian context, leveraging the pre - trained Indo - roberta model and fine - tuning it on relevant datasets.

🚀 Quick Start

Indo-roberta-indonli is a powerful tool for natural language inference. It uses the Indo-roberta model and is fine - tuned on the IndoNLI dataset. The model was tested on validation, test_layer, and test_expert datasets from the GitHub repository, and the results are presented below.

✨ Features

• Based on Indo - roberta: Utilizes the pre - trained Indo - roberta model for effective transfer learning.
• Trained on IndoNLI: Fine - tuned on the IndoNLI dataset for natural language inference tasks.

📦 Installation

No specific installation steps are provided in the original document, so this section is skipped.

💻 Usage Examples

Basic Usage

from transformers import pipeline
pretrained_name = "StevenLimcorn/indonesian-roberta-indonli"
nlp = pipeline(
    "zero-shot-classification",
    model=pretrained_name,
    tokenizer=pretrained_name
)
nlp("Amir Sjarifoeddin Harahap lahir di Kota Medan, Sumatera Utara, 27 April 1907. Ia meninggal di Surakarta, Jawa Tengah, pada 19 Desember 1948 dalam usia 41 tahun. </s></s> Amir Sjarifoeddin Harahap masih hidup.")

📚 Documentation

Result

Model

The model was trained for 5 epochs, with a batch size of 16, a learning rate of 2e - 5, and weight decay of 0.01. The achieved metrics are as follows:

🔧 Technical Details

The model is a transfer - learned natural language inference classifier. It starts with the Indo - roberta model and is fine - tuned on the IndoNLI dataset. The training process involves 5 epochs, a batch size of 16, a learning rate of 2e - 5, and weight decay of 0.01.

📄 License

The model is released under the MIT license.

📖 Reference

The dataset used is from IndoNLI.

@inproceedings{indonli,
    title = "IndoNLI: A Natural Language Inference Dataset for Indonesian",
    author = "Mahendra, Rahmad and Aji, Alham Fikri and Louvan, Samuel and Rahman, Fahrurrozi and Vania, Clara",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    publisher = "Association for Computational Linguistics",
}

⚠️ Important Note

Consider the biases that may come from both the pre - trained RoBERTa model and the INDONLI dataset, as they may affect the model's results.

👨‍💻 Author

Indonesian RoBERTa Base IndoNLI was trained and evaluated by Steven Limcorn. All computation and development were done on Google Colaboratory using their free GPU access.