Indobert-lite-large-p1 Open-Source Indonesian Language Model - Facilitating Indonesian Text Processing and Comprehension

Indobert Lite Large P1

Developed by indobenchmark

IndoBERT is an advanced language model for Indonesian, based on the BERT architecture, trained using masked language modeling and next sentence prediction objectives.

Large Language Model

Transformers

OtherOpen Source License:MIT #Indonesian-specific #Lightweight BERT #Case-insensitive

Downloads 42

Release Time : 3/2/2022

Model Overview

IndoBERT is a pre-trained language model specifically designed for Indonesian, primarily used for natural language understanding tasks. The model is trained on a large-scale Indonesian corpus and can effectively handle semantic understanding tasks for Indonesian text.

Model Features

Optimized for Indonesian

Specially designed and trained for Indonesian, enabling better understanding and processing of Indonesian text.

Lightweight Model

Compared to the full version of IndoBERT, the Lite version has fewer parameters, making it suitable for resource-limited environments.

Large-scale Training Data

Trained using the Indo4B dataset (23.43GB of Indonesian text).

Model Capabilities

Indonesian text understanding

Masked language modeling

Next sentence prediction

Use Cases

Natural Language Processing

Text Classification

Classifying Indonesian text

Named Entity Recognition

Identifying named entities in Indonesian text

🚀 IndoBERT-Lite Large Model (phase1 - uncased)

IndoBERT is a state - of - the - art language model for Indonesian based on the BERT model. The pretrained model is trained using a masked language modeling (MLM) objective and next sentence prediction (NSP) objective. It offers high - quality language understanding capabilities for the Indonesian language.

✨ Features

Based on the BERT architecture, providing a strong foundation for natural language processing tasks in Indonesian.
Trained with MLM and NSP objectives to enhance language understanding.

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

from transformers import BertTokenizer, AutoModel
tokenizer = BertTokenizer.from_pretrained("indobenchmark/indobert-lite-large-p1")
model = AutoModel.from_pretrained("indobenchmark/indobert-lite-large-p1")

Advanced Usage

import torch
x = torch.LongTensor(tokenizer.encode('aku adalah anak [MASK]')).view(1,-1)
print(x, model(x)[0].sum())

📚 Documentation

All Pre - trained Models

Property	Details
Model Type	`indobenchmark/indobert-base-p1`, `indobenchmark/indobert-base-p2`, `indobenchmark/indobert-large-p1`, `indobenchmark/indobert-large-p2`, `indobenchmark/indobert-lite-base-p1`, `indobenchmark/indobert-lite-base-p2`, `indobenchmark/indobert-lite-large-p1`, `indobenchmark/indobert-lite-large-p2`
Training Data	Indo4B (23.43 GB of text)
#params	Varies from 11.7M to 335.2M depending on the model
Arch.	Base or Large

👥 Authors

IndoBERT was trained and evaluated by Bryan Wilie*, Karissa Vincentio*, Genta Indra Winata*, Samuel Cahyawijaya*, Xiaohong Li, Zhi Yuan Lim, Sidik Soleman, Rahmad Mahendra, Pascale Fung, Syafri Bahar, Ayu Purwarianti.

📄 License

This project is licensed under the MIT license.

📖 Citation

If you use our work, please cite:

@inproceedings{wilie2020indonlu,
  title={IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding},
  author={Bryan Wilie and Karissa Vincentio and Genta Indra Winata and Samuel Cahyawijaya and X. Li and Zhi Yuan Lim and S. Soleman and R. Mahendra and Pascale Fung and Syafri Bahar and A. Purwarianti},
  booktitle={Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing},
  year={2020}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご