Sasando-1-25M Open-Source Indonesian Text Generator - Free Deployment for Low-Resource Language Scenarios

Sasando 1 25M

Developed by afrizalha

Sasando-1 is a miniature experimental Indonesian text generator based on the Phi-3 architecture, available in two versions with 7 million and 25 million parameters, focusing on generation capabilities in low-resource language scenarios.

Large Language Model

Transformers

OtherOpen Source License:MIT #Indonesian text generation #Low-resource optimization #Miniature language model

Downloads 47

Release Time : 6/9/2024

Model Overview

This model is a miniature text generator specifically designed for Indonesian, trained with strictly filtered datasets, aiming to explore the possibilities of generative AI in low-resource language environments.

Model Features

Low-resource optimization

Optimized for the scarcity of Indonesian language resources, trained using strictly filtered small-scale datasets

Miniature architecture

Offers two lightweight versions with 7 million and 25 million parameters, suitable for resource-constrained environments

Language preservation

Explores the potential applications of generative AI in preserving Indonesia's multilingual heritage

Model Capabilities

Indonesian text generation

Short text generation

Use Cases

Language research

Low-resource language model research

Used to study the performance of language models with limited data and computational resources

Demonstrates that miniature language models can still generate fluent text with strictly controlled training data

Education

Indonesian language learning aid

Can serve as a text generation reference tool for Indonesian language learners

🚀 Descaling low-resource language models

Sasando-1 is a tiny, highly experimental short-sequence text generator built using the Phi-3 architecture, aiming to leverage limited resources for generative model development.

🚀 Quick Start

You can directly access the Gradio demo of Sasando-1 by clicking the following link: ❕Go straight to the gradio demo❕

✨ Features

Tiny Model Variations: Sasando-1 comes in two microscopic sizes with 7M and 25M parameters.
Phi-3 Architecture: It is built using the Phi - 3 architecture.
Limited Vocabulary Training: Trained on a tightly - controlled Indo4B dataset with only 18000 unique words.
Token Training: Trained on approximately 257M tokens over 4 epochs.

📚 Documentation

🎻 Welcome!

Sasando-1 is a tiny, highly experimental Indonesian text generator built using the Phi-3 architecture. It comes with two variations of microscopic sizes: 7M and 25M parameters. It is trained on a tightly - controlled Indo4B dataset filtered to only have 18000 unique words. The method is inspired by Microsoft's TinyStories paper which demonstrates that a tiny language model can produce fluent text when trained on a tightly - controlled dataset.

🇮🇩 Context

Indonesia has +700 languages, and many of them are dying at an alarming rate. Language technologies like generative AI can play a massive role in language preservation. However, Indonesia has several contextual issues:

Many languages, including those with millions of speakers, have low - volume digital resources.
Running large models can be costly, while Indonesia is a middle - income country with little funding.

Overcoming these challenges require developers to work with what little data and money that they have. Sasando-1 is a prototypical demonstration that thinly - available resources can potentially still be leveraged to develop generative models with cheap compute.

🔭 Out-of-Scope Use

This is a research preview base model. It is not intruction - tuned and has minimal safety curation. It is not intended for commercial or practical applications.

You are also not allowed to use this model without having fun.

📄 License

This project is licensed under the MIT License.

Acknowledgments

Developed by: Afrizal Hasbi Azizy
License: MIT

Training log

Sasando-1 is a tiny, highly experimental short-sequence text generator built using the Phi-3 architecture.

This repo contains the 25M version

Preliminary research preview

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご