Sinhala Gpt2

🚀 Sinhala GPT2 Text Generation Model

This is a smaller GPT2 model trained on Sinhala dataset, suitable for starting training on more downstream tasks for the low - resource Sinhala language.

🚀 Quick Start

This is a smaller GPT2 model trained on the MC4 Sinhala dataset. Since Sinhala is a low - resource language, only a few models have been trained. Thus, it serves as an excellent starting point for training on more downstream tasks.

✨ Features

• Trained on the Sinhala dataset of MC4, which is beneficial for Sinhala text - generation tasks.
• Ideal for initiating training on downstream tasks in the low - resource Sinhala language.

📦 Installation

No specific installation steps are provided in the original README.

💻 Usage Examples

Basic Usage

from transformers import pipeline
generator = pipeline('text-generation', model='keshan/sinhala-gpt2')
generator("මම", max_length=50, num_return_sequences=5)

📚 Documentation

Model Specification

The model chosen for training is GPT2 with the following specifications:

1. vocab_size = 50257
2. n_embd = 768
3. n_head = 12
4. n_layer = 12
5. n_positions = 1024