Bodo Roberta Base

🚀 Bodo RoBERTa Base Configuration Model

A RoBERTa-based configuration model for the Bodo language, offering a Byte Level BPE Tokenizer and RoBERTa base configuration.

🚀 Quick Start

This is a RoBERTa-based configuration model tailored for the Bodo language. It doesn't include checkpoints for the pre-trained model. Instead, it provides two key components:

• A Byte Level BPE Tokenizer for Bodo
• RoBERTa base configuration

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('alayaran/bodo-roberta-base')

t = tokenizer('कौटि नख\'राव दैनि कानेक्सन होबाय')

# {'input_ids': [310, 294, 313, 267, 503, 11, 268, 263, 277, 298, 287, 265, 267, 321, 263, 265, 272, 310, 273, 378, 295, 266, 271, 263, 269], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
# to decode

tokenizer.decode(t['input_ids'],skip_special_tokens=True)

# "कौटि नख'राव दैनि कानेक्सन होबाय"

📄 License

This project is licensed under the MIT License.