Bert Base Buddhist Sanskrit

🚀 bert-base-buddhist-sanskrit

The best performing model for Buddhist Sanskrit embeddings, based on BERT architecture.

🚀 Quick Start

The bert-base-buddhist-sanskrit model is the top - performer in the research presented in the paper 'Embeddings models for Buddhist Sanskrit' published at LREC 2022. (The link to the paper will be added after the publication of conference proceedings).

✨ Features

• The model adopts the bert - base architecture and configuration.
• It was pretrained from scratch as a masked language model on the Sanskrit reference corpus and fine - tuned on a smaller corpus of Buddhist Sanskrit.

💻 Usage Examples

Basic Usage

model = AutoModelForMaskedLM.from_pretrained("Matej/bert-base-buddhist-sanskrit")
tokenizer = AutoTokenizer.from_pretrained("Matej/bert-base-buddhist-sanskrit", use_fast=True)

📚 Documentation

Intended uses & limitations

The model is released under the MIT license and has no limitations.

Training and evaluation data

For details on the corpora and the evaluation procedure, please refer to the paper 'Embeddings models for Buddhist Sanskrit'.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e - 05
• train_batch_size: 28
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon = 1e - 08
• lr_scheduler_type: linear
• num_epochs: 300.0

Framework versions

• Transformers 4.11.2
• Pytorch 1.7.0
• Datasets 1.12.1
• Tokenizers 0.10.3

📄 License

The model is licensed under the MIT license.