Mbart Russian Grammar Corrector

🚀 Fine-tuned Multilingual Bart for Russian Grammatical Error Correction

This project presents a fine-tuned version of Multilingual Bart, specifically trained in Russian for the task of Grammatical Error Correction (GEC). It offers a practical solution for correcting grammatical errors in Russian text.

🚀 Quick Start

Initialize the Model

from transformers import MBartForConditionalGeneration, MBart50TokenizerFast
model = MBartForConditionalGeneration.from_pretrained("MRNH/mbart-russian-grammar-corrector")

Use the Tokenizer

tokenizer = MBart50TokenizerFast.from_pretrained("MRNH/mbart-russian-grammar-corrector", src_lang="ru_RU", tgt_lang="ru_RU")
input = tokenizer("I was here yesterday to studying", text_target="I was here yesterday to study", return_tensors='pt')

Generate Text Using the Model

output = model.generate(input["input_ids"], attention_mask=input["attention_mask"], forced_bos_token_id=tokenizer_it.lang_code_to_id["ru_RU"])

🔧 Technical Details

Loss Computation

Training of the model is performed using the following loss computation based on the hidden state output h:

h.logits, h.loss = model(input_ids=input["input_ids"],
                         attention_mask=input["attention_mask"],
                         labels=input["labels"])

📄 Information Table