bert-base-uncased Open-source Model - Free for Russian Text Processing, Trained on Large-scale Data

Bert Base Uncased

Developed by deepvk

Pre-trained bidirectional encoder for Russian text processing, trained on large-scale social data and Wikipedia

Downloads 465

Release Time : 2/7/2023

Model Overview

Russian pre-trained model based on BERT architecture, specialized for text feature extraction tasks, excluding pre-trained head modules

Russian optimization

Trained on 250GB Russian corpus including Wikipedia and social data

Pure encoder architecture

Contains only the encoder part, suitable for downstream task fine-tuning

Strict data filtering

Training data underwent rigorous quality control

Russian text feature extraction

Contextual semantic encoding

Downstream task fine-tuning foundation

Natural Language Processing

Text classification

Russian news classification/sentiment analysis

Achieved 0.467 accuracy on Russian Super Glue's RCB task

Question answering

Russian reading comprehension tasks

Achieved 0.737 accuracy on DaNetQA task

Model	RCB	PARus	MuSeRC	TERRa	RUSSE	RWSD	DaNetQA	Score
vk-deberta-distill	0.433	0.56	0.625	0.59	0.943	0.569	0.726	0.635
vk-roberta-base	0.46	0.56	0.679	0.769	0.960	0.569	0.658	0.665
vk-deberta-base	0.450	0.61	0.722	0.704	0.948	0.578	0.76	0.682
vk-bert-base	0.467	0.57	0.587	0.704	0.953	0.583	0.737	0.657
sber-bert-base	0.491	0.61	0.663	0.769	0.962	0.574	0.678	0.678

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base