R

Rumodernbert Small

Developed by deepvk
A modern Russian version of a unidirectional and bidirectional encoder Transformer model, pre-trained on approximately 2 trillion tokens of Russian, English, and code data, with a context length of up to 8,192 tokens.
Downloads 619
Release Time : 1/24/2025

Model Overview

RuModernBERT is a modern Russian BERT model that supports masked language model tasks and is suitable for processing Russian and English texts.

Model Features

Long Context Support
Supports context lengths of up to 8,192 tokens, making it suitable for processing long texts.
Multilingual Support
Pre-trained data includes Russian, English, and code data, supporting multilingual tasks.
Efficient Training
Adopts a three-stage training strategy, optimizing model performance using data sources of varying quality.

Model Capabilities

Masked Language Model
Russian Text Processing
English Text Processing
Long Text Understanding

Use Cases

Natural Language Processing
Text Completion
Uses masked language model functionality to complete missing parts of sentences.
Example: 'Mom washed [MASK].' → 'Mom washed the dishes.'
Text Understanding
Processes and understands long text content.
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase