M

Mdeberta V3 Base

Developed by microsoft
mDeBERTa is the multilingual version of DeBERTa, employing ELECTRA-style pretraining and gradient-disentangled embedding sharing technology, demonstrating excellent performance in cross-lingual tasks like XNLI
Downloads 692.08k
Release Time : 3/2/2022

Model Overview

A multilingual pretrained model based on DeBERTa architecture, supporting 15 languages, outperforming comparable models like XLM-R in zero-shot cross-lingual transfer tasks

Model Features

Gradient-disentangled Embedding Sharing
Uses ELECTRA-style pretraining method to optimize embedding layer sharing efficiency through gradient disentanglement technology
Multilingual Support
Supports 15 languages, achieving an average accuracy of 79.8% on XNLI cross-lingual tasks
Disentangled Attention Mechanism
Improved attention mechanism separates content and positional information processing to enhance model understanding

Model Capabilities

Multilingual text understanding
Zero-shot cross-lingual transfer
Masked language modeling
Text classification

Use Cases

Cross-lingual natural language understanding
XNLI Zero-shot Transfer
Trained only with English data then tested on 14 other languages
Average accuracy 79.8%, surpassing XLM-R-base's 76.2%
Multilingual text processing
Multilingual Text Classification
Supports text classification tasks in 15 languages
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
Š 2025AIbase