D

Deberta V2 Xxlarge

Developed by microsoft
DeBERTa V2 XXLarge is an improved BERT model based on disentangled attention and enhanced mask decoding, with 1.5 billion parameters, surpassing BERT and RoBERTa performance on multiple natural language understanding tasks
Downloads 9,179
Release Time : 3/2/2022

Model Overview

DeBERTa improves the BERT architecture through innovative disentangled attention mechanisms and enhanced mask decoders. This XXLarge version features 48 network layers and a 1536-dimensional hidden layer, trained on 160GB of data, achieving excellent performance on benchmarks like GLUE and SQuAD

Model Features

Disentangled Attention Mechanism
Separates content and positional attention calculations for more precise modeling of text dependencies
Enhanced Mask Decoder
Improved masked language modeling objective that considers absolute positional information to enhance prediction capabilities
Large-scale Pretraining
Trained on 160GB of high-quality text data to learn richer language representations

Model Capabilities

Text Understanding
Semantic Analysis
Question Answering Systems
Text Classification
Natural Language Inference

Use Cases

Academic Research
GLUE Benchmark
Achieves state-of-the-art performance on the General Language Understanding Evaluation benchmark
MNLI accuracy 91.7%, SST-2 sentiment analysis 97.2%
Industrial Applications
Intelligent Customer Service
Used to understand user query intent and generate responses
Document Analysis
Automatically classifies and extracts key information from contracts/reports
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
Š 2025AIbase