ZSD Microsoft V2xxlmnli
An enhanced BERT decoding model based on the decoupled attention mechanism, a large-scale version fine-tuned on the MNLI task.
Downloads 59
Release Time : 3/2/2022
Model Overview
DeBERTa improves the BERT architecture through an innovative decoupled attention mechanism and an enhanced masked decoder, achieving SOTA performance on multiple natural language understanding tasks. This version is specifically fine-tuned for the MNLI (Multi-Genre Natural Language Inference) task.
Model Features
Decoupled Attention Mechanism
Separately calculates content and position attention, significantly improving the model's understanding of complex language structures.
Enhanced Masked Decoder
An improved masked language modeling method that better captures the dependencies between words.
Cross-task Transfer Ability
After fine-tuning on MNLI, it can be directly transferred to similar tasks such as RTE/MRPC/STS-B.
Model Capabilities
Natural Language Inference
Text Classification
Semantic Similarity Calculation
Zero-shot Classification
Use Cases
Text Understanding
Multi-genre Text Inference
Determine the logical relationship (entailment/contradiction/neutral) between two texts.
Achieved 91.7/91.9 accuracy on the MNLI test set.
Semantic Similarity Analysis
Evaluate the semantic similarity between sentence pairs.
Achieved a Pearson correlation coefficient of 93.2 on the STS-B dataset.
Transfer Learning
Few-shot Task Adaptation
Quickly adapt to inference tasks such as RTE based on the MNLI fine-tuned model.
Achieved 93.5 accuracy on the RTE task.
Featured Recommended AI Models
Š 2025AIbase