AIbase
EN
Home
BERT Of Theseus MNLI
B

BERT Of Theseus MNLI

Developed by canwenxu
A compressed BERT model achieved through progressive module replacement, reducing model complexity while maintaining performance
Large Language Model#Progressive Module Replacement#BERT Compression#Transfer Learning Optimization
Downloads 31
Release Time : 3/2/2022

Model Overview

BERT-of-Theseus is a novel compressed BERT model implemented by gradually replacing original BERT components, suitable for sentence classification tasks with excellent performance on the GLUE benchmark.

Model Features

Progressive Module Replacement
Achieves model compression by gradually replacing original BERT components, reducing complexity while maintaining performance
Intermediate Task Transfer Learning
The model is suitable for intermediate task transfer learning and requires fine-tuning for specific tasks
Performance Advantage
Outperforms DistillBERT of comparable size in the 6-layer architecture on the GLUE benchmark

Model Capabilities

Sentence Classification
Natural Language Inference
Text Similarity Calculation

Use Cases

Natural Language Processing
Text Classification
Can be used for various text classification tasks
Achieves 91.8% accuracy on SST-2 sentiment analysis task
Natural Language Inference
Determines logical relationships between two sentences
Achieves 82.1% accuracy on MNLI task
Sentence Similarity Calculation
Computes semantic similarity between two sentences
Achieves 87.8% accuracy on STS-B task
Featured Recommended AI Models
AIbase
Empowering the Future, Your AI Solution Knowledge Base
EnglishįŽ€äŊ“中文įšéĢ”中文ãĢãģんご
Š 2025AIbase