ElasticBERT-base Open Source Model - A Practical Tool Efficiently Supporting Dynamic Adjustment of Computing Resources

Home

Elasticbert Base

Developed by fnlp

ElasticBERT is an efficient multi-exit BERT model that supports dynamic adjustment of computational resources.

Large Language Model

Transformers

English#Multi-exit BERT #Dynamic computation #Efficient NLP

Downloads 252

Release Time : 3/2/2022

Model Overview

ElasticBERT is an efficient natural language processing model based on the BERT architecture, featuring a multi-exit structure that allows dynamic adjustment of computational resources according to needs, suitable for various NLP tasks.

Model Features

Multi-exit structure

Supports dynamic adjustment of computational resources, allowing selection of output layers at different depths based on requirements.

Efficient inference

Achieves dynamic allocation of computational resources through the multi-exit structure, improving inference efficiency.

Standardized evaluation

Provides a standardized evaluation framework for easy performance comparison with other models.

Model Capabilities

Text classification

Sequence labeling

Natural language understanding

Use Cases

Text analysis

Sentiment analysis

Analyze the sentiment tendency of text

Text classification

Classify text

🚀 ElasticBERT-BASE

This is an implementation of the base version of ElasticBERT, which aims to contribute to efficient NLP. It provides a strong baseline for related research and applications.

🚀 Quick Start

Model Information

Property	Details
Tags	Multi-exit-BERT
Datasets	wikipedia, bookcorpus, c4

Model Description

This is an implementation of the base version of ElasticBERT.

Towards Efficient NLP: A Standard Evaluation and A Strong Baseline

Xiangyang Liu, Tianxiang Sun, Junliang He, Lingling Wu, Xinyu Zhang, Hao Jiang, Zhao Cao, Xuanjing Huang, Xipeng Qiu

Code Link

fastnlp/elasticbert

💻 Usage Examples

Basic Usage

>>> from transformers import BertTokenizer as ElasticBertTokenizer
>>> from models.configuration_elasticbert import ElasticBertConfig
>>> from models.modeling_elasticbert import ElasticBertForSequenceClassification

>>> num_output_layers = 1
>>> config = ElasticBertConfig.from_pretrained('fnlp/elasticbert-base', num_output_layers=num_output_layers )
>>> tokenizer = ElasticBertTokenizer.from_pretrained('fnlp/elasticbert-base')
>>> model = ElasticBertForSequenceClassification.from_pretrained('fnlp/elasticbert-base', config=config)

>>> input_ids = tokenizer.encode('The actors are fantastic .', return_tensors='pt')
>>> outputs = model(input_ids)

📄 License

Citation

@article{liu2021elasticbert,
  author    = {Xiangyang Liu and
               Tianxiang Sun and
               Junliang He and
               Lingling Wu and
               Xinyu Zhang and
               Hao Jiang and
               Zhao Cao and
               Xuanjing Huang and
               Xipeng Qiu},
  title     = {Towards Efficient {NLP:} {A} Standard Evaluation and {A} Strong Baseline},
  journal   = {CoRR},
  volume    = {abs/2110.07038},
  year      = {2021},
  url       = {https://arxiv.org/abs/2110.07038},
  eprinttype = {arXiv},
  eprint    = {2110.07038},
  timestamp = {Fri, 22 Oct 2021 13:33:09 +0200},
  biburl    = {https://dblp.org/rec/journals/corr/abs-2110-07038.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご