The open-source model: chinese-electra-180g-base-generator - Efficiently handling Chinese natural language tasks with a small number of parameters

Chinese Electra 180g Base Generator

Developed by hfl

ELECTRA model trained on 180G Chinese data, with small parameters but superior performance, suitable for Chinese natural language processing tasks

Large Language Model

Transformers

ChineseOpen Source License:Apache-2.0 #Small Parameter Efficient #Chinese Pre-trained Model #NLP Task Optimization

Downloads 28

Release Time : 3/2/2022

Model Overview

This model is one of the Chinese ELECTRA series released by the HIT-iFLYTEK Joint Lab, adopting the ELECTRA architecture. Compared to BERT, it has a smaller size and higher efficiency, suitable for various Chinese NLP tasks.

Model Features

Efficient Pre-training

Uses ELECTRA's replaced token detection task, which is more efficient than BERT's masked language model training.

Compact Size with High Performance

Parameters are only 1/10 of BERT, but achieve similar or even better performance in multiple NLP tasks.

Chinese Optimization

Specially optimized for Chinese language characteristics, trained on 180G Chinese data.

Model Capabilities

Text Understanding

Text Classification

Named Entity Recognition

Question Answering System

Text Similarity Calculation

Use Cases

Natural Language Processing

Sentiment Analysis

Analyze the sentiment tendency of user reviews

Accuracy comparable to BERT but with faster inference speed

Intelligent Customer Service

Used to build Chinese question-answering systems

Effectively understands user intent and provides accurate responses

🚀 Chinese ELECTRA

This project presents a Chinese ELECTRA model trained on 180G of data. It offers a more compact model size and competitive performance compared to BERT and its variants, making it a great choice for NLP tasks.

🚀 Quick Start

This model is trained on 180G data, and we recommend using this one over the original version.

✨ Features

Google and Stanford University released a new pre - trained model called ELECTRA, which has a much more compact model size and relatively competitive performance compared to BERT and its variants. To further accelerate the research of the Chinese pre - trained model, the Joint Laboratory of HIT and iFLYTEK Research (HFL) has released the Chinese ELECTRA models based on the official code of ELECTRA. ELECTRA - small can achieve similar or even higher scores on several NLP tasks with only 1/10 of the parameters compared to BERT and its variants.

This project is based on the official code of ELECTRA: [https://github.com/google - research/electra](https://github.com/google - research/electra)

📄 License

This project is licensed under the Apache - 2.0 license.

📚 Documentation

Citation

If you find our resource or paper useful, please consider including the following citation in your paper.

https://arxiv.org/abs/2004.13922

@inproceedings{cui-etal-2020-revisiting,
    title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
    author = "Cui, Yiming  and
      Che, Wanxiang  and
      Liu, Ting  and
      Qin, Bing  and
      Wang, Shijin  and
      Hu, Guoping",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
    pages = "657--668",
}

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご