The Chinese-legal-electra-small-generator Open-source Chinese Pretrained Model - Small in Size, Excellent in Performance, and Free to Use

Chinese Legal Electra Small Generator

Developed by hfl

Chinese ELECTRA is a Chinese pre-trained model released by the HIT-iFLYTEK Joint Lab based on Google's ELECTRA model, featuring compact size and superior performance.

Large Language Model

Transformers

ChineseOpen Source License:Apache-2.0 #Lightweight Pre-training #Chinese NLP Optimization #Legal Domain Specialization

Downloads 14

Release Time : 3/2/2022

Model Overview

Chinese ELECTRA is a pre-trained language model specifically designed for Chinese natural language processing tasks, offering efficient training and inference performance.

Model Features

Efficient Training

ELECTRA-small requires only 1/10 the parameters of BERT and its variants, yet achieves comparable or even better performance on multiple NLP tasks.

Chinese Optimization

Specifically optimized for Chinese natural language processing tasks.

Superior Performance

Excels in multiple NLP tasks and remains competitive against larger models.

Model Capabilities

Text Understanding

Text Classification

Question Answering Systems

Named Entity Recognition

Use Cases

Legal Domain

Legal Text Analysis

Used to analyze legal documents and extract key information.

General NLP Tasks

Text Classification

Classifies Chinese text.

Question Answering Systems

Builds Chinese question answering systems.

🚀 Chinese ELECTRA for Legal Domain

This model is specifically designed for the legal domain, leveraging the Chinese ELECTRA pre - trained model to provide efficient NLP solutions.

🚀 Quick Start

Google and Stanford University released a new pre - trained model called ELECTRA. Compared to BERT and its variants, ELECTRA has a much more compact model size while maintaining relatively competitive performance. To further accelerate the research on Chinese pre - trained models, the Joint Laboratory of HIT and iFLYTEK Research (HFL) released Chinese ELECTRA models based on the official ELECTRA code. ELECTRA - small can achieve similar or even higher scores on several NLP tasks with only 1/10 of the parameters compared to BERT and its variants.

This project is based on the official ELECTRA code: https://github.com/google-research/electra

📚 Documentation

Citation

If you find our resource or paper useful, please consider including the following citation in your paper:

https://arxiv.org/abs/2004.13922

@inproceedings{cui-etal-2020-revisiting,
    title = "Revisiting Pre-Trained Models for {C}hinese Natural Language Processing",
    author = "Cui, Yiming  and
      Che, Wanxiang  and
      Liu, Ting  and
      Qin, Bing  and
      Wang, Shijin  and
      Hu, Guoping",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings",
    month = nov,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.findings-emnlp.58",
    pages = "657--668",
}

📄 License

This project is licensed under the Apache - 2.0 license.

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご