Mengzi Bert Base Fin

🚀 Mengzi-BERT base fin model (Chinese)

This model is a continued training of mengzi-bert-base using 20G of financial news and research reports. It employs Masked Language Modeling (MLM), Part-of-Speech (POS) tagging, and Sentence Order Prediction (SOP) as training tasks.

🚀 Quick Start

The Mengzi-BERT base fin model is a specialized pre - trained model for Chinese, fine - tuned on financial data. It can be easily integrated into your NLP projects using the transformers library.

# Import necessary classes from the transformers library
from transformers import BertTokenizer, BertModel
# Initialize the tokenizer
tokenizer = BertTokenizer.from_pretrained("Langboat/mengzi-bert-base-fin")
# Initialize the model
model = BertModel.from_pretrained("Langboat/mengzi-bert-base-fin")

💻 Usage Examples

Basic Usage

from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained("Langboat/mengzi-bert-base-fin")
model = BertModel.from_pretrained("Langboat/mengzi-bert-base-fin")

📚 Documentation

If you find the technical report or resource is useful, please cite the following technical report in your paper.

@misc{zhang2021mengzi,
      title={Mengzi: Towards Lightweight yet Ingenious Pre-trained Models for Chinese}, 
      author={Zhuosheng Zhang and Hanqing Zhang and Keming Chen and Yuhang Guo and Jingyun Hua and Yulong Wang and Ming Zhou},
      year={2021},
      eprint={2110.06696},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

📄 License

This project is licensed under the Apache-2.0 license.