Chinesebert Base

🚀 ChineseBERT-base

This project refines ChineseBERT, allowing users to directly call it via the HuggingFace API without additional code configuration.

Original Paper

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu and Jiwei Li

Original Project

ChineseBERT github link

Original Model

ShannonAI/ChineseBERT-base (This model cannot be directly called via the HuggingFace API)

🚀 Quick Start

📦 Installation

1. Install pypinyin

pip install pypinyin

💻 Usage Examples

Basic Usage

2. Load the tokenizer and model using AutoClass

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("iioSnail/ChineseBERT-base", trust_remote_code=True)
model = AutoModel.from_pretrained("iioSnail/ChineseBERT-base", trust_remote_code=True)

3. Then use it in the same way as a normal BERT model

inputs = tokenizer(["我 喜 [MASK] 猫"], return_tensors='pt')
logits = model(**inputs).logits

print(tokenizer.decode(logits.argmax(-1)[0, 1:-1]))

Output:

tokenizer.decode(logits.argmax(-1)[0, 1:-1])

💡 Usage Tip

To get the hidden_state, use model.bert(**inputs).last_hidden_state

📚 Documentation

Frequently Asked Questions

1. Network issues, e.g., Connection Error Solution: Download the model locally. Refer to this blog for batch download solutions.

2. Error when using the locally downloaded model: ModuleNotFoundError: No module named 'transformers_modules.iioSnail/ChineseBERT-base' Solution: Change iioSnail/ChineseBERT-base to iioSnail\ChineseBERT-base

📄 License

This project is licensed under the AFL-3.0 license.