Macbert4csc Scalarmix Base Chinese

🚀 MacBERT-based Typos Correction Model

This project fine-tunes the Mask Language Model (MLM) on MacBERT for typos correction, offering a practical solution for text error rectification.

🚀 Quick Start

You can refer to shibing624/macbert4csc-base-chinese for usage guidance.

✨ Features

• This project is modified based on shibing624/macbert4csc-base-chinese, and the corresponding source code location.
• Two main modifications are made:
  1. The hyperparameters for MLM and typos detection binary classification are changed to 0.9 and 0.1 (not necessarily the optimal parameters).
  2. A ScalarMix layer is introduced for typos detection binary classification. The original code uses the last layer of hidden_states, which I think is a bit too deep and might be more complex to learn.

📚 Documentation

Thoughts

Overall, the binary classification of typos detection does not have a prominent impact on the overall model performance, and the overall model performance does not exceed that of the original author by much. Therefore, the purpose of uploading this code and model is more for learning, recording, and thinking.

The results obtained by running pycorrector eval.py are as follows:

Corpus Dataset

Sentence Level: acc:0.7200, precision:0.8804, recall:0.6154, f1:0.7244, cost time:5.67 s

Sighan2015 Dataset

Sentence Level: acc:0.7973, precision:0.8265, recall:0.7459, f1:0.7841, cost time:11.19 s

📄 License

This project is licensed under the Apache-2.0 license.