Cl Tohoku Bert Base Japanese V3 Jlpt Classifier

🚀 Text Classifier for JLPT Level Assignment

This project provides a text classifier designed to assign a JLPT level. It offers a way to estimate the difficulty of Japanese sentences, which can be useful for language learners.

🚀 Quick Start

The text classifier is trained at the sentence level. It fine - tunes a pre - trained [cl - tohoku - bert - japanese - v3](https://huggingface.co/cl - tohoku/bert - base - japanese - v3) model on approximately 5000 labeled sentences gathered from language learning websites.

✨ Features

Performance on Similar Distribution Data

The classifier shows good performance on data with a similar distribution to the training data. The following is the performance report:

              precision    recall  f1 - score   support
          N5       0.88      0.88      0.88        25
          N4       0.90      0.89      0.90        53
          N3       0.78      0.90      0.84        62
          N2       0.71      0.79      0.75        47
          N1       0.95      0.77      0.85        73
    accuracy                           0.84       260
   macro avg       0.84      0.84      0.84       260
weighted avg       0.85      0.84      0.84       260

Performance on Official JLPT Material

However, when tested on official JLPT material, the performance is not as good. The performance report is as follows:

              precision    recall  f1 - score   support
          N5       0.62      0.66      0.64       145
          N4       0.34      0.36      0.35       143
          N3       0.33      0.67      0.45       197
          N2       0.26      0.20      0.23       192
          N1       0.59      0.08      0.15       202
    accuracy                           0.38       879
   macro avg       0.43      0.39      0.36       879
weighted avg       0.42      0.38      0.34       879

Estimation Capability

Even though the precision is not very high, the classifier can still give a ballpark estimation of sentence difficulty.

📄 License

The project is licensed under the CC license.

📚 Documentation

Cite

If you use this classifier in your research, please cite the following paper:

@inproceedings{benedetti-etal-2024-automatically,
    title = "Automatically Suggesting Diverse Example Sentences for {L}2 {J}apanese Learners Using Pre-Trained Language Models",
    author = "Benedetti, Enrico  and
      Aizawa, Akiko  and
      Boudin, Florian",
    editor = "Fu, Xiyan  and
      Fleisig, Eve",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.acl-srw.11",
    pages = "114--131"
}