Bert Base 5lang Cased

🚀 bert-base-5lang-cased

This is a smaller version of bert-base-multilingual-cased that supports only 5 languages (English, French, Spanish, German, and Chinese), offering a more lightweight alternative with comparable performance for these languages.

🚀 Quick Start

This model is a scaled - down variant of bert - base - multilingual - cased, designed to handle just 5 languages instead of 104. As a result, it's 30% smaller in size, with 124M parameters compared to the original's 178M. However, it provides identical representations for the specified languages. Starting your project with bert - base - 5lang - cased simplifies model deployment on public cloud platforms while maintaining similar results. For example, Google Cloud Platform mandates that the on - disk model size for serverless deployments (Cloud Functions / Cloud ML) should be under 500 MB, a requirement that the original bert - base - multilingual - cased fails to meet.

For more details on model size, memory footprint, and loading time, refer to the table below:

These measurements were taken on a Google Cloud n1 - standard - 1 machine (1 vCPU, 3.75 GB).

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("amine/bert-base-5lang-cased")
model = AutoModel.from_pretrained("amine/bert-base-5lang-cased")

How to cite

@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}

📄 License

This project is licensed under the Apache - 2.0 license.

📞 Contact

For any questions, feedback, or requests, please contact amine@geotrend.fr.