Distilbert Base 25lang Cased

🚀 distilbert-base-25lang-cased

We're sharing smaller versions of distilbert-base-multilingual-cased that support a custom set of languages. These versions generate identical representations to the original model, maintaining its accuracy.

Supported Languages

• English (en), French (fr), Spanish (es), German (de), Chinese (zh), Arabic (ar), Russian (ru), Vietnamese (vi), Greek (el), Bulgarian (bg), Thai (th), Turkish (tr), Hindi (hi), Urdu (ur), Swahili (sw), Dutch (nl), Ukrainian (uk), Romanian (ro), Portuguese (pt), Italian (it), Lithuanian (lt), Norwegian (no), Polish (pl), Danish (da) and Japanese (ja)

Datasets

• Wikipedia

License

• Apache-2.0

Widget Examples

• "Google generated 46 billion [MASK] in revenue."
• "Paris is the capital of [MASK]."
• "Algiers is the largest city in [MASK]."
• "Paris est la [MASK] de la France."
• "Paris est la capitale de la [MASK]."
• "L'élection américaine a eu [MASK] en novembre 2020."
• "تقع سويسرا في [MASK] أوروبا"
• "إسمي محمد وأسكن في [MASK]."

For more information, please visit our paper: Load What You Need: Smaller Versions of Multilingual BERT.

🚀 Quick Start

💻 Usage Examples

Basic Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Geotrend/distilbert-base-25lang-cased")
model = AutoModel.from_pretrained("Geotrend/distilbert-base-25lang-cased")

To generate other smaller versions of multilingual transformers, please visit our Github repo.

📄 How to cite

@inproceedings{smallermbert,
  title={Load What You Need: Smaller Versions of Multilingual BERT},
  author={Abdaoui, Amine and Pradel, Camille and Sigel, Grégoire},
  booktitle={SustaiNLP / EMNLP},
  year={2020}
}

📧 Contact

Please contact amine@geotrend.fr for any question, feedback or request.