Bert Mini

🚀 BERT-Mini (uncased)

This is a smaller BERT model (English only, uncased, trained with WordPiece masking) that solves the need for more lightweight language processing tasks, offering a balance between performance and resource consumption.

🚀 Quick Start

This is one of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) released by google-research/bert.

These BERT models were released as TensorFlow checkpoints, however, this is the converted version to PyTorch. More information can be found in google-research/bert or lyeoni/convert-tf-to-pytorch.

📚 Documentation

Evaluation

Here are the evaluation scores (F1/Accuracy) for the MPRC task.

References

@article{turc2019,
  title={Well-Read Students Learn Better: On the Importance of Pre-training Compact Models},
  author={Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
  journal={arXiv preprint arXiv:1908.08962v2 },
  year={2019}
}

📄 License

This project is licensed under the MIT license.