Cubert 20210711 Python 512

🚀 CuBERT: Learning and Evaluating Contextual Embedding of Source Code

This model is designed to learn and evaluate the contextual embedding of source code, offering valuable insights for code analysis and understanding.

🚀 Quick Start

This model is the unofficial HuggingFace version of "CuBERT". Specifically, this version is sourced from gs://cubert/20210711_Python/pre_trained_model_epochs_2__length_512. It was trained on July 11, 2021, for 2 epochs with a 512 - token context window on the Python BigQuery dataset. The Tensorflow checkpoint was manually converted to PyTorch and uploaded here. Note that the tokenizer has not been converted yet. All credit goes to Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi.

The other versions are available at the following links:

• cubert - 20210711 - Python - 512
• cubert - 20210711 - Python - 1024
• cubert - 20210711 - Python - 2048
• cubert - 20210711 - Java - 512
• cubert - 20210711 - Java - 1024
• cubert - 20210711 - Java - 2048

📄 License

This project is licensed under the Apache - 2.0 license.

📚 Documentation

Citation

@inproceedings{cubert,
author    = {Aditya Kanade and
             Petros Maniatis and
             Gogul Balakrishnan and
             Kensen Shi},
title     = {Learning and evaluating contextual embedding of source code},
booktitle = {Proceedings of the 37th International Conference on Machine Learning,
               {ICML} 2020, 12 - 18 July 2020},
series    = {Proceedings of Machine Learning Research},
publisher = {{PMLR}},
year      = {2020},
}

Information Table