Cubert 20210711 Python 1024

🚀 CuBERT: Learning and Evaluating Contextual Embedding of Source Code

This project focuses on learning and evaluating the contextual embedding of source code, providing an effective model for code - related tasks.

🚀 Quick Start

This model is an unofficial HuggingFace version of "CuBERT". Specifically, this version is sourced from gs://cubert/20210711_Python/pre_trained_model_epochs_2__length_1024. It was trained on July 11, 2021, for 2 epochs with a 1024 - token context window on the Python BigQuery dataset. The author manually converted the Tensorflow checkpoint to PyTorch and uploaded it here. Note that the tokenizer has not been converted yet. All the credit for this work goes to Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi.

The other versions are available at the following links:

• cubert - 20210711 - Python - 512
• cubert - 20210711 - Python - 1024
• cubert - 20210711 - Python - 2048
• cubert - 20210711 - Java - 512
• cubert - 20210711 - Java - 1024
• cubert - 20210711 - Java - 2048

📄 License

The model is released under the Apache - 2.0 license.

📚 Documentation

Citation

@inproceedings{cubert,
author    = {Aditya Kanade and
             Petros Maniatis and
             Gogul Balakrishnan and
             Kensen Shi},
title     = {Learning and evaluating contextual embedding of source code},
booktitle = {Proceedings of the 37th International Conference on Machine Learning,
               {ICML} 2020, 12 - 18 July 2020},
series    = {Proceedings of Machine Learning Research},
publisher = {{PMLR}},
year      = {2020},
}