Netbert

🚀 NetBERT 📶

NetBERT is a pre - trained model based on [BERT - base](https://huggingface.co/bert - base - cased). It has been further trained on a large corpus of computer networking text (about 23Gb), which can be applied to various natural language processing tasks in the field of computer networking.

🚀 Quick Start

NetBERT can be used in multiple ways. You can use the raw model for masked language modeling (MLM), but it's mainly designed to be fine - tuned for downstream tasks, especially those that make decisions based on the whole sentence, such as text classification, extractive question answering, or semantic search.

💻 Usage Examples

Basic Usage

from transformers import pipeline

unmasker = pipeline('fill - mask', model='antoinelouis/netbert')
unmasker("The nodes of a computer network may include [MASK].")

Advanced Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained('antoinelouis/netbert')
model = AutoModel.from_pretrained('antoinelouis/netbert')

text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

📚 Documentation

Detailed documentation on the pre - trained model, its implementation, and the data can be found on Github.

📄 License

This project is licensed under the Apache - 2.0 license.

📄 Citation

For attribution in academic contexts, please cite this work as:

@mastersthesis{louis2020netbert,
    title={NetBERT: A Pre - trained Language Representation Model for Computer Networking},
    author={Louis, Antoine},
    year={2020},
    school={University of Liege}
}