Procbert

🚀 ProcBERT

ProcBERT is a pre - trained language model tailored for procedural text. It was pre - trained on a large - scale procedural corpus with over 12B tokens, including PubMed articles, chemical patents, and cooking recipes. This model demonstrates excellent performance on downstream tasks.

🚀 Quick Start

ProcBERT is a pre - trained language model specifically designed for procedural text. It has been pre - trained on a large - scale corpus, showing great potential in downstream tasks.

✨ Features

• Domain - Specific: Trained on a diverse set of procedural text datasets such as PubMed, chemical patents, and cooking recipes.
• High Performance: Demonstrates excellent results on downstream tasks.

📦 Installation

Ensure you have the transformers library installed. You can install it using the following command:

pip install transformers

💻 Usage Examples

Basic Usage

from transformers import *
tokenizer = AutoTokenizer.from_pretrained("fbaigt/procbert")
model = AutoModelForTokenClassification.from_pretrained("fbaigt/procbert")

More usage details can be found here.

📚 Documentation

More details can be found in the following paper:

@inproceedings{bai-etal-2021-pre,
    title = "Pre-train or Annotate? Domain Adaptation with a Constrained Budget",
    author = "Bai, Fan  and
              Ritter, Alan  and
              Xu, Wei",
    booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2021",
    address = "Online and Punta Cana, Dominican Republic",
    publisher = "Association for Computational Linguistics",
}