Bert Large NER

🚀 bert-large-NER

bert-large-NER is a fine-tuned BERT model designed for Named Entity Recognition, delivering state-of-the-art performance. It can recognize four entity types: location, organizations, person, and miscellaneous. If you find this open - source model useful, your support can help build more small and practical AI models.

📚 Documentation

✨ Features

• Fine - tuned BERT: Specifically optimized for Named Entity Recognition.
• State - of - the - art performance: Achieves excellent results on the NER task.
• Entity recognition: Capable of identifying four types of entities: location (LOC), organizations (ORG), person (PER), and Miscellaneous (MISC).

📦 Installation

No specific installation steps are provided in the original document.

💻 Usage Examples

Basic Usage

You can use this model with Transformers pipeline for NER.

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("dslim/bert-large-NER")
model = AutoModelForTokenClassification.from_pretrained("dslim/bert-large-NER")

nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "My name is Wolfgang and I live in Berlin"

ner_results = nlp(example)
print(ner_results)

🔧 Technical Details

Intended uses & limitations

• How to use: As shown in the code example above, it can be used with the Transformers pipeline for NER.
• Limitations and bias: This model is restricted by its training dataset of entity - annotated news articles from a specific time period. It may not generalize well for all use cases in different domains. Also, the model sometimes tags subword tokens as entities, and post - processing of results may be required.

Training data

This model was fine - tuned on the English version of the standard CoNLL - 2003 Named Entity Recognition dataset.

The training dataset differentiates between the beginning and continuation of an entity. Each token is classified as one of the following classes:

CoNLL - 2003 English Dataset Statistics

• # of training examples per entity type | Dataset | LOC | MISC | ORG | PER | | ---- | ---- | ---- | ---- | ---- | | Train | 7140 | 3438 | 6321 | 6600 | | Dev | 1837 | 922 | 1341 | 1842 | | Test | 1668 | 702 | 1661 | 1617 |
• # of articles/sentences/tokens per dataset | Dataset | Articles | Sentences | Tokens | | ---- | ---- | ---- | ---- | | Train | 946 | 14,987 | 203,621 | | Dev | 216 | 3,466 | 51,362 | | Test | 231 | 3,684 | 46,435 |

Training procedure

This model was trained on a single NVIDIA V100 GPU with recommended hyperparameters from the original BERT paper, which trained and evaluated the model on the CoNLL - 2003 NER task.

📊 Eval results

The test metrics are a little lower than the official Google BERT results which encoded document context and experimented with CRF. More on replicating the original results here.

📄 License

This project is licensed under the MIT license.

📖 BibTeX entry and citation info

@article{DBLP:journals/corr/abs-1810-04805,
  author    = {Jacob Devlin and
               Ming{-}Wei Chang and
               Kenton Lee and
               Kristina Toutanova},
  title     = {{BERT:} Pre-training of Deep Bidirectional Transformers for Language
               Understanding},
  journal   = {CoRR},
  volume    = {abs/1810.04805},
  year      = {2018},
  url       = {http://arxiv.org/abs/1810.04805},
  archivePrefix = {arXiv},
  eprint    = {1810.04805},
  timestamp = {Tue, 30 Oct 2018 20:39:56 +0100},
  biburl    = {https://dblp.org/rec/journals/corr/abs-1810-04805.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

@inproceedings{tjong-kim-sang-de-meulder-2003-introduction,
    title = "Introduction to the {C}o{NLL}-2003 Shared Task: Language-Independent Named Entity Recognition",
    author = "Tjong Kim Sang, Erik F.  and
      De Meulder, Fien",
    booktitle = "Proceedings of the Seventh Conference on Natural Language Learning at {HLT}-{NAACL} 2003",
    year = "2003",
    url = "https://www.aclweb.org/anthology/W03-0419",
    pages = "142--147",
}