xlm-roberta-base-wikiann-ner Open-source Model - Supports Identification of Locations, Organizations, and People's Names in 20 Languages

Xlm Roberta Base Wikiann Ner

Developed by Davlan

A multilingual named entity recognition model based on XLM-RoBERTa, supporting 20 languages, capable of identifying three types of entities: locations, organizations, and person names.

Sequence Labeling

Transformers

#Multilingual NER #High-precision entity recognition #Cross-lingual transfer learning

Downloads 68

Release Time : 3/2/2022

Model Overview

This model is a fine-tuned XLM-RoBERTa model on the WikiANN dataset, specifically designed for multilingual named entity recognition tasks, covering 20 languages.

Model Features

Multilingual Support

Supports named entity recognition in 20 languages, including several low-resource languages.

High Performance

Achieves state-of-the-art performance in NER tasks.

Entity Type Recognition

Capable of identifying three types of entities: locations (LOC), organizations (ORG), and person names (PER).

Model Capabilities

Multilingual named entity recognition

Entity classification

Use Cases

Text Analysis

News Article Entity Extraction

Extract entities such as locations, organizations, and person names from news articles.

Accurately identifies key entities in multilingual news.

Multilingual Applications

Cross-lingual Entity Recognition

Perform entity recognition in supported multilingual texts.

Suitable for text processing needs in multilingual environments.

🚀 xlm-roberta-base-wikiann-ner

The first Named Entity Recognition model for 20 languages, achieving state-of-the-art performance on the NER task.

✨ Features

Supports 20 languages including Arabic, Assamese, Bengali, Catalan, English, Spanish, Basque, French, Gujarati, Hindi, Indonesia, Igbo, Marathi, Punjabi, Portuguese, Swahili, Urdu, Vietnamese, Yoruba, and Chinese.
Based on a fine - tuned XLM - RoBERTa large model.
Trained to recognize three types of entities: location (LOC), organizations (ORG), and person (PER).

📦 Installation

No specific installation steps are provided in the original document.

🚀 Quick Start

How to use

You can use this model with Transformers pipeline for NER.

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline
tokenizer = AutoTokenizer.from_pretrained("xlm-roberta-base-wikiann-ner")
model = AutoModelForTokenClassification.from_pretrained("xlm-roberta-base-wikiann-ner")
nlp = pipeline("ner", model=model, tokenizer=tokenizer)
example = "Ìbọn ń ró kù kù gẹ́gẹ́ bí ọwọ́ ọ̀pọ̀ aráàlù ṣe tẹ ìbọn ní Kyiv láti dojú kọ Russia"
ner_results = nlp(example)
print(ner_results)

📚 Documentation

Intended uses & limitations

Limitations and bias

This model is limited by its training dataset of entity - annotated news articles from a specific span of time. This may not generalize well for all use cases in different domains.

🔧 Technical Details

Training data

This model was fine - tuned on 20 NER datasets (Arabic, Assamese, Bengali, Catalan, English, Spanish, Basque, French, Gujarati, Hindi, Indonesia, Igbo, Marathi, Punjabi, Portuguese, Swahili, Urdu, Vietnamese, Yoruba, Chinese)wikiann.

The training dataset distinguishes between the beginning and continuation of an entity so that if there are back - to - back entities of the same type, the model can output where the second entity begins. As in the dataset, each token will be classified as one of the following classes:

Property	Details
O	Outside of a named entity
B - PER	Beginning of a person’s name right after another person’s name
I - PER	Person’s name
B - ORG	Beginning of an organisation right after another organisation
I - ORG	Organisation
B - LOC	Beginning of a location right after another location
I - LOC	Location

BibTeX entry and citation info

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご