Keyphrase Extraction Kbir Openkp

🚀 ml6team/keyphrase-extraction-kbir-openkp

This model is designed for keyphrase extraction, leveraging AI to quickly and efficiently identify important keyphrases from English documents, overcoming the time - consuming nature of manual extraction.

🚀 Quick Start

Keyphrase extraction is a technique in text analysis where you extract the important keyphrases from a document. Thanks to these keyphrases, humans can understand the content of a text very quickly and easily without reading it completely. Initially, keyphrase extraction was mainly done by human annotators, which was time - consuming for a large number of documents. Now, with the help of Artificial Intelligence, especially deep learning, the semantic meaning of a text can be better captured compared to classical machine learning methods.

✨ Features

• Utilizes KBIR as the base model and fine - tunes it on the OpenKP dataset.
• Solves the keyphrase extraction problem as a token classification task.
• Can quickly extract keyphrases from English texts.

📦 Installation

No specific installation steps are provided in the original README. If you want to use this model, you need to install relevant Python libraries such as transformers, datasets and numpy. You can use the following command to install them:

pip install transformers datasets numpy

💻 Usage Examples

Basic Usage

from transformers import (
    TokenClassificationPipeline,
    AutoModelForTokenClassification,
    AutoTokenizer,
)
from transformers.pipelines import AggregationStrategy
import numpy as np

# Define keyphrase extraction pipeline
class KeyphraseExtractionPipeline(TokenClassificationPipeline):
    def __init__(self, model, *args, **kwargs):
        super().__init__(
            model=AutoModelForTokenClassification.from_pretrained(model),
            tokenizer=AutoTokenizer.from_pretrained(model),
            *args,
            **kwargs
        )

    def postprocess(self, all_outputs):
        results = super().postprocess(
            all_outputs=all_outputs,
            aggregation_strategy=AggregationStrategy.SIMPLE,
        )
        return np.unique([result.get("word").strip() for result in results])

# Load pipeline
model_name = "ml6team/keyphrase-extraction-kbir-openkp"
extractor = KeyphraseExtractionPipeline(model=model_name)

# Inference
text = """
Keyphrase extraction is a technique in text analysis where you extract the
important keyphrases from a document. Thanks to these keyphrases humans can
understand the content of a text very quickly and easily without reading it
completely. Keyphrase extraction was first done primarily by human annotators,
who read the text in detail and then wrote down the most important keyphrases.
The disadvantage is that if you work with a lot of documents, this process
can take a lot of time. 

Here is where Artificial Intelligence comes in. Currently, classical machine
learning methods, that use statistical and linguistic features, are widely used
for the extraction process. Now with deep learning, it is possible to capture
the semantic meaning of a text even better than these classical methods.
Classical methods look at the frequency, occurrence and order of words
in the text, whereas these neural approaches can capture long-term
semantic dependencies and context of words in a text.
""".replace("\n", " ")

keyphrases = extractor(text)

print(keyphrases)

# Output
['keyphrase extraction' 'text analysis']

Advanced Usage

There is no advanced usage example in the original README. If you want to customize the model, you may need to adjust the training parameters, pre - processing and post - processing steps according to your own needs.

📚 Documentation

Model Description

This model uses KBIR as its base model and fine - tunes it on the OpenKP dataset. KBIR or Keyphrase Boundary Infilling with Replacement is a pre - trained model which utilizes a multi - task learning setup for optimizing a combined loss of Masked Language Modeling (MLM), Keyphrase Boundary Infilling (KBI) and Keyphrase Replacement Classification (KRC). You can find more information about the architecture in this paper.

Keyphrase extraction models are transformer models fine - tuned as a token classification problem where each word in the document is classified as being part of a keyphrase or not.

Intended Uses & Limitations

Limitations

• Limited amount of predicted keyphrases.
• Only works for English documents.

How To Use

The usage code has been provided in the "Usage Examples" section.

Training Dataset

OpenKP is a large - scale, open - domain keyphrase extraction dataset with 148,124 real - world web documents along with 1 - 3 most relevant human - annotated keyphrases. You can find more information in the paper.

Training Procedure

Training Parameters

Preprocessing

The documents in the dataset are already preprocessed into a list of words with the corresponding labels. The only thing that must be done is tokenization and the realignment of the labels so that they correspond with the right subword tokens.

from datasets import load_dataset
from transformers import AutoTokenizer

# Labels
label_list = ["B", "I", "O"]
lbl2idx = {"B": 0, "I": 1, "O": 2}
idx2label = {0: "B", 1: "I", 2: "O"}

# Tokenizer
tokenizer = AutoTokenizer.from_pretrained("bloomberg/KBIR")
max_length = 512

# Dataset parameters
dataset_full_name = "midas/openkp"
dataset_subset = "raw"
dataset_document_column = "document"
dataset_biotags_column = "doc_bio_tags"

def preprocess_fuction(all_samples_per_split):
    tokenized_samples = tokenizer.batch_encode_plus(
        all_samples_per_split[dataset_document_column],
        padding="max_length",
        truncation=True,
        is_split_into_words=True,
        max_length=max_length,
    )
    total_adjusted_labels = []
    for k in range(0, len(tokenized_samples["input_ids"])):
        prev_wid = -1
        word_ids_list = tokenized_samples.word_ids(batch_index=k)
        existing_label_ids = all_samples_per_split[dataset_biotags_column][k]
        i = -1
        adjusted_label_ids = []

        for wid in word_ids_list:
            if wid is None:
                adjusted_label_ids.append(lbl2idx["O"])
            elif wid != prev_wid:
                i = i + 1
                adjusted_label_ids.append(lbl2idx[existing_label_ids[i]])
                prev_wid = wid
            else:
                adjusted_label_ids.append(
                    lbl2idx[
                        f"{'I' if existing_label_ids[i] == 'B' else existing_label_ids[i]}"
                    ]
                )

        total_adjusted_labels.append(adjusted_label_ids)
    tokenized_samples["labels"] = total_adjusted_labels
    return tokenized_samples

# Load dataset
dataset = load_dataset(dataset_full_name, dataset_subset)

# Preprocess dataset
tokenized_dataset = dataset.map(preprocess_fuction, batched=True)
    

Postprocessing (Without Pipeline Function)

If you do not use the pipeline function, you must filter out the B and I labeled tokens. Each B and I will then be merged into a keyphrase. Finally, you need to strip the keyphrases to make sure all unnecessary spaces have been removed.

# Define post_process functions
def concat_tokens_by_tag(keyphrases):
    keyphrase_tokens = []
    for id, label in keyphrases:
        if label == "B":
            keyphrase_tokens.append([id])
        elif label == "I":
            if len(keyphrase_tokens) > 0:
                keyphrase_tokens[len(keyphrase_tokens) - 1].append(id)
    return keyphrase_tokens


def extract_keyphrases(example, predictions, tokenizer, index=0):
    keyphrases_list = [
        (id, idx2label[label])
        for id, label in zip(
            np.array(example["input_ids"]).squeeze().tolist(), predictions[index]
        )
        if idx2label[label] in ["B", "I"]
    ]

    processed_keyphrases = concat_tokens_by_tag(keyphrases_list)
    extracted_kps = tokenizer.batch_decode(
        processed_keyphrases,
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    )
    return np.unique([kp.strip() for kp in extracted_kps])

Evaluation Results

Traditional evaluation methods are the precision, recall and F1 - score @k,m where k is the number that stands for the first k predicted keyphrases and m for the average amount of predicted keyphrases. The model achieves the following results on the OpenKP test set:

🔧 Technical Details

• Base Model: KBIR, a pre - trained model using multi - task learning for optimizing combined loss.
• Dataset: OpenKP, a large - scale open - domain keyphrase extraction dataset.
• Training Method: Fine - tune the base model on the OpenKP dataset as a token classification problem.

📄 License

This project is licensed under the MIT license.

🚨 Issues

Please feel free to start discussions in the Community Tab.