Rured2 Ner Microsoft Mdeberta V3 Base

🚀 denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base

This is a Russian NER model fine-tuned on RURED2. It's a fine-tuned version of microsoft/mdeberta-v3-base, aiming to output multiple possible labels for a single token.

🚀 Quick Start

If you have any questions, you can message the author at https://t.me/nlp_party.

Here is an example of how to use this model:

import torch
from torch import nn
from transformers import (AutoTokenizer, AutoModelForTokenClassification, 
                          TrainingArguments, Trainer)

model_name = "denis-gordeev/rured2-ner-microsoft-mdeberta-v3-base"
model = AutoModelForTokenClassification.from_pretrained(
    model_name).to('cuda')

tokenizer = AutoTokenizer.from_pretrained(model_name)

def predict(text:str, glue_tokens=False, output_together=True, glue_words=True):
    sigmoid = nn.Sigmoid()
    tokenized = tokenizer(text)
    input_ids = torch.tensor(
            [tokenized["input_ids"]], dtype=torch.long
        ).to("cuda")
    token_type_ids = torch.tensor(
            [tokenized["token_type_ids"]], dtype=torch.long
        ).to("cuda")
    attention_mask = torch.tensor(
            [tokenized["attention_mask"]], dtype=torch.long
        ).to("cuda")
    preds = model(**{"input_ids": input_ids, "token_type_ids": token_type_ids, "attention_mask": attention_mask})
    logits = sigmoid(preds.logits)

    output_tokens = []
    output_preds = []
    id_to_label = {int(k): v for k, v in model.config.id2label.items()}
    for i, token in enumerate(input_ids[0]):
        if token > 3:
            class_ids = (logits[0][i] > 0.5).nonzero()
            if class_ids.shape[0] >= 1:
                class_names = [id_to_label[int(cl)] for cl in class_ids]
            else:
                class_names = [id_to_label[int(logits[0][i].argmax())]]
            converted_token = tokenizer.convert_ids_to_tokens([token])[0]
            new_word_bool = converted_token.startswith("▁")
            converted_token = converted_token.replace("▁", "")
            if glue_words and not(new_word_bool) and output_tokens:
                output_tokens[-1] += converted_token
            else:
                output_tokens.append(converted_token)
                output_preds.append(class_names)
        else:
            class_names = []
    if output_together:
        return [[output_tokens[t_i], output_preds[t_i]] for t_i in range(len(output_tokens))]
    return output_tokens, output_preds

📚 Documentation

This model is a fine-tuned version of microsoft/mdeberta-v3-base on the None dataset. It achieves the following results on the evaluation set:

## 📄 License

This model is released under the MIT license.