Scibertner

🚀 SciBERT-based Scientific Entity Recognition Model

This is a SciBERT-based model designed for scientific entity recognition, capable of identifying entities such as 'Generic', 'Material', 'Method', etc. in scientific texts.

🚀 Quick Start

You can use the following code to start using this model:

from transformers import AutoConfig, AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained('Kashob/SciBERTNER')
model = AutoModelForTokenClassification.from_pretrained('Kashob/SciBERTNER')
config = AutoConfig.from_pretrained('Kashob/SciBERTNER')
id2tag = config.id2label

text = 'The paper tackles the problem of endowing Transformers with the ability to encode information about the past via recurrence. The proposed architecture can leverage the recurrent connections to improve the sample efficiency while maintaining expressivity due to the use of self-attention.'.split()

inputs = tokenizer(text, is_split_into_words=True, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
    predictions = outputs.logits.argmax(-1)

tokenized_text = tokenizer.convert_ids_to_tokens(inputs['input_ids'].tolist()[0])
predicted_labels = [id2tag[label_id] for label_id in predictions[0].tolist()]
print(tokenized_text)
print(predicted_labels)

Output: 
['[CLS]', 'the', 'paper', 'tackle', '##s', 'the', 'problem', 'of', 'endow', '##ing', 'transformers', 'with', 'the', 'ability', 'to', 'encode', 'information', 'about', 'the', 'past', 'via', 'recurrence', '.', 'the', 'proposed', 'architecture', 'can', 'leverage', 'the', 'recurrent', 'connections', 'to', 'improve', 'the', 'sample', 'efficiency', 'while', 'maintaining', 'express', '##ivity', 'due', 'to', 'the', 'use', 'of', 'self', '-', 'attention', '.', '[SEP]']
['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B-OtherScientificTerm', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'B-Method', 'O', 'O', 'O', 'B-Generic', 'O', 'O', 'O', 'B-OtherScientificTerm', 'I-OtherScientificTerm', 'O', 'O', 'O', 'B-Metric', 'I-Metric', 'O', 'O', 'B-Metric', 'I-OtherScientificTerm', 'O', 'O', 'O', 'O', 'O', 'B-Method', 'I-OtherScientificTerm', 'I-OtherScientificTerm', 'O', 'O']

✨ Features

• Entity Recognition: This model can recognize predefined scientific entity types, including 'Generic', 'Material', 'Method', 'Metric', 'OtherScientificTerm', and 'Task'.

📚 Documentation

Model Details

Model Description

This is a SciBERT-based model for the Scientific Entity recognition task. The predefined entity types are: 'Generic', 'Material', 'Method', 'Metric', 'OtherScientificTerm', and 'Task'.

Model Sources

• Repository: NA
• Paper [optional]: In progress
• Demo [optional]: NA

Model Card Authors

Kashob Kumar Roy
CS, UIUC

Model Card Contact

• Email: kkroy2 at illinois dot edu

Feel free to reach out if you have any queries regarding this pre-trained model.

📄 License

This project is licensed under the MIT license.