đ Pre-trained BERT on Twitter US Election 2020 for Stance Detection towards Donald Trump (f-BERT)
This project provides pre-trained weights for f-BERT, which can be used for stance detection towards Donald Trump based on tweets from the 2020 US Election.
⨠Features
- Pre-trained on over 5 million English tweets about the 2020 US Presidential Election.
- Fine-tuned with stance-labeled data for stance detection towards Donald Trump.
- Initialized with BERT-base and trained with a normal MLM objective, with the classification layer fine-tuned for the specific stance detection task.
đĻ Installation
The installation steps are not provided in the original document, so this section is skipped.
đģ Usage Examples
Basic Usage
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
import numpy as np
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
pretrained_LM_path = "kornosk/bert-election2020-twitter-stance-trump"
tokenizer = AutoTokenizer.from_pretrained(pretrained_LM_path)
model = AutoModelForSequenceClassification.from_pretrained(pretrained_LM_path)
id2label = {
0: "AGAINST",
1: "FAVOR",
2: "NONE"
}
sentence = "Hello World."
inputs = tokenizer(sentence.lower(), return_tensors="pt")
outputs = model(**inputs)
predicted_probability = torch.softmax(outputs[0], dim=1)[0].tolist()
print("Sentence:", sentence)
print("Prediction:", id2label[np.argmax(predicted_probability)])
print("Against:", predicted_probability[0])
print("Favor:", predicted_probability[1])
print("Neutral:", predicted_probability[2])
sentence = "Go Go Trump!!!"
inputs = tokenizer(sentence.lower(), return_tensors="pt")
outputs = model(**inputs)
predicted_probability = torch.softmax(outputs[0], dim=1)[0].tolist()
print("Sentence:", sentence)
print("Prediction:", id2label[np.argmax(predicted_probability)])
print("Against:", predicted_probability[0])
print("Favor:", predicted_probability[1])
print("Neutral:", predicted_probability[2])
sentence = "Trump is the worst."
inputs = tokenizer(sentence.lower(), return_tensors="pt")
outputs = model(**inputs)
predicted_probability = torch.softmax(outputs[0], dim=1)[0].tolist()
print("Sentence:", sentence)
print("Prediction:", id2label[np.argmax(predicted_probability)])
print("Against:", predicted_probability[0])
print("Favor:", predicted_probability[1])
print("Neutral:", predicted_probability[2])
đ Documentation
Training Data
This model is pre-trained on over 5 million English tweets about the 2020 US Presidential Election. Then fine-tuned using our stance-labeled data for stance detection towards Donald Trump.
Training Objective
This model is initialized with BERT-base and trained with normal MLM objective with classification layer fine-tuned for stance detection towards Donald Trump.
Usage Details
This pre-trained language model is fine-tuned to the stance detection task specifically for Donald Trump. Please see the official repository for more detail.
đ§ Technical Details
The technical details in the original document do not meet the requirement of more than 50 words, so this section is skipped.
đ License
This project is licensed under the GPL-3.0 license.
đ Reference
đ Citation
@inproceedings{kawintiranon2021knowledge,
title={Knowledge Enhanced Masked Language Model for Stance Detection},
author={Kawintiranon, Kornraphop and Singh, Lisa},
booktitle={Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
year={2021},
publisher={Association for Computational Linguistics},
url={https://www.aclweb.org/anthology/2021.naacl-main.376}
}