🚀 COVID-Twitter-BERT v2 MNLI
This model offers a zero - shot classifier for scenarios where finetuning CT - BERT on a specific task is unfeasible due to the lack of labeled data.
🚀 Quick Start
The easiest way to try this out is by using the Hugging Face pipeline. This uses the default English template where it puts the text "This example is " in front of the text.
from transformers import pipeline
classifier = pipeline("zero-shot-classification", model="digitalepidemiologylab/covid-twitter-bert-v2-mnli")
You can then use this pipeline to classify sequences into any of the class names you specify.
sequence_to_classify = 'To stop the pandemic it is important that everyone turns up for their shots.'
candidate_labels = ['health', 'sport', 'vaccine', 'guns']
hypothesis_template = 'This example is {}.'
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template, multi_class=True)
✨ Features
- This model provides a zero - shot classifier for situations where finetuning CT - BERT on a specific task is impossible because of the lack of labelled data.
- The technique is based on Yin et al., which describes a clever way of using pre - trained MNLI models as zero - shot sequence classifiers.
- The model is already finetuned on 400,000 generic logical tasks.
📚 Documentation
Model description
This model provides a zero - shot classifier to be used in cases where it is not possible to finetune CT - BERT on a specific task, due to lack of labelled data.
The technique is based on Yin et al.. The article describes a very clever way of using pre - trained MNLI models as zero - shot sequence classifiers. The model is already finetuned on 400,000 generic logical tasks. We can then use it as a zero - shot classifier by reformulating the classification task as a question.
Let's say we want to classify COVID - tweets as vaccine - related and not vaccine - related. The typical way would be to collect a few hundred pre - annotated tweets and organise them in two classes. Then you would finetune the model on this.
With the zero - shot mnli - classifier, you can instead reformulate your question as "This text is about vaccines", and use this directly on inference - without any training.
Find more info about the model on our GitHub page.
Usage
Please note that how you formulate the question can give slightly different results. Collecting a training set and finetuning on this, will most likely give you better accuracy.
Training procedure
The model is finetuned on the 400k large MNLI - task.
📄 License
This project is licensed under the MIT license.
🔗 Additional Information
- Tags: Twitter, COVID - 19, text - classification, pytorch, tensorflow, bert
- Datasets: mnli
- Pipeline Tag: zero - shot - classification
- Thumbnail: COVID - Twitter - BERT_small
📖 References
@article{muller2020covid,
title={COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter},
author={M{\"u}ller, Martin and Salath{\'e}, Marcel and Kummervold, Per E},
journal={arXiv preprint arXiv:2005.07503},
year={2020}
}
or
Martin Müller, Marcel Salathé, and Per E. Kummervold.
COVID-Twitter-BERT: A Natural Language Processing Model to Analyse COVID-19 Content on Twitter.
arXiv preprint arXiv:2005.07503 (2020).