Roberta Base Zeroshot V2.0 C

🚀 roberta-base-zeroshot-v2.0-c

The roberta-base-zeroshot-v2.0-c model is designed for efficient zeroshot classification, enabling classification without training data and running on both GPUs and CPUs.

🚀 Quick Start

This model is part of the zeroshot-v2.0 series, which is designed for efficient zeroshot classification using the Hugging Face pipeline. These models can perform classification without the need for training data and are compatible with both GPUs and CPUs. You can find an overview of the latest zeroshot classifiers in my Zeroshot Classifier Collection.

✨ Features

• Universal Classification: These models can handle a universal classification task of determining whether a hypothesis is "true" or "not true" given a text (entailment vs. not_entailment). This task format is based on the Natural Language Inference task (NLI), and any classification task can be reformulated into this task by the Hugging Face pipeline.
• Commercially-Friendly Data: The main update of the zeroshot-v2.0 series is that several models are trained on fully commercially-friendly data, suitable for users with strict license requirements.

📦 Installation

No specific installation steps are provided in the original document. However, to use the model, you need to have the transformers library installed. You can install it using the following command:

!pip install transformers[sentencepiece]

💻 Usage Examples

Basic Usage

#!pip install transformers[sentencepiece]
from transformers import pipeline
text = "Angela Merkel is a politician in Germany and leader of the CDU"
hypothesis_template = "This text is about {}"
classes_verbalized = ["politics", "economy", "entertainment", "environment"]
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v2.0")  # change the model identifier here
output = zeroshot_classifier(text, classes_verbalized, hypothesis_template=hypothesis_template, multi_label=False)
print(output)

multi_label=False forces the model to decide on only one class. multi_label=True enables the model to choose multiple classes.

📚 Documentation

Model description

Models in the zeroshot-v2.0 series are designed for efficient zeroshot classification with the Hugging Face pipeline. They can perform classification without training data and run on both GPUs and CPUs.

The main update of this zeroshot-v2.0 series of models is that several models are trained on fully commercially-friendly data for users with strict license requirements.

These models can do one universal classification task: determine whether a hypothesis is "true" or "not true" given a text (entailment vs. not_entailment). This task format is based on the Natural Language Inference task (NLI), and any classification task can be reformulated into this task by the Hugging Face pipeline.

Training data

Models with a "-c" in the name are trained on two types of fully commercially-friendly data:

1. Synthetic data generated with Mixtral-8x7B-Instruct-v0.1. The author first created a list of 500+ diverse text classification tasks for 25 professions in conversations with Mistral-large. The data was manually curated. Then, this was used as seed data to generate several hundred thousand texts for these tasks with Mixtral-8x7B-Instruct-v0.1. The final dataset used is available in the synthetic_zeroshot_mixtral_v0.1 dataset in the subset mixtral_written_text_for_tasks_v4. Data curation was done in multiple iterations and will be improved in future iterations.
2. Two commercially-friendly NLI datasets: (MNLI, FEVER-NLI). These datasets were added to increase generalization.
3. Models without a "-c" in the name also included a broader mix of training data with a broader mix of licenses: ANLI, WANLI, LingNLI, and all datasets in this list where used_in_v1.1==True.

Metrics

The models were evaluated on 28 different text classification tasks with the f1_macro metric. The main reference point is facebook/bart-large-mnli which, at the time of writing (03.04.24), is the most used commercially-friendly 0-shot classifier.

📄 License

The model is released under the MIT license.