Deberta V3 Large Zeroshot V2.0

🚀 Model description: deberta-v3-large-zeroshot-v2.0

This project focuses on the deberta-v3-large-zeroshot-v2.0 model, which belongs to the zeroshot-v2.0 series. These models are designed for efficient zero-shot classification using the Hugging Face pipeline, capable of performing classification tasks without the need for training data and running on both GPUs and CPUs.

✨ Features

Zeroshot-v2.0 Series Models

• Universal Classification: These models can handle a universal classification task of determining whether a hypothesis is "true" or "not true" given a text (entailment vs. not_entailment). This task format is based on the Natural Language Inference task (NLI), and any classification task can be reformulated into this task by the Hugging Face pipeline.
• Commercially-Friendly Data: Some models in the zeroshot-v2.0 series are trained on fully commercially-friendly data, meeting the strict license requirements of users.

Training Data

• Models with "-c": Trained on two types of fully commercially-friendly data:
  • Synthetic Data: Generated with Mixtral-8x7B-Instruct-v0.1. The final dataset is available in the synthetic_zeroshot_mixtral_v0.1 dataset in the subset mixtral_written_text_for_tasks_v4.
  • NLI Datasets: Two commercially-friendly NLI datasets, MNLI and FEVER-NLI, are added to increase generalization.
• Models without "-c": Include a broader mix of training data with a broader mix of licenses, such as ANLI, WANLI, LingNLI, and datasets in this list where used_in_v1.1==True.

📦 Installation

The model can be used with the Hugging Face transformers library. You can install it using the following command:

#!pip install transformers[sentencepiece]

💻 Usage Examples

Basic Usage

from transformers import pipeline
text = "Angela Merkel is a politician in Germany and leader of the CDU"
hypothesis_template = "This text is about {}"
classes_verbalized = ["politics", "economy", "entertainment", "environment"]
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v2.0")  # change the model identifier here
output = zeroshot_classifier(text, classes_verbalized, hypothesis_template=hypothesis_template, multi_label=False)
print(output)

multi_label=False forces the model to decide on only one class, while multi_label=True enables the model to choose multiple classes.

📚 Documentation

Metrics

The models were evaluated on 28 different text classification tasks using the f1_macro metric. The main reference point is facebook/bart-large-mnli, which is, at the time of writing (03.04.24), the most used commercially-friendly 0-shot classifier.

📄 License

This project is licensed under the MIT license.