Bge M3 Zeroshot V2.0

🚀 bge-m3-zeroshot-v2.0

This model is designed for efficient zeroshot classification with the Hugging Face pipeline, enabling classification without training data and running on both GPUs and CPUs.

🚀 Quick Start

The zeroshot-v2.0 series of models are tailored for efficient zeroshot classification using the Hugging Face pipeline. These models can perform classification tasks without the need for training data and are compatible with both GPUs and CPUs. You can find an overview of the latest zeroshot classifiers in the Zeroshot Classifier Collection.

The key update in this zeroshot-v2.0 series is that several models are trained on fully commercially - friendly data, catering to users with strict license requirements.

These models handle a universal classification task: determining whether a hypothesis is "true" or "not true" given a text (entailment vs. not_entailment). This task format is based on the Natural Language Inference task (NLI). The Hugging Face pipeline can reformulate any classification task into this universal format.

✨ Features

• Efficient Zeroshot Classification: Perform classification without the need for training data.
• GPU and CPU Compatibility: Run on both GPUs and CPUs.
• Commercially - Friendly Data: Some models are trained on fully commercially - friendly data.
• Universal Task Format: Based on the NLI task, allowing any classification task to be reformulated.

📦 Installation

The library used for this model is transformers. You can install it using the following command:

#!pip install transformers[sentencepiece]

💻 Usage Examples

Basic Usage

from transformers import pipeline
text = "Angela Merkel is a politician in Germany and leader of the CDU"
hypothesis_template = "This text is about {}"
classes_verbalized = ["politics", "economy", "entertainment", "environment"]
zeroshot_classifier = pipeline("zero-shot-classification", model="MoritzLaurer/deberta-v3-large-zeroshot-v2.0")  # change the model identifier here
output = zeroshot_classifier(text, classes_verbalized, hypothesis_template=hypothesis_template, multi_label=False)
print(output)

multi_label=False forces the model to choose only one class, while multi_label=True allows the model to select multiple classes.

📚 Documentation

Training data

• Models with "-c" in the name: These models are trained on two types of fully commercially - friendly data:
  1. Synthetic data: Generated with Mixtral - 8x7B - Instruct - v0.1. A list of 500+ diverse text classification tasks for 25 professions was created in conversations with Mistral - large, manually curated, and then used to generate several hundred thousand texts with Mixtral - 8x7B - Instruct - v0.1. The final dataset is available in the synthetic_zeroshot_mixtral_v0.1 dataset, subset mixtral_written_text_for_tasks_v4.
  2. Commercially - friendly NLI datasets: MNLI and FEVER - NLI were added to enhance generalization.
• Models without "-c" in the name: These models use a broader mix of training data with a wider range of licenses, including ANLI, WANLI, LingNLI, and all datasets in this list where used_in_v1.1==True.

Metrics

The models were evaluated on 28 different text classification tasks using the [f1_macro](https://scikit - learn.org/stable/modules/generated/sklearn.metrics.f1_score.html) metric. The main reference point is facebook/bart-large-mnli, which, as of 03.04.24, is the most commonly used commercially - friendly 0 - shot classifier.

📄 License

This model is licensed under the MIT license.