Gbert Large Zeroshot Nli

🚀 SVALabs - Gbert Large Zeroshot Nli

In this repository, we present our German zeroshot classification model, which is based on the German BERT large model and finetuned for natural language inference.

🚀 Quick Start

The simplest way to use the model is the huggingface transformers pipeline tool. Just initialize the pipeline specifying the task as "zero-shot-classification" and select "svalabs/gbert-large-zeroshot-nli" as model.

The model requires you to specify labels, a sequence (or list of sequences) to classify and a hypothesis template. In our tests, if the labels comprise only single words, "In diesem Satz geht es um das Thema {}" performed the best. However, for multiple words, especially when they combine nouns and verbs, simple hypothesis such as "Weil {}" or "Daher {}" may work better.

Here is an example of how to use the model:

from transformers import pipeline

zershot_pipeline = pipeline("zero-shot-classification",
                             model="svalabs/gbert-large-zeroshot-nli")

sequence = "Ich habe ein Problem mit meinem Iphone das so schnell wie möglich gelöst werden muss" 
labels = ["Computer", "Handy", "Tablet", "dringend", "nicht dringend"] 
hypothesis_template = "In diesem Satz geht es um das Thema {}."    

zershot_pipeline(sequence, labels, hypothesis_template=hypothesis_template)

✨ Features

• This model was trained on the basis of the German BERT large model from deepset.ai and finetuned for natural language inference based on 847.862 machine-translated nli sentence pairs, using the mnli, anli and snli datasets.
• If you are a German speaker you may also have a look at our Blog post about this model and about Zeroshot Classification.

📚 Documentation

Model Details

Performance

We evaluated our model for the nli task using the TEST set of the German part of the xnli dataset.

XNLI TEST-Set Accuracy: 85.6%

Zeroshot Text Classification Task Benchmark

We further tested our model for a zeroshot text classification task using a part of the 10kGNAD Dataset. Specifically, we used all articles that were labeled "Kultur", "Sport", "Web", "Wirtschaft" and "Wissenschaft".

The next table shows the results as well as a comparison with other German language and multilanguage zeroshot options performing the same task:

📄 License

No license information provided in the original document.

📞 Contact

• Nicole Wochatz, nicole.wochatz@sva.de
• Stefan Kammer, stefan.kammer@sva.de