Distilbert Base Uncased Go Emotions Student

🚀 distilbert-base-uncased-go-emotions-student

A distilled model for text classification on the GoEmotions dataset, demonstrating zero-shot distillation.

🚀 Quick Start

This model is ready to be used for text classification tasks. You can integrate it into your projects following the general procedures for Hugging Face models.

✨ Features

• Distilled Model: Distilled from the zero - shot classification pipeline on the unlabeled GoEmotions dataset.
• Mixed Precision Training: Trained with mixed precision for 10 epochs.
• Demo of Zero - Shot Distillation: Shows how to distill an expensive NLI - based zero - shot model to a more efficient student.

📚 Documentation

Model Description

This model is distilled from the zero - shot classification pipeline on the unlabeled GoEmotions dataset using [this script](https://github.com/huggingface/transformers/tree/master/examples/research_projects/zero - shot - distillation). It was trained with mixed precision for 10 epochs and otherwise used the default script arguments.

Intended Usage

The model can be used like any other model trained on GoEmotions, but will likely not perform as well as a model trained with full supervision. It is primarily intended as a demo of how an expensive NLI - based zero - shot model can be distilled to a more efficient student, allowing a classifier to be trained with only unlabeled data. Note that although the GoEmotions dataset allow multiple labels per instance, the teacher used single - label classification to create psuedo - labels.

📄 License

This model is licensed under the MIT license.

Information Table

Widget Information

The widget can be used with the following sample text: "I feel lucky to be here."