Unieval Intermediate

🚀 Pre-trained Evaluator in EMNLP 2022 Paper

This project presents a pre - trained evaluator introduced in an EMNLP 2022 paper, aiming to provide a more comprehensive evaluation for text generation.

🚀 Quick Start

This README provides an overview of the pre - trained evaluator. For detailed usage, please refer to our GitHub repository.

✨ Features

Introduction

Multi - dimensional evaluation is the primary approach for human evaluation in Natural Language Generation (NLG). It involves assessing the generated text from multiple interpretable dimensions, such as coherence and fluency.

However, automatic evaluation in NLG is still mainly based on similarity - based metrics (e.g., ROUGE, BLEU), which are insufficient to distinguish between advanced generation models.

Therefore, we propose UniEval to address this issue and enable a more comprehensive and fine - grained evaluation of NLG systems.

Pre - trained Evaluator

unieval - intermediate is a pre - trained Boolean Answer Generator that has undergone intermediate multi - task learning. You can further train a custom evaluator for a specific NLG task based on this checkpoint.

💻 Usage Examples

Please refer to our GitHub repository for practical code examples and usage guides.