frugalscore_medium_bert-base_mover-score Open Source Text Evaluation Model - Lightweight Simulation of Large Model Scoring

Frugalscore Medium Bert Base Mover Score

Developed by moussaKam

A lightweight text evaluation model based on knowledge distillation, using a small student model to mimic the scoring behavior of a large teacher model

Large Language Model

Transformers

#Low-cost evaluation #NLG metric optimization #Knowledge distillation

Downloads 43

Release Time : 3/2/2022

Model Overview

Achieves expensive NLG evaluation metrics like BERTScore in a fixed low-cost manner, retaining over 90% of original performance

Model Features

Efficient knowledge distillation

Distills teacher model's scoring behavior through regression tasks, student model parameters are only 1% of teacher's

Multi-teacher compatibility

Supports knowledge transfer from various teacher models like BERT/RoBERTa/DeBERTa

Evaluation metric retention

Retains 90-95% of original performance on BERTScore and MoverScore metrics

Model Capabilities

Text generation quality assessment

Machine translation evaluation

Summarization evaluation

Dialogue system evaluation

Use Cases

NLG system development

Machine translation quality monitoring

Low-cost evaluation of translation model output quality in continuous integration

8-12x faster than original BERTScore

Text summarization evaluation

Automated evaluation pipeline for summarization generation models

0.85-0.92 correlation with human evaluation

Academic research

Lightweight evaluation benchmark

Replaces expensive evaluation metrics in resource-constrained environments

80% reduction in memory usage

FrugalScore	Student	Teacher	Method
moussaKam/frugalscore_tiny_bert-base_bert-score	BERT-tiny	BERT-Base	BERTScore
moussaKam/frugalscore_small_bert-base_bert-score	BERT-small	BERT-Base	BERTScore
moussaKam/frugalscore_medium_bert-base_bert-score	BERT-medium	BERT-Base	BERTScore
moussaKam/frugalscore_tiny_roberta_bert-score	BERT-tiny	RoBERTa-Large	BERTScore
moussaKam/frugalscore_small_roberta_bert-score	BERT-small	RoBERTa-Large	BERTScore
moussaKam/frugalscore_medium_roberta_bert-score	BERT-medium	RoBERTa-Large	BERTScore
moussaKam/frugalscore_tiny_deberta_bert-score	BERT-tiny	DeBERTa-XLarge	BERTScore
moussaKam/frugalscore_small_deberta_bert-score	BERT-small	DeBERTa-XLarge	BERTScore
moussaKam/frugalscore_medium_deberta_bert-score	BERT-medium	DeBERTa-XLarge	BERTScore
moussaKam/frugalscore_tiny_bert-base_mover-score	BERT-tiny	BERT-Base	MoverScore
moussaKam/frugalscore_small_bert-base_mover-score	BERT-small	BERT-Base	MoverScore
moussaKam/frugalscore_medium_bert-base_mover-score	BERT-medium	BERT-Base	MoverScore

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご

Frugalscore Medium Bert Base Mover Score

Model Overview

Model Features

Model Capabilities

Use Cases

🚀 FrugalScore

🚀 Quick Start

📚 Documentation