t5-weighter_cnndm-en Open-source Classifier - Free Assessment of Question Importance and Screening of Answers Relevant to Summaries

T5 Weighter Cnndm En

Developed by ThomasNLG

A classifier based on the T5-small architecture, used to evaluate the importance of answer/question pairs and determine whether the answer is sufficiently relevant to appear in a summary.

Question Answering System Supports Multiple LanguagesOpen Source License:MIT #Question-Answer Weighting Evaluation #Summary Relevance Judgment #T5 Architecture

Downloads 178

Release Time : 3/2/2022

Model Overview

This model is a component of the QuestEval evaluation metric, designed to predict whether an answer/question pair is considered an important fact. It can be used independently for text summarization-related tasks.

Model Features

Summary Relevance Assessment

Automatically determines whether an answer is important enough to be included in a text summary.

QuestEval Component

Serves as the core module of the factuality evaluation metric QuestEval, supporting end-to-end summary quality assessment.

Lightweight Architecture

Based on the T5-small architecture, it reduces computational resource requirements while maintaining performance.

Model Capabilities

Text Importance Evaluation

Question-Answer Pair Relevance Judgment

Summary Quality Auxiliary Assessment

Use Cases

Text Summarization Evaluation

News Summary Generation

Evaluates whether automatically generated news summaries include key factual information.

Can identify important facts omitted in summaries (e.g., the incident of a guard slipping).

Question-Answer System Optimization

Answer Relevance Filtering

Filters highly relevant answers for knowledge base construction.

🚀 t5-weighter_cnndm-en

A classifier model based on T5-small that predicts the importance of an answer/question pair.

🚀 Quick Start

This model is a Classifier model based on T5-small, which predicts whether an answer/question pair is considered an important fact (i.e., whether the answer is relevant enough to appear in a plausible summary). It is actually a component of QuestEval metric but can be used independently.

✨ Features

Predicts the importance of answer/question pairs.
Can be used as part of the QuestEval metric or independently.

📦 Installation

No specific installation steps are provided in the original document other than the code to load the model. So, here is the code to get started with the model:

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

model = T5ForConditionalGeneration.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

💻 Usage Examples

Basic Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

model = T5ForConditionalGeneration.from_pretrained("ThomasNLG/t5-weighter_cnndm-en")

# The text input format should follow this template
text_input = "{ANSWER} </s> {QUESTION} </s> {CONTEXT}"

Advanced Usage

You can play with the model using the inference API. The text input format should follow the template text_input = "{ANSWER} </s> {QUESTION} </s> {CONTEXT}" according to the training stage of the model.

📚 Documentation

Training data

The model was trained on synthetic data as described in Questeval: Summarization asks for fact-based evaluation.

Citation info

@article{scialom2021questeval,
  title={Questeval: Summarization asks for fact-based evaluation},
  author={Scialom, Thomas and Dray, Paul-Alexis and Gallinari, Patrick and Lamprier, Sylvain and Piwowarski, Benjamin and Staiano, Jacopo and Wang, Alex},
  journal={arXiv preprint arXiv:2103.12693},
  year={2021}
}

📄 License

This model is licensed under the MIT license.

Property	Details
Model Type	Classifier based on T5-small
Training Data	Synthetic data as described in Questeval: Summarization asks for fact-based evaluation
Tags	qa, classification, question, answering, SQuAD, metric, nlg, t5-small
Datasets	squad, cnndm

Featured Recommended AI Models

Empowering the Future, Your AI Solution Knowledge Base

English 简体中文繁體中文にほんご