Qurater 1.3B

🚀 QuRater Model

QuRater is a sequence classification model fine - tuned from a 1.3B Sheared - LLaMA model, which predicts quality ratings across 4 criteria.

🚀 Quick Start

The QuRater model is a sequence classification model. It predicts quality ratings across 4 criteria, and the unnormalized quality ratings can be accessed via the 4 logits in the model's output. The order of these logits is as follows:

• Logit 0: Writing Style
• Logit 1: Required Expertise
• Logit 2: Facts and Trivia
• Logit 3: Educational Value

📚 Documentation

Dataset

The model uses the following dataset:

• princeton - nlp/QuRating - GPT3.5 - Judgments

Paper Reference

This model is based on the paper: QuRating: Selecting High - Quality Data for Training Language Models

Important Notes

• The model was only fine - tuned on sequences of up to 512 tokens. It should not be used for longer documents. Instead, compute the quality ratings for windows of up to 512 tokens and average them weighted by the window length.

Guidance on Responsible Use

In the paper, various types of bias present in the quality ratings from the QuRater model are documented. These biases are related to domains, topics, social roles, regions, and languages (see Section 6 of the paper).

• Important Note
  • Be aware that data selection with QuRating could have unintended and harmful effects on the language model being trained. A comprehensive evaluation of the language model for these and other types of bias is strongly recommended, especially before real - world deployment.
  • The quality ratings do not measure the social or literary value of a text and should not be used for textual or demographic studies.
• Usage Tip We hope that releasing the data/models can facilitate future research aimed at uncovering and mitigating such biases.