Rank Zephyr 7b V1 Full

🚀 Model Card for RankZephyr 7B V1 - Full

RankZephyr is a series of language models designed as helpful reranking assistants, built upon the Zephyr - 7B - β model. The RankZephyr Base model undergoes single - stage fine - tuning on the RankGPT - 3.5 model. Meanwhile, RankZephyr Full is further fine - tuned on RankGPT - 4 reorderings of OpenAI's Ada2 orderings for 5K queries.

🚀 Quick Start

The model is to be used in conjunction with the RankLLM repository. While rank - llm exists as a PyPI package, we are currently in the early stages of development and encourage users to directly check install from source.

✨ Features

• RankZephyr-7B-Full is the state - of - the - art open - source reranking model on various datasets like DL19/20/21/22, TREC - COVID, and TREC - News at the time of release.
• It is fine - tuned to act as a listwise reranking agent, taking a query and documents and returning a reordered list of document identifiers.

📦 Installation

The README does not provide specific installation steps, so this section is skipped.

📚 Documentation

Model description

Model Sources

• Repository: https://github.com/castorini/rank_llm
• Paper: https://arxiv.org/abs/2312.02724

Effectiveness

At the time of release, RankZephyr - 7B - Full is the state - of - the - art open - source reranking model on various datasets like DL19/20/21/22 and TREC - COVID and TREC - News.

With the MS MARCO v1 collection:

More details can be found in the paper.

Intended uses & limitations

The original Zephyr model is trained for chat. In our case, RankZephyr is fine - tuned to act as a listwise reranking agent. You provide it with a query and documents and get back a reordered list of document identifiers.

Bias, Risks, and Limitations

⚠️ Important Note

Zephyr - 7B - β has not been aligned to human preferences for safety within the RLHF phase or deployed with in - the - loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model (mistralai/Mistral - 7B - v0.1), however it is likely to have included a mix of Web data and technical sources like books and code. See the [Falcon 180B model card](https://huggingface.co/tiiuae/falcon - 180B#training - data) for an example of this.

💡 Usage Tip

Our model is trained specifically on monolingual English data, effectiveness on multilingual sets is not guaranteed.

Citation

If you find RankZephyr is useful in your work, please cite the following paper:

@ARTICLE{pradeep2023rankzephyr,
  title   = {{RankZephyr}: Effective and Robust Zero - Shot Listwise Reranking is a Breeze!},
  author  = {Ronak Pradeep and Sahel Sharifymoghaddam and Jimmy Lin},
  year    = {2023},
  journal = {arXiv:2312.02724}
}