Dragon Multiturn Query Encoder

🚀 Dragon-multiturn：多轮对话检索器

Dragon-multiturn 是专门为对话式问答场景设计的检索器，能够处理结合对话历史与当前查询的对话式查询。它基于 Dragon 检索器构建，可有效提升多轮对话问答的检索效果。

🚀 快速开始

模型简介

我们推出了 Dragon-multiturn，这是一款专门为对话式问答场景设计的检索器。它能够处理将对话历史与当前查询相结合的对话式查询，基于 Dragon 检索器构建。Dragon-multiturn 的详细信息可在 此处 找到。请注意，Dragon-multiturn 是一个双编码器，由查询编码器和上下文编码器组成。此仓库仅用于获取查询嵌入的 Dragon-multiturn 查询编码器，您还需要上下文编码器来获取上下文嵌入，可在 此处 找到。查询编码器和上下文编码器共享相同的分词器。

其他资源

• Llama3-ChatQA-1.5-8B
• Llama3-ChatQA-1.5-70B
• 评估数据
• 训练数据
• 项目网站
• 相关论文

基准测试结果

以上是在五个多轮问答数据集（Doc2Dial、QuAC、QReCC、TopiOCQA、INSCIT）上的检索结果，包含平均 top-1 和 top-5 召回率分数。*由于 TopiOCQA 和 INSCIT 中的平均上下文长度比其他数据集小，我们报告了 top-5 和 top-20，以大致匹配这些数据集中 top-1 和 top-5 的上下文长度。

如何使用

import torch
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained('nvidia/dragon-multiturn-query-encoder')
query_encoder = AutoModel.from_pretrained('nvidia/dragon-multiturn-query-encoder')
context_encoder = AutoModel.from_pretrained('nvidia/dragon-multiturn-context-encoder')

query = [
    {"role": "user", "content": "I need help planning my Social Security benefits for my survivors."},
    {"role": "agent", "content": "Are you currently planning for your future?"},
    {"role": "user", "content": "Yes, I am."}
]
contexts = [
    "Benefits Planner: Survivors | Planning For Your Survivors \nAs you plan for the future , you'll want to think about what your family would need if you should die now. Social Security can help your family if you have earned enough Social Security credits through your work. You can earn up to four credits each year. In 2019 , for example , you earn one credit for each $1,360 of wages or self - employment income. When you have earned $5,440 , you have earned your four credits for the year. The number of credits needed to provide benefits for your survivors depends on your age when you die. No one needs more than 40 credits 10 years of work to be eligible for any Social Security benefit. But , the younger a person is , the fewer credits they must have for family members to receive survivors benefits. Benefits can be paid to your children and your spouse who is caring for the children even if you don't have the required number of credits. They can get benefits if you have credit for one and one - half years of work 6 credits in the three years just before your death.  For Your Widow Or Widower \nThere are about five million widows and widowers receiving monthly Social Security benefits based on their deceased spouse's earnings record.",
    "Benefits Planner: Retirement \nOther Things to Consider \nWhat Is The Best Age To Start Your Benefits? The answer is that there is no one \" best age \" for everyone and, ultimately, it is your choice. You should make an informed decision about when to apply for benefits based on your individual and family circumstances. Your monthly benefit amount can differ substantially based on the age when you start receiving benefits. If you decide to start benefits : before your full retirement age , your benefit will be smaller but you will receive it for a longer period of time. at your full retirement age or later , you will receive a larger monthly benefit for a shorter period of time. The amount you receive when you first get benefits sets the base for the amount you will receive for the rest of your life. You may want to consider the following when you make that decision : If you plan to continue working , there are limits on how much you can earn each year between age 62 and full retirement age and still get all your benefits. Depending on the amount of your benefit and your earnings for the year , you may have to give up some of your benefits."
]

## convert query into a format as follows:
## user: {user}\nagent: {agent}\nuser: {user}
formatted_query = '\n'.join([turn['role'] + ": " + turn['content'] for turn in query]).strip()

## get query and context embeddings
query_input = tokenizer(formatted_query, return_tensors='pt')
ctx_input = tokenizer(contexts, padding=True, truncation=True, max_length=512, return_tensors='pt')
query_emb = query_encoder(**query_input).last_hidden_state[:, 0, :]
ctx_emb = context_encoder(**ctx_input).last_hidden_state[:, 0, :]

## Compute similarity scores using dot product
similarities = query_emb.matmul(ctx_emb.transpose(0, 1)) # (1, num_ctx)

## rank the similarity (from highest to lowest)
ranked_results = torch.argsort(similarities, dim=-1, descending=True) # (1, num_ctx)

多轮问答检索基准评估

(更新！！) 我们在五个数据集（Doc2Dial、QuAC、QReCC、TopiOCQA 和 INSCIT）上对多轮问答检索进行了评估，这些数据集可在 ChatRAG Bench 中找到。评估脚本可在 此处 找到。

许可证

Dragon-multiturn 基于 Dragon 构建。我们建议用户参考 Dragon 模型的原始许可证。Dragon-multiturn 也受 使用条款 的约束。

联系方式

• 刘梓涵 (zihanl@nvidia.com)
• 平伟 (wping@nvidia.com)

引用

@article{liu2024chatqa,
  title={ChatQA: Surpassing GPT-4 on Conversational QA and RAG},
  author={Liu, Zihan and Ping, Wei and Roy, Rajarshi and Xu, Peng and Lee, Chankyu and Shoeybi, Mohammad and Catanzaro, Bryan},
  journal={arXiv preprint arXiv:2401.10225},
  year={2024}
}