Xor Tydi Doctquery Mt5 Large

🚀 mT5-large Query Generation Model

This is an mT5-large query generation model trained with XOR QA data. It's designed to generate queries for text passages, which is useful in information retrieval tasks.

🚀 Quick Start

Installation

The necessary library for using this model is transformers. You can install it using the following command:

pip install transformers

Usage

Here is a Python code example to show how to use this model:

from transformers import pipeline

lang2mT5 = dict(
    ar='Arabic',
    bn='Bengali',
    fi='Finnish',
    ja='Japanese',
    ko='Korean',
    ru='Russian',
    te='Telugu'
)
PROMPT = 'Generate a {lang} question for this passage: {title} {passage}'

title = 'Transformer (machine learning model)'
passage = 'A transformer is a deep learning model that adopts the mechanism of self-attention, differentially ' \
          'weighting the significance of each part of the input (which includes the recursive output) data.'


model_name_or_path = 'ielabgroup/xor-tydi-docTquery-mt5-large'
input_text = PROMPT.format_map({'lang': lang2mT5['ja'],
                                'title': title,
                                'passage': passage})

generator = pipeline(model=model_name_or_path,
                     task='text2text-generation',
                     device="cuda:0",
                     )

results = generator(input_text,
                    do_sample=True,
                    max_length=64,
                    num_return_sequences=10,
                    )

for i, result in enumerate(results):
    print(f'{i + 1}. {result["generated_text"]}')

📚 Documentation

Model Details

Inference Parameters

The model uses the following parameters during inference:

• do_sample: true
• max_length: 64
• top_k: 10
• temperature: 1
• num_return_sequences: 10

Widget Examples

The model can handle the following input examples:

• Generate a Japanese question for the passage about Transformer (machine learning model).
• Generate an Arabic question for the passage about Transformer (machine learning model).

📄 License

This project is licensed under the Apache-2.0 license.

🔧 Technical Details

This model is used in the following research papers:

• Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation
• Augmenting Passage Representations with Query Generation for Enhanced Cross-Lingual Dense Retrieval

BibTeX entry and citation info

@article{zhuang2022bridging,
  title={Bridging the gap between indexing and retrieval for differentiable search index with query generation},
  author={Zhuang, Shengyao and Ren, Houxing and Shou, Linjun and Pei, Jian and Gong, Ming and Zuccon, Guido and Jiang, Daxin},
  journal={arXiv preprint arXiv:2206.10128},
  year={2022}
}

@inproceedings{zhuang2023augmenting,
	title={Augmenting Passage Representations with Query Generation for Enhanced Cross-Lingual Dense Retrieval},
	author={Zhuang, Shengyao and Shou, Linjun and Zuccon, Guido},
	booktitle={Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval},
	year={2023}
}