đ Hypencoder Model
Hypencoder is a hypernetwork-based model for information retrieval, offering efficient and effective text encoding and relevance scoring.
đ Quick Start
This is the official model from the paper Hypencoder: Hypernetworks for Information Retrieval. To use this model, please refer to the Github page, which contains the necessary code and details on how to run the model.
⨠Features
- Dual Encoder Architecture: Comprising a text encoder and Hypencoder, it can convert text into vectors and small neural networks for relevance scoring.
- Multiple Model Variants: Four models with different numbers of hidden layers in the q - nets are available.
đĻ Installation
The installation details are not provided in the original document.
đģ Usage Examples
Basic Usage
from hypencoder_cb.modeling.hypencoder import Hypencoder, HypencoderDualEncoder, TextEncoder
from transformers import AutoTokenizer
dual_encoder = HypencoderDualEncoder.from_pretrained("jfkback/hypencoder.6_layer")
tokenizer = AutoTokenizer.from_pretrained("jfkback/hypencoder.6_layer")
query_encoder: Hypencoder = dual_encoder.query_encoder
passage_encoder: TextEncoder = dual_encoder.passage_encoder
queries = [
"how many states are there in india",
"when do concussion symptoms appear",
]
passages = [
"India has 28 states and 8 union territories.",
"Concussion symptoms can appear immediately or up to 72 hours after the injury.",
]
query_inputs = tokenizer(queries, return_tensors="pt", padding=True, truncation=True)
passage_inputs = tokenizer(passages, return_tensors="pt", padding=True, truncation=True)
q_nets = query_encoder(input_ids=query_inputs["input_ids"], attention_mask=query_inputs["attention_mask"]).representation
passage_embeddings = passage_encoder(input_ids=passage_inputs["input_ids"], attention_mask=passage_inputs["attention_mask"]).representation
passage_embeddings_single = passage_embeddings.unsqueeze(1)
scores = q_nets(passage_embeddings_single)
passage_embeddings_double = passage_embeddings.repeat(2, 1).reshape(2, 2, -1)
scores = q_nets(passage_embeddings_double)
đ Documentation
Model Details
This is a Hypencoder Dual Encoder. It contains two trunks: the text encoder and Hypencoder. The text encoder converts items into 768 - dimension vectors, while the Hypencoder converts text into a small neural network which takes the 768 - dimension vector from the text encoder as input. This small network is then used to output a relevance score.
Model Variants
We released the four models used in the paper. Each model is identical except the small neural networks, which we refer to as q - nets, have different numbers of hidden layers.
đ License
The model is released under the MIT license.
đ Citation
BibTeX:
@misc{killingback2025hypencoderhypernetworksinformationretrieval,
title={Hypencoder: Hypernetworks for Information Retrieval},
author={Julian Killingback and Hansi Zeng and Hamed Zamani},
year={2025},
eprint={2502.05364},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/2502.05364},
}
đ Model Information
Property |
Details |
Base Model |
google - bert/bert - base - uncased |
Datasets |
microsoft/ms_marco |
Language |
en |
Library Name |
transformers |
Pipeline Tag |
feature - extraction |