Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval

Overview

Matryoshka Hash Representations (MHR) learns compact binary document codes for semantic search and retrieval-augmented generation. One stored code supports multiple directly searchable prefix lengths, allowing a system to adjust its storage–quality trade-off without re-encoding the corpus.

Method

MHR first trains a full-width, ranking-aware binary representation, then freezes that model and learns zero-initialized residual code adaptors for shorter prefixes. Documents use one bit per coordinate; queries retain continuous logits. A FAISS FastScan implementation supports efficient asymmetric scoring.

Evaluation

Trained on MS MARCO and evaluated by zero-shot transfer to seven BEIR datasets, MHR reports NDCG@10 of 0.5561 and Recall@100 of 0.6535 at 32 bytes per document, compared with 0.5239 and 0.6426 for the best evaluated same-budget baseline. The paper also evaluates candidate shortlisting and low-storage graph-index pruning.

Citation

@misc{hua2026matryoshka,
      title={Matryoshka Hash Representations for Model-Aware Compact Semantic Retrieval}, 
      author={Peichun Hua and Yunming Xiao},
      year={2026},
      eprint={2609.07276},
      archivePrefix={arXiv},
      primaryClass={cs.IR},
      url={https://arxiv.org/abs/2609.07276}, 
}