Details
| Original language | English |
|---|---|
| Article number | 117 |
| Number of pages | 34 |
| Journal | ACM Transactions on Information Systems |
| Volume | 42 |
| Issue number | 5 |
| Early online date | 8 Nov 2023 |
| Publication status | Published - 29 Apr 2024 |
Abstract
Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency.We propose Fast-Forward indexes - vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially.We perform an evaluation to show the effectiveness and efficiency of Fast-Forward indexes - our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.
Keywords
- dual-encoders, efficiency, Information retrieval, IR, latency, ranking
ASJC Scopus subject areas
- Computer Science(all)
- Information Systems
- Business, Management and Accounting(all)
- Computer Science(all)
- Computer Science Applications
Cite this
- Standard
- Harvard
- Apa
- Vancouver
- BibTeX
- RIS
In: ACM Transactions on Information Systems, Vol. 42, No. 5, 117, 29.04.2024.
Research output: Contribution to journal › Article › Research › peer review
}
TY - JOUR
T1 - Efficient Neural Ranking Using Forward Indexes and Lightweight Encoders
AU - Leonhardt, Jurek
AU - Müller, Henrik
AU - Rudra, Koustav
AU - Khosla, Megha
AU - Anand, Abhijit
AU - Anand, Avishek
N1 - Publisher Copyright: © 2024 Copyright held by the owner/author(s).
PY - 2024/4/29
Y1 - 2024/4/29
N2 - Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency.We propose Fast-Forward indexes - vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially.We perform an evaluation to show the effectiveness and efficiency of Fast-Forward indexes - our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.
AB - Dual-encoder-based dense retrieval models have become the standard in IR. They employ large Transformer-based language models, which are notoriously inefficient in terms of resources and latency.We propose Fast-Forward indexes - vector forward indexes which exploit the semantic matching capabilities of dual-encoder models for efficient and effective re-ranking. Our framework enables re-ranking at very high retrieval depths and combines the merits of both lexical and semantic matching via score interpolation. Furthermore, in order to mitigate the limitations of dual-encoders, we tackle two main challenges: Firstly, we improve computational efficiency by either pre-computing representations, avoiding unnecessary computations altogether, or reducing the complexity of encoders. This allows us to considerably improve ranking efficiency and latency. Secondly, we optimize the memory footprint and maintenance cost of indexes; we propose two complementary techniques to reduce the index size and show that, by dynamically dropping irrelevant document tokens, the index maintenance efficiency can be improved substantially.We perform an evaluation to show the effectiveness and efficiency of Fast-Forward indexes - our method has low latency and achieves competitive results without the need for hardware acceleration, such as GPUs.
KW - dual-encoders
KW - efficiency
KW - Information retrieval
KW - IR
KW - latency
KW - ranking
UR - http://www.scopus.com/inward/record.url?scp=85178950416&partnerID=8YFLogxK
U2 - 10.48550/arXiv.2311.01263
DO - 10.48550/arXiv.2311.01263
M3 - Article
AN - SCOPUS:85178950416
VL - 42
JO - ACM Transactions on Information Systems
JF - ACM Transactions on Information Systems
SN - 1046-8188
IS - 5
M1 - 117
ER -