TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy
arxiv(2024)
Abstract
Large Language Models (LLMs) are increasingly employed in zero-shot documents
ranking, yielding commendable results. However, several significant challenges
still persist in LLMs for ranking: (1) LLMs are constrained by limited input
length, precluding them from processing a large number of documents
simultaneously; (2) The output document sequence is influenced by the input
order of documents, resulting in inconsistent ranking outcomes; (3) Achieving a
balance between cost and ranking performance is quite challenging. To tackle
these issues, we introduce a novel documents ranking method called TourRank,
which is inspired by the tournament mechanism. This approach alleviates the
impact of LLM's limited input length through intelligent grouping, while the
tournament-like points system ensures robust ranking, mitigating the influence
of the document input sequence. We test TourRank with different LLMs on the
TREC DL datasets and the BEIR benchmark. Experimental results show that
TourRank achieves state-of-the-art performance at a reasonable cost.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined