Multimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedback

In the rapidly evolving landscape of information retrieval, search enginesstrive to provide more personalized and relevant results to users. Querysuggestion systems play a crucial role in achieving this goal by assistingusers in formulating effective queries. However, existing query suggestionsystems mainly rely on textual inputs, potentially limiting user searchexperiences for querying images. In this paper, we introduce a novel MultimodalQuery Suggestion (MMQS) task, which aims to generate query suggestions based onuser query images to improve the intentionality and diversity of searchresults. We present the RL4Sugg framework, leveraging the power of LargeLanguage Models (LLMs) with Multi-Agent Reinforcement Learning from HumanFeedback to optimize the generation process. Through comprehensive experiments,we validate the effectiveness of RL4Sugg, demonstrating a 18compared to the best existing approach. Moreover, the MMQS has been transferredinto real-world search engine products, which yield enhanced user engagement.Our research advances query suggestion systems and provides a new perspectiveon multimodal information retrieval.