Unsupervised query-adaptive implicit subtopic discovery for diverse image retrieval based on intrinsic cluster quality

Given the complex search tasks imposed to social multimedia retrieval systems, the generated similarity-based ranked results often represent redundant item sets, including, e.g., near-duplicates or unrepresentative samples. In this context, several real-world search tasks demand broad coverage of multiple implicit subtopics of a given query in order to properly fulfill the user need. Many works have proposed the use of result diversification for addressing such problem. As a popular approach, the diversification is achieved by grouping similar items obtained from the original ranked list. Hence, a new and diverse ranked list is constructed by iteratively selecting a representative item from each cluster. However, the definition of the number of clusters (subtopics) to be discovered is a long-lasting challenge. Moreover, most clustering optimization approaches for diversification rely on offline training for the selection of a general best configuration used for all queries at run-time. However, this is a complex task given the multiple heterogeneity associated (data, user, query, concepts, etc.) and the consequent impact on the effectiveness of retrieval algorithms. Therefore, such approaches are usually prone to overfit. Hence, in order to attenuate such problems, this work proposes a novel diverse image retrieval approach as an unsupervised query-adaptive subtopic discovery based on intrinsic clustering quality optimization. Our experimental analysis have shown significant improvements in relation to the baseline, both in terms of relevance and diversity.