Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation
ICMR '23: Proceedings of the 2023 ACM International Conference on Multimedia Retrieval(2023)
Abstract
Different task interpretations are a highly undesired element in interactive video retrieval evaluations. When a participating team focuses partially on a wrong goal, the evaluation results might become partially misleading. In this paper, we propose a process for refining known-item and open-set type queries, and preparing the assessors that judge the correctness of submissions to open-set queries. Our findings from recent years reveal that a proper methodology can lead to objective query quality improvements and subjective participant satisfaction with query clarity.
MoreTranslated text
Key words
video retrieval, evaluation, benchmarking, quality assurance
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined