Context in Video Search: Is Close-by Good Enough When Using Linking?

Information or feedback about users' information needs beyond a brief query is crucial for improving the effectiveness of video search. One paradigm that addresses this issue is search and linking, i.e., after an initial search the user selects an item and requests a set of items that share properties with the one selected. We propose an approach for the linking step using both textual and visual features. We first evaluate the proposed method on the MediaEval 2013 Search and Hyperlinking data set. The video segment from which linking starts cannot be assumed to be relevant and to have well-defined boundaries in a real-world setting, in contrast to the benchmarking data set. We thus investigate the performance of the proposed linking method on actual search results of different systems and using only a coarse estimate of a contextual segment (e.g., scene, story). The linking results starting from a related segment among the top-ranked search results are nearly as good as when using the correct segment as starting point. The results show that contextual information surrounding the segment used for generating the linking results plays a crucial role. When starting from a relevant or at least related item in the search result, the use of context information significantly improves results. For segments related to the query, the linking results are similar in terms of performance to those starting from relevant segments (without context), but contextual information does not improve the results.