A Hybrid Ontology and Content-Based Search Engine For Multimedia Retrieval

Huge amounts of digital visual content are currently avail- able, thus placing a demand for advanced multimedia search engines. The contribution of this paper is the presentation of a search engine that is capable of retrieving images based on their keyword annotation with the help of an ontology, or based on the image content to flnd similar images, or on both these strategies. To this end, the search engine is composed of two difierent subsystems, a low-level image feature analysis and retrieval system and a high-level ontology-based metadata structure. The novel feature is that the two subsystems can co-operate during the evaluation of a single query in a hybrid fashion. The system has been evaluated and experimental results on real cultural heritage collections are presented. Multimedia content management plays a key role in modern information sys- tems. From personal photo collections to media archives, cultural heritage col- lections and bio-medical applications, an extremely valuable information asset is in the form of images and video. To provide the same functionalities for the manipulation of such visual content as those provided for text processing, the development of search engines that perform the retrieval of the material is of high signiflcance. Such an advanced, semantic-enabled image search engine is the subject of the work presented in this paper. To date, two main approaches to image search engine techniques have been proposed, annotation-based and content-based. The former is based on image metadata or keywords that annotate the visual content. A well known exam- ple that falls into this category is Images Google Search1. The metadata that a search engine of this kind typically relies on refers either to the properties