A Hybrid Ontology and Visual-based Retrieval Model for Cultural Heritage Multimedia Collections

Nowadays, an increasingly growing demand for advanced multimedia search engines is arising, as huge amounts of digital visual content are becoming available. The contribution of this paper is the introduction of a hybrid multimedia retrieval model accompanied by the presentation of a search engine that is capable of retrieving visual content from cultural heritage multimedia libraries as in three modes: (i) based on their semantic annotation with the help of an ontology; (ii) based on the visual features with a view to finding similar content; and (iii) based on the combination of these two strategies in order to produce recommendations. To achieve this, the retrieval model is composed of two different parts, a low-level visual feature analysis and retrieval and a high-level ontology infrastructure. The main novelty is the way in which these two co-operate transparently during the evaluation of a single query in a hybrid fashion, making recommendations to the user and retrieving content that is both visually and semantically similar. A search engine has been developed implementing this model which is capable of searching through digital libraries of cultural heritage collections, and indicative examples are discussed, along with insights into its performance.