Querying multimedia presentations based on content

In this paper, we consider the problem of querying multimedia presentations based on content information. We believe that presentations should become an integral part of multimedia database systems and users should be able to store, query, and, possibly, manipulate multimedia presentations using a single database management system software. Multimedia presentations are modeled as presentation graphs, which are directed acyclic graphs that visually specify multimedia presentations. Each node of a presentation graph represents a media stream. Edges depict sequential or concurrent playout of streams during the presentation. Information captured in each individual stream and the presentation order of streams constitute the content information of the presentation. Querying multimedia presentation graphs based on content is important for the retrieval of information from a database. We present a graph data model for the specification of multimedia presentations and discuss query languages as effective tools to query and manipulate multimedia presentation graphs with respect to content information. To query the information flow throughout a multimedia presentation, as well as in each individual multimedia stream, we use revised versions of temporal operators Next, Connected, and Until, together with path formulas. These constructs allow us to specify and query paths along a presentation graph. We present an icon-based, graphical query language, GVISUAL, that provides iconic representations for these constructs and a user-friendly graphical interface for query specification. We also present an OQL-like language, GOQL (Graph OQL), with similar constructs, that allows textual and more traditional specifications of graph queries. Finally, we introduce GCalculus (Graph Calculus), a calculus-based language that establishes the formal grounds for the use of temporal operators in path formulas and for querying presentation graphs with respect to content information. We also discuss GCalculus/S (GCalculus with sets) which avoids highly complex query expressions by eliminating universal path quantifier, the negation operator, and the universal quantifier. GCalculus/S represents the formal basis for GVISUAL, i.e., GVISUAL uses the constructs of GCalculus/S directly.