Markov Logic Networks for Scene Interpretation and Complex Event Recognition in Videos

Automatic extraction and representation of visual concepts and semantic information in scenes is a desired capability in surveillance operations. We target the problem of complex event recognition in network information environment, where lack of effective visual processing tools and incomplete domain knowledge frequently cause uncertainty in the datasets and consequently, in the visual primitives extracted from it. We employ Markov Logic Network (MLN) to address the task of reasoning under uncertainty. In this work we demonstrate use of MLN as a domain knowledge representation language that can be used for inferring complex events in real world. MLN is a knowledge representation language that combines domain knowledge, visual concepts and experience to infer simple and complex real-world events. MLN generalizes over the existing probabilistic models, including hidden Markov models, Bayesian networks, and stochastic grammars. The framework can be made scalable to support variety of entities, their activities and interactions that are typically observed in the real world. Experiments with real-world data in a variety of urban settings illustrate the mathematical soundness and wide-ranging applicability of our approach.