Color correlogram representation and differential earth mover's distance matching

The computer vision community has established the importance of low-level image representations and efficient inference methods in dictating the success of a vision algorithm. This thesis explores the following two aspects of the these themes. Over the years, the effectiveness of image statistics as a representation has been observed. The major challenge along this direction is the lack of spatial information and the difficulty in incorporating structural knowledge into the representation. We approach this problem by designing a simplified form of color correlogram (SCC) that encodes spatial correlation between colors and achieves a good balance between discrimination and efficiency. The representation is applied in the visual tracking context and demonstrates to be efficient and robust against appearance variations. We explore the representation further by a motion observability analysis. By analyzing the capability of the SCC in detecting and estimating object motion, a principled way to obtain motion observable SCCs as an object representation is proposed. With effective and efficient representations, the similarity measures to compare different objects are also key to computer vision problems. The Earth Mover's Distance (EMD) is a similarity measure that captures perceptual difference between two distributions. Its computational complexity, however, prevents a direct use in many practical applications. We propose a novel differential EMD (DEMD) algorithm that is based on the sensitivity analysis of the simplex method, and offers a speedup at orders of magnitude compared with its brute force counterparts. The DEMD algorithm is discussed and empirically verified in the visual tracking context. The deformations of the distributions for objects at different time instances are accommodated well by the EMD, and the gradient descent algorithm makes the use of EMD in real-time tracking possible. We employ compact representations, i.e., signatures, to further reduce the computation when EMD is used as a similarity measure. The new algorithm models and estimates local background scenes as well as foreground objects to handle scale changes in a principled way. Extensive quantitative evaluations of the proposed algorithm has been conducted using benchmark sequences and the improvement over its counterparts is demonstrated.