Occlusion Matting: Realistic Occlusion Handling for Augmented Reality Applications

Nowadays, visualizations in Augmented Reality have to be as realistic as possible with lowest possible computational cost. In this paper, we present a real-time solution to realize dynamic occlusions. Sometimes virtual objects are softly, partially or totally occluded by real objects. Incorrect and inaccurate occlusion handling breaks the illusion of co-existence between the real and virtual world on the one hand and can result in wrong depth perception on the other hand. Our approach formulates the occlusion problem as alpha matting problem. Instead of calculating the visibility for each pixel of the virtual objects we estimate a blending coefficient. This enables a seamless integration of virtual objects in the real world, even for fuzzy foreground objects (like hair). Our approach takes raw depth information of the real scene (e.g. obtained by a low cost depth sensor) to realize rough foreground background segmentation. The blending coefficient between transitions where depth values are typically noisy is estimated based on the color image. Experimental evaluations of several scenes demonstrate that our algorithm produces consistent and visually appealing occlusions between the real and virtual scene with low computational cost. Furthermore, we compare the results with related depth-based approaches and show that our algorithm overcomes previous limitations.