Illumination-aware live videos background replacement using antialiasing optimization

We propose a real-time illumination-aware live videos background replacement approach with antialiasing optimization on GPU in this paper. The aim of background replacement for live videos is to substitute the current real-time backgrounds with specially-chosen background images. Here we assume that the camera is stationary and the beginning of the video is only with a pure background scene. We propose the colored locality sensitive histograms (CLSH) considering the influence of other pixels to each pixel in every color channel to improve the performance of background segmentation, which makes the segmentation results robust enough to illumination differences. With the segmentation results, we then introduce a blocked real-time matting approach to enhance the accuracy of the objects’ boundary. Finally, to make the video composition more realistic, we propose a local antialiasing method to recover the distortions on edges. Compared with existing background replacement methods, our approach does not require costly blue/green screen or depth camera, but can produce more reliable video composition results. We have applied hardware GPU parallelism to speed up the live background replacement. Our illumination-aware video background replacement runs very efficiently in real-time, which can be applied for various video applications. The experimental results have shown the efficiency and high-quality rendering of our video background replacement in real-time.