Spatiotemporal Outdoor Lighting Aggregation on Image Sequences.

In this work, we focus on outdoor lighting estimation by aggregating individual noisy estimates from images, exploiting the rich image information from wide-angle cameras and/or temporal image sequences. Photographs inherently encode information about the scene’s lighting in the form of shading and shadows. Whereas computer graphic (CG) methods target accurately reproducing the image formation process knowing the exact lighting in the scene, the inverse rendering is an ill-posed problem attempting to estimate the geometry, material, and lighting behind a recorded 2D picture. Recent work based on deep neural networks has shown promising results for single image lighting estimation despite its difficulty. However, the main challenge remains on the stability of measurements. We tackle this problem by combining lighting estimates from many image views sampled in the angular and temporal domain of an image sequence. Thereby, we make efficient use of the camera calibration and camera ego-motion estimation to globally register the individual estimates and apply outlier removal and filtering algorithms. Our method not only improves the stability for rendering applications like virtual object augmentation but also shows higher accuracy for single image based lighting estimation compared to the state-of-the-art.