Image-based interactive exploration of real-world environments.

This article describes an image-based acquisition and rendering system that enables users to explore remote real-world locations. The system is based on a lightweight, high-resolution, multisensor camera that can be used to quickly film a tour through a large environment such as a house or a garden. During the offline authoring phase, the raw video is processed to produce stabilized high-quality, high-dynamic-range video using a combination of novel techniques. These include a stitching algorithm to remove parallax in areas of image overlap, registration and blending algorithms to produce high dynamic range video from alternately exposed video frames, feature tracking to perform camera stabilization, and a video compression scheme that supports selective runtime decompression and random access. The final user experience is enhanced with multimedia elements such as overview maps, video textures, and sound.The user controls the viewpoint and location through a standard game controller, and the resulting user experience feels much like a computer game. The combination of high-resolution, continuous imagery with real-time interactivity provides viewers with an unprecedented sense of presence in interesting real world environments.