Integrating videos with LIDAR scans for virtual reality

LIDAR range scans can be used to quickly create accurate 3D models for virtual reality and as a basis to visualize sets of photographs, videos, and virtual objects in a cohesive environment. The number of existing virtual reality programs that use LIDAR data as input has motivated our group to develop methods for fusing images with 3D scans and for augmenting the scans with both dynamic objects present in videos and virtual models. Bringing together as many data sources as possible increases users' abilities to present related information in one, intuitive venue. We demonstrate how to register a variety of 2D imagery with a range scan to construct photo-realistic models and to extract walking people captured in videos and model them in a 3D space. We also present a method for determining the sun position from a set of stitched photographs in order to apply correct lighting to virtual objects placed amongst real world data. Naturally lit objects can be inserted into original photographs using our 2D-3D registration information. These methods are all combined to display and study photos, videos, and virtual objects in a complete 3D environment.