Real Time Rectification for Stereo Correspondence

Duplicating the full dynamic capabilities of the human eye-brain combinationis a difficult task but an important goal because of the wideapplication for a system which can acquire {\it accurate} 3D models of a scene in realtime.Such a system must be able to correct images to remove lens distortionand camera misalignments from high resolution images at video framerates - 30 fps or better.The images then need to be matched todetermine the distance to scene objects.We have constructed a system whichuses reconfigurable hardware (FPGAs) to handle the very large number ofcalculations required and is capable of processing 1 Mpixel imagesfor disparity ranges of $\sim 100$ (allowing $\sim 1\%$ depth accuracy)at 30 fps.This paper focuses on the use of \LUT s in the hardwareto correct images in real time with latencies that are determined more bythe quality of the optics and mechanical alignment than by calculationdemand.Sample results from the full system (which uses the Symmetric Dynamic Programming Stereo matching algorithm) operating at 30 fps are also shown.