Direct Imaging of Stabilized Optical Flow and Possible Anomalies From Moving Vehicle

Machine perception of dynamic scenes becomes more and more important for autonomous vehicles and vision-based driver-assistance systems. Even with other 3D ranging devices, dense, detailed and instantaneous detection of optical flow is essential for early distinguishing small moving objects in the 3D environment from the moving vehicle. To overcome the limited performance in the immediacy, resolution, accuracy and acuity of existing methods, we provide an optical flow detection scheme based on a three-phase correlation image sensor (3PCIS) that is capable of Fourier-coefficient imaging combined with an exact and direct algorithm derived from the weighted integral method of identifying the differential equation model from a short-duration observation. To utilize inherent performances of the detection scheme by removing the large and rapid disturbances induced by the rotational fluctuations of the platform, we introduce a software operation of gaze in which the image coordinates are fixed on and smoothly pursue a forward stable object so that the optical flow field is relative to the moving coordinate system. In it, the gaze subsystem continuously provides angular velocity and pose between the camera and gaze target, while the imaging subsystem instantaneously obtains two optical flow distributions by cancelling the ego-rotation components and then removing the outwardly diverging components derived mainly from stationary 3D environments. Possible anomalies captured in each frame instantaneously provide candidates of hazardous objects that should be tracked and further investigated. We examine the performance of optical flow stabilization and anomaly detection using image sequences of monocular 3PCIS mounted on a moving vehicle on town roads and a highway.