A 4.2pJ/Pixel 480 fps Stereo Vision Processor with Pixel Level Pipelined Architecture and Two-Path Aggregation Semi-Global Matching

Binocular stereo vision, leveraging the principle of human visual perception to recover depth information from planar images, finds widespread deployment in 3D reconstruction, obstacle avoidance, and object detection. The semi-global matching (SGM) algorithm is renowned for its ability to provide precise depth information through semi-global optimization of the disparity across entire frames. However, the core operation of SGM, namely cost aggregation, requires massive computation and high-density on-chip memory, as depicted in Fig. 1(top). Furthermore, the data dependency issue within the aggregation process significantly impacts the system throughput, leading to performance degradation. Prior ASIC solutions [1]–[2] utilized external DRAM to minimize on-chip memory usage, leading to a notable increase in power consumption. The work in [3] proposed a diagonal scan method to mitigate the data dependency issue. However, it increases the complexity of data fetching, and the throughput is still limited since only 5~7 clock cycles slack is extended for the next pixel's aggregation process.