A 0.57 mW@1 FPS In-Column Analog CNN Processor Integrated Into CMOS Image Sensor

This article presents a high-performance, low-power analog convolutional neural network (CNN) circuit integrated into a CMOS image sensor (CIS) for face detection applications. The main block of the proposed in-column analog CNN circuits is an analog multiplication-and-accumulation (MAC) circuit consisting of an operational transconductance amplifier-based switched capacitor circuit enabling the programmable weight function. With the proposed MAC, a 3-layer analog CNN processor is implemented into the column-parallel readout circuit in conventional CIS. Furthermore, for low-power CNN operations, we use a low-resolution analog-to-digital converter with the proposed nonlinear quantization method resulting in an increase in the accuracy of face detection from 92.8% to 98.75% at 120 frame rates with 2.8 V/1.5 V supply voltage. A prototype sensor with $160\times120$ effective image resolution was fabricated using a 110 nm CMOS image sensor process. The measurement results showed that the maximum power consumption was 0.57 mW and 4.02 mW at 1 and 120 frame rates, respectively.