Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network

This paper proposed a modified faster region-based convolutional neural network (faster R-CNN) for the multitype seismic damage identification and localization (i.e., concrete cracking, concrete spalling, rebar exposure, and rebar buckling) of damaged reinforced concrete columns from images. Four hundred raw images containing different damages and complicated background information are taken by a consumer-grade camera in various locations and arbitrary perspectives to simulate the diverse situations where real-world postearthquake damaged structural images are taken by nonprofessionals. Rectangular bounding boxes are obtained to localize multitype structural damages along with the corresponding category labels and classification probabilities. Data augmentation is implemented by rotation at every 90 degrees, vertical and horizontal flipping operations. An interactive labeling process for the ground-truth regions of the aforementioned damages is performed by a semiautomatic MATLAB program. A four-step alternating training procedure is adopted on the basis of the mini-batch stochastic gradient decent algorithm with momentum by backpropagation. Test results show that the trained faster R-CNN can automatically identify and localize the aforementioned multitype seismic damages and the overall average precision reaches 80%. The relative errors of coordinates of the left-top point obey minimum extreme value distributions, and those of width and height obey three-parameter lognormal distributions. The intersection ratio between the identification and ground truth has a mean value of 0.88, and the width-height ratio obeys a two-parameter lognormal distribution. Updated convolutional kernels in the first layer have shown trending, focusing, and line detectors for the feature extraction of multitype damages. Trending and focusing detectors contribute to the recognition of local damage regions, for example, concrete spalling and rebar exposure, whereas line detectors are more sensitive to the segmentation geometry, that is, concrete cracks.