Automatic recognition of craquelure and paint loss on polychrome paintings of the Palace Museum using improved U-Net

Craquelure is the most common defect on ancient polychrome paintings, which may deteriorate further to paint loss. Previous image processing methods, which can accurately recognize paint loss, have limited precision and efficiency in segmenting craquelure. This paper proposes a semantic segmentation method, Res-UNet, for the recognition of craquelure and paint loss in the Palace Museum, Beijing. The residual structure of ResNet-50 enables the avoidance of network degradation, and image features can be fully extracted. Using the unique skip connection module of U-Net, features of different levels are fused to improve segmentation accuracy and provide smoother craquelure edges. Three loss functions are combined to accelerate stable convergence. The model was tested on a newly built dataset based on 600 images. Experimental results supported by statistical tests show that Res-UNet is a capable method of craquelure recognition, with an accuracy rate of 98.19%, and F1-score of 93.42%. Hence, the proposed hybrid approach is a promising tool to support the preservation and restoration of valuable traditional Chinese polychrome architectural paintings.