A sketch recognition method based on transfer deep learning with the fusion of multi-granular sketches

Most of existing sketch recognition methods focus on the contour/shape of whole sketches. They ignore different granularities of sketches during sketching. Stroke sequences of sketches often demonstrate the change of various granularities. In the progress of sketching, a coarser-grained contour gradually changes to a finer-grained object. Different granularities of sketch imply different levels of semantic information and play different roles in sketch recognition. In this paper, a transfer-deep-learning-based sketch recognition method--“sketch-transfer-net” is proposed. Sketch-transfer-net designs a novel fine-tuning strategy to use different granular sketches to fine-tune different layers of neural network. The extensive comparative experiments show that the proposed sketch-transfer-net can capture descriptive information of various granular sketches and therefore improve the performance of sketch recognition. In addition, the novel fine-turning strategy could weaken the negative effect in transfer learning and enable CNNs to be well trained on small sketch datasets.