Self-designed hierarchical network based hand shape intelligent recognition and evaluation for Chinese zither performing

A proper hand shape is the foundation for professional musical instrument performance. In this paper, the image recognition technology based on artificial intelligence is introduced into the hand shape recognition of performing Chinese zither, which is referred to as Zheng in Chinese, for the first time to realize the function of hand shape intelligent evaluation and the self-designed hierarchical network is proposed to recognize and evaluate correct hand shape for Zheng performing. The intra-class difference is larger than the inter-class difference for Zheng performing hand shape image, which belongs to fine-grained image. Therefore, we use the first layer network to determine four classes of images acquired from different viewpoints. Meanwhile, the feature maps from different convolutional blocks of this layer are concatenated as the input of the second layer, which performs fine classification of Zheng performing hand shape images. Consequently, the learning ability of the network can be improved and the complexity of the network can be reduced at the same time. We design an experimental paradigm for instrumentalist hand shape performance evaluation, formulate a Zheng hand shape evaluation merit based on image recognition, and construct a Chinese zither hand shape Dataset (CZ-dataset V3) for the real scene. The experiments show that the method proposed in this paper can effectively improve the recognition accuracy of fine-grained hand shape images and the result is consistent with the evaluation of professional advisors, which realizes the perfect combination of the intelligent image recognition and the hand shape evaluation for Chinese traditional instrument performing.