A multi-scale framework based on jigsaw patches and focused label smoothing for bone age assessment

The clinical assessment of bone age is critical. Because it allows the study of the endocrine, genetic, and growth disorders in children's growth. However, clinical assessment of bone age is time-consuming and labor-intensive and is susceptible to observer error. Fully automated bone age assessment (BAA) systems can be a good solution to this problem, assisting or even replacing specialists in bone age assessment. Most of the existing fully automated BAA systems use more discriminative areas (carpal bones, finger bones, etc.) from the whole image as local information input. However, extracting multiple local regions is still complex and time-consuming. Selective extraction of regions is not objective enough and can lose some useful global information. In this paper, we propose a fully automated end-to-end BAA system that requires no additional annotation and no extraction of regions of interest. Specifically, we use a puzzle generator to generate skeletal puzzles containing information at different scales. By inputting smaller-scale skeletal puzzles, the network is forced to mining local fine-grained information first, then input larger-scale skeletal puzzles to obtain coarse-grained information, and finally learn the intact picture to obtain global information. BAA task is suboptimal as a general classification or regression task using single-valued labels due to the high similarity between hand images of similar ages. We propose a new label processing method called focused label smoothing for the BAA task and combine it with expectation regression to obtain a more robust age estimate. We perform adequate experiments on the public dataset from the 2017 Pediatric Bone Age Challenge organized by the Radiological Society of North America and we achieve great experimental performance compared to the method without manual annotation and competitive results with the method using additional manual annotation.