A multi-scale keypoint estimation network with self-supervision for spinal curvature assessment of idiopathic scoliosis from the imperfect dataset

Idiopathic scoliosis (IS) is a common lifetime disease, which exhibits an obvious deformity of spinal curvature to seriously affect heart and lung function. Accurate radiographic assessment of spinal curvature is vitally important for the clinical diagnosis and treatment planning of idiopathic scoliosis. Deep learning algorithms have been widely adopted to the medical image analysis with the remarkable advancement in computer vision. The automated methods can improve the efficiency of clinical diagnosis to relieve the burden of doctors, which have advantage in dealing with the tedious and repetitive tasks. However, existing methods usually require sufficiently large training datasets with strict annotation, which are costly and laborious especially for medical images. Moreover, the medical images of serious IS always contain the blurry and occlusive parts, which would make the accurate and robust estimation of the spinal curvature more difficult. In this paper, a dot annotation approach is presented to train the spinal curvature assessment model, rather than using strict annotation of IS X-ray images. We develop a multi-scale keypoint estimation network to reduce the requirement for large training datasets, in which the Squeeze-and-Excitation (SE) blocks are incorporated to improve the representational capacity of the model. Then, a self-supervision module is designed to alleviate the blurry and occlusive problem, and we use the two-view radiographic assessments of IS to generate a 3D spinal curvature. Finally, extensive experiments are conducted on a collected clinical dataset, in which we obtain 81.5 AP and the average E-d between the predicted keypoints and the ground truths is 0.43, making an improvement over the mainstream approaches.