A partial convolution generative adversarial network for lesion synthesis and enhanced liver tumor segmentation

Lesion segmentation is critical for clinicians to accurately stage the disease and determine treatment strategy. Deep learning based automatic segmentation can improve both the segmentation efficiency and accuracy. However, training a robust deep learning segmentation model requires sufficient training examples with sufficient diversity in lesion location and lesion size. This study is to develop a deep learning framework for generation of synthetic lesions with various locations and sizes that can be included in the training dataset to enhance the lesion segmentation performance. The lesion synthesis network is a modified generative adversarial network (GAN). Specifically, we innovated a partial convolution strategy to construct a U-Net-like generator. The discriminator is designed using Wasserstein GAN with gradient penalty and spectral normalization. A mask generation method based on principal component analysis (PCA) was developed to model various lesion shapes. The generated masks are then converted into liver lesions through a lesion synthesis network. The lesion synthesis framework was evaluated for lesion textures, and the synthetic lesions were used to train a lesion segmentation network to further validate the effectiveness of the lesion synthesis framework. All the networks are trained and tested on the LITS public dataset. Our experiments demonstrate that the synthetic lesions generated by our approach have very similar distributions for the two parameters, GLCM-energy and GLCM-correlation. Including the synthetic lesions in the segmentation network improved the segmentation dice performance from 67.3% to 71.4%. Meanwhile, the precision and sensitivity for lesion segmentation were improved from 74.6% to 76.0% and 66.1% to 70.9%, respectively. The proposed lesion synthesis approach outperforms the other two existing approaches. Including the synthetic lesion data into the training dataset significantly improves the segmentation performance.