Impact of Distributed Training on Mask R-CNN Model Performance for Image Segmentation

The paper compares and discusses the impact of distributed deep learning training using the Horovod framework on the performance metrics of image segmentation model trained using the Mask R-CNN (Region-based Convolutional Neural Network) architecture in comparison with the baseline metrics obtained from running the training on a single node. Image segmentation is a complex problem in comparison with image classification and the fluctuations introduced due to the distributed gradient computations while running the training distributed is less pronounced on simpler problems like image classification while it is more pronounced for image segmentation where we measure the performance at pixel level. It is important to know how the latest algorithms and distribution frameworks perform when training the model on distributed compute with parallel gradient calculations, adjusted learning rates and averaged neural network weight updates. The paper also aimed to understand if the effective batch size per training iteration while performing the training distributed did not negatively impact the ability of the model to generalize when increasing the batch size as indicated in the previous studies. The distributed training with Horovod was performed using the Azure machine learning service on scalable cloud compute cluster. The aim of the study was not comparison in-terms of the execution time which has been well established on previous studies but on understanding the impact on the quality of the predictions measured with standard validation metrics for image segmentation using the ISIC (International Skin Imaging Collaboration) 2018 Skin Lesion image dataset.