An Experimental Approach For Evaluating Superpixel's Consistency Over 2D Gaussian Blur and Impulse Noise Using Jaccard Similarity Coefficient

This article proposes a rigorous method to assess the consistency of superpixels for different superpixel segmentation algorithms. The proposed method extracts the superpixels that remain unchanged over certain levels of noise by adopting the Jaccard Similarity Coefficient (JSC). Technically, we developed a measure of Jaccard similarity for superpixel segmentation algorithms to compare the similarity between sets of superpixels (original and noisy). The algorithm calls the superpixel segmentation algorithm to generate the superpixel results of the original images and saves their boundary masks and labels. It then applies varying degrees of noise to the images and produces the superpixels results, and the process is repeated for four levels with increased noise value at each iteration. We chose 2D Gaussian Blur, Impulse Noise and a combination of both to corrupt the images. The proposed algorithm generates similarity indices of superpixels (original and noisy) using Jaccard Similarity (JS). To be categorized as a consistent superpixel, the similarity index must meet a predefined coefficient threshold (τ) of JSC. The superpixels consistency of four different superpixel segmentation algorithms including Bilateral geodesic distance (BGD), Flooding based superpixels generation (FBS), superpixels via geodesic distance (GDS), and Turbopixel (TP) are evaluated. Precisely, the experimental results demonstrated that no single algorithm was able to yield an optimal outcome and failed to maintain consistent superpixels at each level of noise. Conclusively, more robust superpixel algorithms must be developed to solve such problems effectively.