Detecting Anomalies in Image Classification by Means of Semantic Relationships

This paper presents a semantic anomaly detection method (SAD) to detect anomalies in the predictions of any pixelwise semantic segmentation algorithm. This semantic information (e.g., relative positions and sizes of all the object pairs in an image), learned from the training set and stored in a knowledge base as configuration rules, allows the detection of potential misclassifications in the baseline model predictions. Our approach highlights the objects which are not consistent with the contextual information in the knowledge base. It also provides an interpretable motivation for the detected anomaly, based on the semantic information provided by the configuration rules.