The familiarity hypothesis: Explaining the behavior of deep open set methods

•Open set detection using the maximum logit score of a softmax classifier matches the current state of the art.•Familiarity Hypothesis: The Max Logit method detects the absence of familiarity rather than the presence of novelty.•The reduced activity of positively-weighted object-relevant features accounts for most of the Max Logit score.