Understanding trained CNNs by indexing neuron selectivity.

•A framework to Analyze and Visualize a trained CNN by dissecting Individual Neurons.•Review, Comparison and Classification of previous Works related to CNN visualization.•A Color selectivity index of a neuron, extendible to other low-level attributes.•A Class selectivity index of a neuron, extendible to other high-level attributes.•Global contribution is to give some light to the black-box nature of trained CNNs.