Characterizing & Exploring Deep CNN Representations Using Factorization ∗

Deep neural networks have gained enormous popularity in machine learning and data science alike, and rightfully so, since they have demonstrated impeccable performance in a variety of supervised learning tasks, especially a number of computer vision problems. Albeit very successful in providing accurate classifications, deep neural networks are notorious for being hard to interpret, explain, and debug, a problem amplified by their increasing complexity. This is an extremely challenging problem and the jury is still out on whether it can be solved in its entirety. Here, we propose a novel factorization framework, aiming to answer the following questions: Given an already trained deep neural network, and a set of test inputs, how can we gain insight into how those inputs interact with different layers of the neural network? Furthermore, can we characterize a given deep neural network based on its observed behavior on different inputs? The proposed approach will give a more flexible yet still interpetable mechanism for understanding and interacting with deep networks. 1 Overview of Proposed Method & Key Results The key idea behind our proposal, shown in Figure 1, is the following: we jointly factorize the raw inputs to the deep neural network and the outputs of each layer, to the same low-dimensional space. Intuitively, such a factorization will seek to identify commonalities in different parts of the raw input and how those are reflected and processed within the network. For instance, if we are dealing with a Deep CNN that is classifying handwritten digits, such a joint latent factor will seek to identify different shapes or patterns that are common in a variety of input classes and identify correlations on how different layers behave collectively for such high-level latent patterns. Here, we present a proof-of-concept approach. Suppose we have an already trained deep CNN and we have a separate hold-out validation set. If we feed this validation set to the network, we express a coupled factorization of the raw validation inputs and the intermediate outputs of the activation layers as shown in Figure 1.