Realization of Autoencoders by Kernel Methods.

An autoencoder is a neural network to realize an identity mapping with hidden layers of a relatively small number of nodes. However, the role of the hidden layers is not clear because they are automatically determined through the learning process. We propose to realize autoencoders by a set of linear combinations of kernels instead of neural networks. In this framework, the roles of the encoder and/or decoder, are explicitly determined by a user. We show that it is possible to replace almost every type of autoencoders realized by neural networks with this approach. We compare the pros and cons of this kernel approach and the neural network approach.