On Optimizing the Structure of Neural Networks Through a Compact Codification of Their Architecture.

Neural networks stand out in Artificial Intelligence for their capacity of being applied to multiple challenging tasks such as image classification. However, designing a neural network to address a particular problem is also a demanding task that requires expertise and time-consuming trial-and-error stages. The design of methods to automate the designing of neural networks define a research field that generally relies on different optimization algorithms, such as population meta-heuristics. This work studies utilizing Teaching-Learning-based Optimization (TLBO), which had not been used before for this purpose up to the authors’ knowledge. It is widespread and does not have specific parameters. Besides, it would be compatible with deep neural network design, i.e., architectures with many layers, due to its conception as a large-scale optimizer. A new encoding scheme has been proposed to make this continuous optimizer compatible with neural network design. This method, which is of general application, i.e., not linked to TLBO, can represent different network architectures with a plain vector of real values. A compatible objective function that links the optimizer and the representation of solutions has also been developed. The performance of this framework has been studied by addressing the design of an image classification neural network based on the CIFAR-10 dataset. The achieved result outperforms the initial solutions designed by humans after letting them evolve.