EvoNN: a customizable evolutionary neural network with heterogenous activation functions.

While some attention has been given to the process of using Evolutionary Computation (EC) to optimize the activation functions within hidden layers, available activation function sets have always been hard coded by the developers, and were immutable by users. In this paper, we present EvoNN. While many other Neuro-evolution based tools or algorithms focus primarily on the evolution of either Neural Network (NN) architecture, or its weights, EvoNN focuses on simultaneous evolution of weights and the activation functions within hidden layers. The main novely offered by EvoNN lies in that users can provide additional activation functions to the EvoNN system to be employed as part of the "alphabet" of available functions. This feature gives users a greater degree of flexibility over which functions the evolutionary optimizer can utilize. We employ a set of three test cases where we compare EvoNN to a standard NN, and observe encouraging results showing a superior performance of the EvoNN system. We also observe this increase in performance comes at the cost of additional run time, but note that for some applications, this can be a worthwhile trade-off.