A Fitness Landscape Analysis Approach for Reinforcement Learning in the Control of the Coupled Inverted Pendulum Task.

Fitness Landscape Analysis (FLA) for loss/gain functions for Machine Learning is an emerging research trend in Computational Intelligence that offered an alternative view on how learning algorithms work and should be designed. The vast majority of these recent studies investigate supervised learning whereas reinforcement learning remains so far nearly unaddressed. This paper performs a FLA on the reinforcement learning of a deep neural network for a simulated robot control task and focuses on ruggedness and neutrality. Two configurations of the physical system under investigation are considered and studied separately to highlight differences and similarities. Furthermore, the results of the performed FLA are put into relation with performance of the learning algorithm with the aim of achieving an understanding of most suitable parameter setting. Numerical results indicate a correlation between ruggedness and exploration to enable more optimal reinforcement learning. In the presence of high ruggedness the algorithm displays its best performance when the control parameters are set to enable a high degree of exploration. Conversely, when the landscape appears less rugged a less exploratory behaviour seems to contribute to the best performance of the learning algorithm.