An In-depth Benchmarking of Evolutionary and Swarm Intelligence Algorithms for Autoscaling Parameter Sweep Applications on Public Clouds

Many important computational applications in science, engineering, industry, and technology are represented by PSE (parameter sweep experiment) applications. These applications involve a large number of resource-intensive and independent computational tasks. Because of this, cloud autoscaling approaches have been proposed to execute PSE applications on public cloud environments that offer instances of different VM (virtual machine) types, under a pay-per-use scheme, to execute diverse applications. One of the most recent approaches is the autoscaler MOEA (multiobjective evolutive algorithm), which is based on the multiobjective evolutionary algorithm NSGA-II (nondominated sorting genetic algorithm II). MOEA considers on-demand and spot VM instances and three optimization objectives relevant for users: minimizing the computing time, monetary cost, and spot instance interruptions of the application’s execution. However, MOEA’s performance regarding these optimization objectives depends significantly on the optimization algorithm used. It has been shown recently that MOEA’s performance improves considerably when NSGA-II is replaced by a more recent algorithm named NSGA-III. In this paper, we analyze the incorporation of other multiobjective optimization algorithms into MOEA to enhance the performance of this autoscaler. First, we consider three multiobjective optimization algorithms named E-NSGA-III (extreme NSGA-III), SMS-EMOA (S-metric selection evolutionary multiobjective optimization algorithm), and SMPSO (speed-constrained multiobjective particle swarm optimization), which have behavioral differences with NSGA-III. Then, we evaluate the performance of MOEA with each of these algorithms, considering the three optimization objectives, on four real-world PSE applications from the meteorology and molecular dynamics areas, considering different application sizes. To do that, we use the well-known CloudSim simulator and consider different VM types available in Amazon EC2. Finally, we analyze the obtained performance results, which show that MOEA with E-NSGA-III arises as the best alternative, reaching better and significant savings in terms of computing time (10%–17%), monetary cost (10%–40%), and spot instance interruptions (33%–100%).