DQN based Reinforcement Learning Algorithm for Scheduling Workflows in the Cloud

The 9th International Symposium on Computational Intelligence and Industrial Applications (ISCIIA2020) CITIC Jingling Hotel Beijing, Beijing, China, Oct.31-Nov.3, 2020 1 Abstract: With more and more scientific and social media applications, the amount of data is growing exponentially. Any type of computing applications, such as data calculation or analysis, can be described as workflows. Cloud computing provides an effective platform for executing large and complex workflow applications conveniently and cheaply through its delivering internet-based services as a pay-as-you-go model. However, the performance of workflow scheduler directly affects the Quality of Service (QoS) of the cloud users, and how to efficiently allocate the heterogeneous cloud resources to execute workflows still faces big challenges. In this work, an improved Deep Q Network (DQN)-based reinforcement learning (RL) algorithm for workflow scheduling is developed to optimize dual objectives like makespan and cost simultaneously. First, we test the performance of DQN and Actor-critic (AC) based RL algorithm in scheduling workflows respectively, then modify the reward function for the DQN algorithm to improve its convergence and universality for optimization problems. Extensive experiments are conducted to verify our approach and the simulation results show that the proposed algorithm can minimize both makespan and cost, as well as adjust user preference for the specific optimization objective and accordingly increase the diversity of generated scheduling schemes.