Dynamic Optimization of Provider-Based Scheduling for HPC Workloads
2023 International Conference on Software, Telecommunications and Computer Networks (SoftCOM)(2023)
摘要
The vast array of cloud providers present in today’s market proffer a suite of High-Performance Computing (HPC) services. However, these offerings are characterized by significant variations in execution times and cost structures. Consequently, selecting the optimal cloud provider and configuring the features of the chosen computing instance (e.g. virtual machines) proves to be a challenging task for users intending to execute HPC workloads. This paper introduces a novel component designed for effortless integration with existing HPC scheduling systems. This module’s primary function is to facilitate the selection of the most appropriate cloud provider for each distinct job, thereby empowering dynamic and adaptive cost-minimization strategies. Through the application of data augmentation techniques and the employment of Continuous Machine Learning, the system is endowed with the capability to operate efficiently with cloud providers that have not been previously utilized. Furthermore, it is capable of tracking the evolution of jobs over time. Our results show that this component can achieve consistent economic savings, based on the quality of the data used in the training phase.
更多查看译文
关键词
cloud computing,hpc,machine learning
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要