Data Analysis-Oriented Stochastic Scheduling for Cost Efficient Resource Allocation in NFV Based MEC Network

Recently, there has been increasing interest in data analysis services based on MapReduce framework, which is a programming model and an associated implementation for processing and generating Big Data sets with a parallel, distributed algorithm on a cluster. Due to limited resource configuration, it is hard for end users to deal with computing-intensive data analysis tasks by themselves. Therefore, outsourcing these tasks to nearby edge computing servers (i.e., multi-access edge computing) becomes a key step towards the next generation mobile networks. However, new challenges arise for designing cost efficient distributed system by jointly managing communication and computing resources with unpredictable tasks arrival mode in complex spatial and temporal domains. In this paper, based on the idea of network function virtualization (NFV), we distribute a series of operation functions of MapReduce framework on distributed edge computing servers to execute data analysis service and investigate how to dynamically minimize the overall operational cost with joint consideration of workflow scheduling, network function management, resource allocation, and system stabilization. Our method leverages Lyapunov optimization technique to make a tradeoff between the queue backlog and overall cost without using future information. Simulation results show that our method can effectively reduce computing and communication cost while guaranteeing quality of service for end users.