A Multi-Layer Deep Reinforcement Learning Approach for Joint Task Offloading and Scheduling in Vehicular Edge Networks.

Mobile Edge Computing (MEC) is emerging as a promising computing scheme to support AI-enabled applications in vehicular networks, via offloading some tasks to edge servers deployed on Road Side Units (RSUs) that approximates to vehicles. In this work, we consider a general vehicular edge network (VEN), where each vehicle can offload tasks to edge servers or cloud server via a vehicle-to-infrastructure (V2I) transmission link, or to other vehicles via a vehicle-to-vehicle (V2V) transmission link. To characterize different task flows in different transmission links or computing servers, we introduce a V2V transmission queue, a V2V transmission queue, and a local computation queue for each vehicle, and an edge computation queue for each edge server. In such a queue-based VEN, we focus on the joint task offloading and scheduling problem for vehicles, which consists of (i) offloading problem, i.e., whether to offload tasks, and (ii) scheduling problem, i.e., where and how to offload tasks. The problem is challenging due to the online and asynchronous offloading and scheduling decisions for each task. We propose a Multi-layer Deep Reinforcement Learning (DRL) approach, where each vehicle trains three neural networks (called agents) to make different layers' decisions: (i) offloading agent, determining whether to offload each task when tasks arrive, and (ii) V2I and V2V scheduling agents, determining where and how to offloading each task in V2I and V2V transmission queues, respectively. We provide the detailed algorithm design of each agent by using the Double Deep Q-Network (DDQN) approach. Simulation results show that our proposed multi-layer DRL approach outperforms the existing baseline approaches in terms of both the cost performance and the convergence speed.