Artificial Intelligence Based Handoff Management for Dense WLANs: A Deep Reinforcement Learning Approach

So far, the handoff management involved in the wireless local area network (WLAN) has mainly fallen into the handoff mechanism and the decision algorithm. The traditional handoff mechanism generates noticeable delays during the handoff process, resulting in discontinuity of service, which is more evident in dense WLANs. Inspired by software-defined networking (SDN), prior works put forward many seamless handoff mechanisms to ensure service continuity. With respect to the handoff decision algorithm, when to trigger handoff and which access point to reconnect to, however, are still tricky problems. In this paper, we first design a self-learning architecture applicable to the SDN-based WLAN frameworks. Along with it, we propose DCRQN, a novel handoff management scheme based on deep reinforcement learning, specifically deep $Q$ -network. The proposed scheme enables the network to learn from actual users’ behaviors and network status from scratch, adapting its learning in time-varying dense WLANs. Due to the temporal correlation property, the handoff decision is modeled as the Markov decision process (MDP). In the modeled MDP, the proposed scheme depends on the real-time network statistics at the time of decisions. Moreover, the convolutional neural network and the recurrent neural network are leveraged to extract fine-grained discriminative features. The numerical results through simulation demonstrate that DCRQN can effectively improve the data rate during the handoff process, outperforming the traditional handoff scheme.