Deep reinforcement learning-based contention window optimization for IEEE 802.11 networks

Abstract This study focuses on optimizing the contention window (CW) in IEEE 802.11 networks using deep reinforcement learning (DRL) to enhance the effectiveness of the contention mechanism. Recent research has employed a deep Q-learning network (DQN) as one type of DRL for CW size selection tasks to maximize network throughput. However, a notable limitation of DQN is the substantial overestimation error, which means the predicted reward value significantly deviates from the actual value. To address this issue, our study introduces the smart exponential-threshold-linear with double deep Q-learning network (SETL-DDQN) in a wireless networks scenario, with the aim to mitigate the overestimation error via the CW threshold size optimization with the help of a DDQN-based approach during the learning phase. We experimented with our proposed SETL-DDQN in both static and dynamic scenarios and conducted an analysis to solve the overestimation problem, then enhance the long-term simulation stability. Our experimental results demonstrate that SETL-DDQN achieves more efficient packet transmissions than related existing mechanisms.