Q-Learning-Based Spectrum Access for Multimedia Transmission Over Cognitive Radio Networks

In order to meet the dramatic wireless bandwidth demands of emerging multimedia applications, cognitive radio has been proposed as one of promising solutions to improve the spectrum efficiency. This article aims at pursuing high spectrum efficiency via accessing the idle spectrum intelligently without information exchange among users. Different from infrastructure-based wireless networks, users in cognitive radio networks tend to compete with each other to access limited idle spectrum, thus leading to a dynamically heterogeneous radio environment. In this article, a Q-learning based spectrum access scheme is proposed to adaptively allocate multimedia data over multiple idle spectrum holes. Taking into consideration the rigorous delay and throughput performance requirements of multimedia applications, we integrate these two indicators into the definition of reward function in the proposed Q-learning algorithm. The simulation results show that the proposed scheme can quickly converge to a stable state in terms of throughput, power efficiency, and collision probability. Furthermore, the proposed learning rate adjustment strategy makes the performance of the spectrum access algorithm converge the quickest and only consumes 78% time to achieve the targeted collision probability, i.e., 0.1, compared with two other typical parameter adjustment strategies.