Resource-Constraint Network Selection for IoT Under the Unknown and Dynamic Heterogeneous Wireless Environment

We investigate the problem of network selection for the Internet of Things (IoT) to maximize the Quality of Experience (QoE) in a heterogeneous wireless environment. Different from the traditional network access approaches with the assumption that the network state information (NSI) is static and known a priori, a scenario where the NSI of networks is unknown and dynamic to IoT devices is considered. Due to hardware limitations in IoT, the device has a limited resource budget and consumes resources, e.g., the energy, during the process of network access. To maximize the cumulative QoE before resource exhausts, the device should learn and estimate the NSI of networks, and make appropriate decisions to balance the network selection and resource consumption. To address this issue, we formulate the problem as a combination of multiarmed bandit and optimization problems and propose two algorithms NCSA and network selection algorithm (NSA). Moreover, we consider the fact that the device has various traffic types and propose an algorithm MT-NSA. Theoretical analysis shows that the regret of all algorithms has a sublinear relationship with the resource budget. The effectiveness is validated by simulations.