Leveraging Digital Cousins for Ensemble Q-Learning in Large-Scale Wireless Networks
IEEE Transactions on Signal Processing(2024)
摘要
Optimizing large-scale wireless networks, including optimal resource
management, power allocation, and throughput maximization, is inherently
challenging due to their non-observable system dynamics and heterogeneous and
complex nature. Herein, a novel ensemble Q-learning algorithm that addresses
the performance and complexity challenges of the traditional Q-learning
algorithm for optimizing wireless networks is presented. Ensemble learning with
synthetic Markov Decision Processes is tailored to wireless networks via new
models for approximating large state-space observable wireless networks. In
particular, digital cousins are proposed as an extension of the traditional
digital twin concept wherein multiple Q-learning algorithms on multiple
synthetic Markovian environments are run in parallel and their outputs are
fused into a single Q-function. Convergence analyses of key statistics and
Q-functions and derivations of upper bounds on the estimation bias and variance
are provided. Numerical results across a variety of real-world wireless
networks show that the proposed algorithm can achieve up to 50
policy error with up to 40
reinforcement learning algorithms. It is also shown that theoretical results
properly predict trends in the experimental results.
更多查看译文
关键词
Reinforcement learning,Q-learning,Markov decision processes,wireless networks
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要