UAV Control Optimization via Decentralized Markov Decision Processes

Unmanned aerial vehicle (UAV) swarm control has applications including target tracking, surveillance, terrain mapping, and precision agriculture. Decentralized control methods are particularly useful when the swarm is large, as centralized methods (a single command center controlling the UAVs) suffer from exponential computational complexity, i.e., the computing time to obtain the optimal control for the UAVs grow exponentially with the number of UAVs in the swarm in centralized approaches. Although many centralized control methods exist, literature lacks decentralized control frameworks with broad applicability. To address this knowledge gap, we present a novel decentralized UAV swarm control strategy using a decision-theoretic framework called decentralized Markov decision process (Dec-MDP). We build these control strategies in the context of two case studies: a) swarm formation control problem; b) swarm control for multitarget tracking. As most decision theoretic formulations suffer from the curse of dimensionality, we adapt an approximate dynamic programming method called nominal belief-state optimization (NBO) to solve the decentralized control problems approximately in both the case studies. In the formation control case study, the objective is to drive the swarm from a geographical region to another geographical region where the swarm must form a certain geometrical shape (e.g., selected location on the surface of a sphere). The motivation for studying such problems comes from data fusion applications with UAV swarms where the fusion performance depends on the strategic relative separation of the UAVs from each other. In the target tracking case study, the objective is the control the motion of the UAVs in a decentralized manner while maximizing the overall target tracking performance. Motivation for this case study comes from the surveillance applications using UAV swarms.