Predictive control of linear discrete-time Markovian jump systems by learning recurrent patterns

Incorporating pattern-learning for prediction (PLP) in many discrete-time or discrete-event systems allows for computation-efficient controller design by memorizing patterns to schedule control policies based on their future occurrences. In this paper, we demonstrate the effect of PLP by designing a controller architecture for a class of linear Markovian jump systems (MJSs) where the aforementioned “patterns” correspond to finite-length sequences of modes. In our analysis of recurrent patterns, we use martingale theory to derive closed-form solutions to quantities pertaining to the occurrence of patterns: (1) the expected minimum occurrence time of any pattern from some predefined collection, (2) the probability of a pattern being the first to occur among the collection. To make our method applicable to real-world dynamics, we make two extensions to common assumptions in prior pattern-occurrence literature. First, the distribution of the mode process is unknown, and second, the true realization of the mode process is not observable. As demonstration, we consider fault-tolerant control of a dynamic topology-switching network, and empirically compare PLP to two controllers without PLP: a baseline based on the novel System Level Synthesis (SLS) approach and a topology-robust extension of the SLS baseline. We show that PLP is able to reject disturbances just as effectively as the topology-robust controller at reduced computation time and control effort. We discuss several important tradeoffs, such as the size of the pattern collection and the system scale versus the accuracy of the mode predictions, which show how different PLP implementations affect stabilization and runtime performance.