Decentralized Anomaly Detection via Deep Multi-Agent Reinforcement Learning

We consider a decentralized anomaly detection problem, where multiple agents collaborate to localize a single anomalous process among a finite number $M$ of processes. At each time, a subset of the processes can be observed by each agent, and the observations from each chosen process follow two different distributions, depending on whether the process is normal or abnormal. The communication channel between agents is rate-limited. The objective is a sequential search strategy that minimizes the Bayes risk, consisting of the sampling cost, and the joint terminal cost among the agents. This problem generalizes previous studies that considered anomaly detection by a single detector. We develop a novel algorithm based on deep multi-agent reinforcement learning optimization to minimize the Bayes risk. Numerical experiments demonstrate the ability of the algorithm to learn good policies in this challenging problem, and improve the single-agent performance by applying the proposed multi-agent collaborative learning method.