Behavioral model summarisation for other agents under uncertainty

Modelling other agents is one of the fundamental research challenges in multiagent decision making under uncertainty. A subject agent expects to optimise its own decisions by correctly modelling what other agents are going to act when they share a common environment and their actions have a joint impact in the transition of environmental states and their individual rewards. However, a true model of the other agent is often unknown due to limited prior knowledge, which particularly occurs to a competitive agent setting. Consequently, the subject agent needs to consider a large number of potential models (the number is infinite in theory) for the other agent, which turns to be computationally hard in its decision making process. A feasible solution is to select a subset of potential models that are representative of all the models for the other agent. In this article, we investigate such a model selection given some known behavioral stereotypes ascribed to the other agents. We propose a new measurement to diversify the selected models under the constraint of the number of models (K models). We formulate the top-K model selection as a sub-modular function optimisation problem and develop a greedy algorithm to obtain high-quality solutions. We develop the proposed model selection techniques in a well-recognised multiagent decision model, namely interactive dynamic influence diagrams, that represents individual agents’ decision making process with the consideration of other agents. By doing this, we can test the performance of the new model selection techniques over two problem domains and provide experimental results in support. Hence, our research in this article contributes to a new model selection technique for modelling other agents therefore improving decision quality of a subject agent interacting with the other agents. It also inspires a creative usage of sub-modular function optimisation in multiagent decision making.