Multi-Agent Reinforcement Learning with Bidding for Segmenting Action Sequences

The paper presents a bidding approach for developing multi-agent reinforcement learning systems that are made up of a coalition of agents. We focus on learning to segment action sequences in sequential decision tasks through a bidding process that is based on reinforcements received during task execution. The approach segments sequences (and divides segments up among agents) to reduce non-Markovian temporal dependencies, to facilitate the learning of the overall task. Notably, our approach does not rely on a priori domain knowledge or a priori domain-specific structures. Thus the approach deals with a more difficult problem compared with most existing hierarchical learning models. Initial experiments demonstrate the basic promise of this approach.