Multi-Hyperedge Hypergraph for Group Activity Recognition

Group activity recognition aims to identify group activities from the videos. Most of the previous methods focus on modeling between individuals (one-to- one), which ignores the fact that a single individual's behavior may be jointly determined by multiple individual behaviors (many-to-one). For this reason, we propose a Multi-Hyperedge Hypergraph (MHH) to capture high-order relationships between multiple people. Specifically, we build three different types of hyperedges on the hypergraph structure. Each hyperedge can accommodate the characteristics of multiple nodes to capture different types of high-order relationships between nodes. Then, we use the late fusion method to fuse the three features to further enhance the overall behavioral representation. Finally, we perform a series of experiments on two of the most widely used benchmarks in group activity recognition, which have proved the effectiveness of MHH. More importantly, as far as we know, this is the first case of using a hypergraph structure for group activity recognition.