Graphs, Entities, and Step Mixture for Enriching Graph Representation

Graph neural networks have shown promising results on representing and analyzing diverse graph-structured data, such as social networks, traffic flow, drug discovery, and recommendation systems. Existing approaches for graph neural networks typically suffer from the oversmoothing issue that results in indistinguishable node representation, as recursive and simultaneous neighborhood aggregation deepens. Also, most methods focus on transductive scenarios that are limited to fixed graphs, which do not generalize properly to unseen graphs. To address these issues, we propose a novel graph neural network that considers both edge-based neighborhood relationships and node-based entity features with multiple steps, i.e. Graph Entities with Step Mixture via random walk (GESM). GESM employs a mixture of various steps through random walk to alleviate the oversmoothing problem, an attention mechanism to dynamically reflect interrelations depending on node information, and structure-based regularization to enhance embedding representation. With intensive experiments, we show that our proposed GESM achieves state-of-the-art or comparable performances on eight benchmark graph datasets in both transductive and inductive learning tasks. Furthermore, we empirically demonstrate the superiority of our method on the oversmoothing issue with rich graph representations. Our source code is available at https://github.com/ShinKyuY/GESM.