AutoTGRL: an automatic text-graph representation learning framework

Text-graph representation learning is a critical and important area of research with extensive applications in natural language processing (NLP). Recently, graph learning models based on graph neural networks (GNNs) have been effectively utilized for encoding text-graph representation for various tasks due to their ability to handle complex structures and capture global information. However, existing text-graph representation learning models are heavily based on the manual design of model architectures and fine-tuning hyperparameters, which is time-consuming and relies on expert knowledge. To address this challenge, we propose an automatic text-graph representation learning (AutoTGRL) framework for transductive and inductive learning-based downstream tasks on text graphs. Specifically, the AutoTGRL framework first builds a general text-graph representation learning model (TGRL model) for text-graph transductive and inductive learning. Then, to enable the automatic design of TGRL models, we propose an automated TGRL model search module. In the automated TGRL model search module, we propose an effective and customized search space called text-graph representation learning (TGRL) search space, which consists of three subspaces, including large-scale embedding strategy space, text-graph representation strategy space, and GNN structure and hyperparameter space, to build TGRL models. We propose to use a search algorithm to search for the best combinations to construct TGRL models to fulfill different downstream tasks from the TGRL search space. To demonstrate the effectiveness AutoTGRL framework, we apply it to text classification, aspect-based sentiment analysis (ABSA), and entity and relation extraction tasks. The extensive experiments demonstrate the superiority of AutoTGRL to design the optimal TGRL models, which outperform the state-of-the-art models over multiple datasets.