Multimodal reasoning for nutrition and human health via knowledge graph embedding.

The established links between nutrition and human health are widely acknowledged. Dietary nutrients play a crucial role in regulating gut microbial communities, influencing various human diseases. With a growing number of related studies, there’s a need to systematically organize these associations for coherent knowledge reasoning. However, due to the diverse and extensive nature of the knowledge landscape, significant challenges persist. To address this, we propose an approach using multimodal data and knowledge embeddings for effective knowledge reasoning in nutrition and human health. We create a comprehensive knowledge graph, KG4NH, covering dietary nutrition, gut microbiota, and human diseases. To ensure efficient knowledge representation, we employ knowledge embedding techniques to develop modality-specific encoders for structure, category, and description. Additionally, we introduce a mul-timodal fusion method to capture shared information across modalities. Our experimental results demonstrate the superiority of our approach over other state-of-the-art methods.