Fact Finder – Enhancing Domain Expertise of Large Language Models by Incorporating Knowledge Graphs
arxiv(2024)
摘要
Recent advancements in Large Language Models (LLMs) have showcased their
proficiency in answering natural language queries. However, their effectiveness
is hindered by limited domain-specific knowledge, raising concerns about the
reliability of their responses. We introduce a hybrid system that augments LLMs
with domain-specific knowledge graphs (KGs), thereby aiming to enhance factual
correctness using a KG-based retrieval approach. We focus on a medical KG to
demonstrate our methodology, which includes (1) pre-processing, (2) Cypher
query generation, (3) Cypher query processing, (4) KG retrieval, and (5)
LLM-enhanced response generation. We evaluate our system on a curated dataset
of 69 samples, achieving a precision of 78% in retrieving correct KG nodes.
Our findings indicate that the hybrid system surpasses a standalone LLM in
accuracy and completeness, as verified by an LLM-as-a-Judge evaluation method.
This positions the system as a promising tool for applications that demand
factual correctness and completeness, such as target identification – a
critical process in pinpointing biological entities for disease treatment or
crop enhancement. Moreover, its intuitive search interface and ability to
provide accurate responses within seconds make it well-suited for
time-sensitive, precision-focused research contexts. We publish the source code
together with the dataset and the prompt templates used.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要