Updating Language Models with Unstructured Facts: Towards Practical Knowledge Editing
CoRR(2024)
摘要
Knowledge editing aims to inject knowledge updates into language models to
keep them correct and up-to-date. However, its current evaluation strategies
are notably impractical: they solely update with well-curated structured facts
(triplets with subjects, relations, and objects), whereas real-world knowledge
updates commonly emerge in unstructured texts like news articles. In this
paper, we propose a new benchmark, Unstructured Knowledge Editing (UKE). It
evaluates editing performance directly using unstructured texts as knowledge
updates, termed unstructured facts. Hence UKE avoids the laborious construction
of structured facts and enables efficient and responsive knowledge editing,
becoming a more practical benchmark. We conduct extensive experiments on newly
built datasets and demonstrate that UKE poses a significant challenge to
state-of-the-art knowledge editing methods, resulting in their critical
performance declines. We further show that this challenge persists even if we
extract triplets as structured facts. Our analysis discloses key insights to
motivate future research in UKE for more practical knowledge editing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要