WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing
CoRR(2024)
摘要
Knowledge editing aims to rectify inaccuracies in large language models
(LLMs) without costly retraining for outdated or erroneous knowledge. However,
current knowledge editing methods primarily focus on single editing, failing to
meet the requirements for lifelong editing. In this paper, lifelong editing is
synonymous with lifelong knowledge editing. This study reveals a performance
degradation encountered by knowledge editing in lifelong editing, characterized
by toxicity buildup and toxicity flash, with the primary cause identified as
pattern unmatch. We introduce a knowledge editing approach named WilKE, which
selects editing layer based on the pattern matching degree of editing knowledge
across different layers. Experimental results demonstrate that, in lifelong
editing, WilKE exhibits an average improvement of 46.2% and 67.8% on editing
GPT2-XL and GPT-J relative to state-of-the-art knowledge editing methods.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要