Neuron-level LLM Patching for Code Generation
arxiv(2023)
摘要
Large Language Models (LLMs) have found widespread adoption in software
engineering, particularly in code generation tasks. However, updating these
models with new knowledge can be prohibitively expensive, yet it is essential
for maximizing their utility. In this paper, we propose a novel and effective
model editing approach, MENT, to patch LLMs in coding tasks.
MENT is effective, efficient, and reliable. It can correct a neural
model by patching 1 or 2 neurons. As the pioneer work on neuron-level model
editing of generative models, we formalize the editing process and introduce
the involved concepts. Besides, we also introduce new measures to evaluate its
generalization ability, and build a benchmark for further study. Our approach
is evaluated on three coding tasks, including API-seq recommendation,
line-level code generation, and pseudocode-to-code transaction. The
experimental results show that the proposed approach outperforms the state of
the arts by a significant margin in both effectiveness and efficiency measures.
In addition, we demonstrate the usages of MENT for LLM reasoning in
software engineering. By editing LLM knowledge, the directly or indirectly
dependent behaviors of API invocation in the chain-of-thought will change
accordingly. It explained the significance of repairing LLMs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要