Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models
CoRR(2024)
摘要
The aim of this study is to investigate Machine Unlearning (MU), a burgeoning
field focused on addressing concerns related to neural models inadvertently
retaining personal or sensitive data. Here, a novel approach is introduced to
achieve precise and selective forgetting within language models. Unlike
previous methodologies that adopt completely opposing training objectives, this
approach aims to mitigate adverse effects on language model performance,
particularly in generation tasks. Furthermore, two innovative evaluation
metrics are proposed: Sensitive Information Extraction Likelihood (S-EL) and
Sensitive Information Memory Accuracy (S-MA), designed to gauge the
effectiveness of sensitive information elimination. To reinforce the forgetting
framework, an effective method for annotating sensitive scopes is presented,
involving both online and offline strategies. The online selection mechanism
leverages language probability scores to ensure computational efficiency, while
the offline annotation entails a robust two-stage process based on Large
Language Models (LLMs).
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要