PerLTQA: A Personal Long-Term Memory Dataset for Memory Classification, Retrieval, and Synthesis in Question Answering
CoRR(2024)
摘要
Long-term memory plays a critical role in personal interaction, considering
long-term memory can better leverage world knowledge, historical information,
and preferences in dialogues. Our research introduces PerLTQA, an innovative QA
dataset that combines semantic and episodic memories, including world
knowledge, profiles, social relationships, events, and dialogues. This dataset
is collected to investigate the use of personalized memories, focusing on
social interactions and events in the QA task. PerLTQA features two types of
memory and a comprehensive benchmark of 8,593 questions for 30 characters,
facilitating the exploration and application of personalized memories in Large
Language Models (LLMs). Based on PerLTQA, we propose a novel framework for
memory integration and generation, consisting of three main components: Memory
Classification, Memory Retrieval, and Memory Synthesis. We evaluate this
framework using five LLMs and three retrievers. Experimental results
demonstrate that BERT-based classification models significantly outperform LLMs
such as ChatGLM3 and ChatGPT in the memory classification task. Furthermore,
our study highlights the importance of effective memory integration in the QA
task.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要