Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement
CoRR(2024)
摘要
The increasing demand for personalized interactions with large language
models (LLMs) calls for the development of methodologies capable of accurately
and efficiently identifying user opinions and preferences. Retrieval
augmentation emerges as an effective strategy, as it can accommodate a vast
number of users without the costs from fine-tuning. Existing research, however,
has largely focused on enhancing the retrieval stage and devoted limited
exploration toward optimizing the representation of the database, a crucial
aspect for tasks such as personalization. In this work, we examine the problem
from a novel angle, focusing on how data can be better represented for more
efficient retrieval in the context of LLM customization. To tackle this
challenge, we introduce Persona-DB, a simple yet effective framework consisting
of a hierarchical construction process to improve generalization across task
contexts and collaborative refinement to effectively bridge knowledge gaps
among users. In the task of response forecasting, Persona-DB demonstrates
superior efficiency in maintaining accuracy with a significantly reduced
retrieval size, a critical advantage in scenarios with extensive histories or
limited context windows. Our experiments also indicate a marked improvement of
over 15
Furthermore, our analysis reveals the increasing importance of collaborative
knowledge as the retrieval capacity expands.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要