Comparing Forgetting A lgorithms for A rtificial Episodic Memory Systems

Episodic memory is essential for human cognition and has also proven necessary for some intelligent agents. The size of an episodic store grows over time unless some forgetting mechanism is in place to keep it in check. In this research, we investigate the effect that different forgetting mechanisms have on the episodic memory performance. We compare three different forgetting algorithms using three distinct episodic memory architectures in a domain that is designed to highlight the differences between then. Our results show that the choice of forgetting algorithm has a significant impact on overall agent performance. 1 IN T R O DU C T I O N Any intelligent agent must have a memory that it uses to store knowledge that it applies to a domain. At the most fundamental level this contains basic procedural knowledge of how to perform its function. Most agents also possess semantic knowledge, or knowledge of facts about the agent’s domain. Recently, there has been a growing interest in episodic knowledge or memory of specific past events [4, 5, 8, 13, 16, and 17]. Episodic memory is a history, or chronological record of specific events, called episodes. The ability to remember past events enables a system to improve its performance as well as its competence. In this research, we examine and compare three simple forgetting algorithms for episodic memory:  Forgetting a randomly selected memory in the episodic store.  Forgetting the oldest memory in the episodic store.  Forgetting the memory with a lowest activation value that is calculated based upon the frequency and recency of use. We conducted our research using three distinct, general-purpose artificial episodic memory systems that have already been applied to a variety of different problems [8, 13, and 17]. Each of the nine unique combinations of system and algorithm has been applied to a domain that we have specifically designed to highlight the effectiveness of the agent. Our results show that while the random and oldest forgetting algorithms yield similar behaviour, an activation-based forgetting algorithm can be more effective at selecting memories with lower utility and, thus, improving the agent’s performance. 1 Dept. of Electrical Engineering and Computer Science, Univ. of Portland, Portland, Oregon USA. Email: {nuxoll, wangn10}@up.edu. 2 Dept. of Computer Science, Univ. of Texas at Austin, Austin, TX, USA. Email: tecuci@cs.utexas.edu. 3 School of Computer Science, Univ. of Hertfordshire, AL10 9AB, UK. Email: w.c.ho@herts.ac.uk. Our results also show that agent performance does not decline linearly as the size of the episodic store decreases. Instead, there is a sudden decrease in performance when the size drops below a particular threshold. Because we have performed our experiment on three different systems, we expect the results of the research to be indicative of the overall effectiveness of the forgetting algorithms. Furthermore, the test bed we have designed is well suited for future research into forgetting for episodic memory. 2 EPISO DI C M E M O R Y Our goal in building artificial episodic memory modules is not to mimic the human episodic memory, but rather to endow artificial agents with such episodic storage and recall strategies and to explore the additional capabilities they allow. Research in human episodic memory plays a guiding role in our endeavors. An artificial episodic memory functions by storing events (or episodes) from the agent’s past. Retrieval is performed based on a cue; the system presents the agent with episodes that are most similar to a given cue. These recalled episodes can help the agent make better decisions faster. For example, an agent might be able solve problems faster by adapting previous solutions. Additional tasks, such as avoiding unwanted behavior by detecting potential problems, monitoring long-term goals by remembering what subgoals have been achieved, and reflection on past actions, become feasible. The size of the episodic store grows over time as the agent acquires more and more experiences. A large episodic memory raises a number of problems: it increases the search space of the retrieval algorithm thus negatively impacting retrieval time. An agent that indiscriminately stores all its past experiences might also run into external constraints related to the physical memory available to it. While opportunities for using data compression to reduce the size of the episodic store exist, ultimately it will continue to grow. The only technique for keeping episodic store below a fixed size is for the agent to regularly forget portions of this knowledge. As a result, we anticipate that forgetting must be an essential part of maintaining a long-term episodic store for an intelligent agent. A successful forgetting algorithm must fulfil two goals. First, the algorithm must remove knowledge that is least likely to be useful in the future. Logically, an agent that always forgets knowledge that will never be needed again performs equally well to an agent that does not forget anything. Second, the algorithm must be efficient. An algorithm that takes too long to select content to remove will affect the performance of the entire agent. Proceedings of the Remembering Who We Are – Human Memory for Artificial Agents Symposium, Mei Yii Lim and Wan Ching Ho (Eds.), at the AISB 2010 convention, 29 March – 1 April 2010, De Montfort University, Leicester, UK