Dynamic Embedding-based Retrieval for Personalized Item Recommendations at Instacart

Personalization is essential in e-commerce, with item recommendation as a critical task. In this paper, we describe a hybrid embedding-based retrieval system for real-time personalized item recommendations on Instacart. Our system addresses unique challenges in the multi-source retrieval system, and includes several key components to make it highly personalized and dynamic. Specifically, our system features a hybrid embedding model that includes a long-term user interests embedding model and a real-time session-based model, which are combined to capture users' immediate intents and historical interactions. Additionally, we have developed a contextual bandit solution to dynamically adjust the number of candidates from each source and optimally allocate retrieval slots given a limited computational budget. Our modeling and system optimization efforts have enabled us to provide highly personalized item recommendations in real-time at scale to all our customers, including new and long-standing users.