Identifying Task Boundaries in Digital Assistants.

Digital assistants are emerging to become more prevalent in our daily lives. In interacting with these assistants, users may engage in multiple tasks within a short period of time. Identifying task boundaries and isolating them within a session is critical for measuring the performance of the system on each individual task. In this paper we aim to automatically identify sequences of interactions that together form a task. To this end, we sample interactions from a real world digital assistant and use crowd judges to segment a session into multiple tasks. After that, we use a machine learned model to identify task boundaries. Our learned model with its features significantly outperform the baselines. To the best of our knowledge, this is the first work that aims to identify tasks within digital assistant sessions.