Interpreting Voice Assistant Interaction Quality From Unprompted User Feedback

Assessing the quality of a task performed by an Intelligent Voice Assistant (IVA) system such as Alexa, Siri, etc. is vital for maintaining a high bar for Customer Experience (CX) with the system. In this paper, we propose an approach to determine the quality of an IVA utterance using a ‘feedback’ utterance that is interpretable and scalable. Basing the IVA quality assessments on user feedback in a scalable manner helps the AI systems address problems important to users, bridges the gap between qualitative and quantitative measurements. We propose inexpensive techniques to make quality assessments available to IVA components with low latency, such that the downstream interactions can use the context to avoid negative CX.