Marketing Through the Machine's Eyes: Image Analytics and Interpretability

The growth of social media and the sharing economy is generating abundant unstructured image and video data. Computer vision techniques can derive rich insights from unstructured data and can inform recommendations for increasing profits and consumer utility – if only the model outputs are interpretable enough to earn the trust of consumers and buy-in from companies. To build a foundation for understanding the importance of model interpretation in image analytics, the first section of this article reviews the existing work along three dimensions: the data type (image data vs. video data), model structure (feature-level vs. pixel-level), and primary application (to increase company profits vs. to maximize consumer utility). The second section discusses how the “black box” of pixel-level models leads to legal and ethical problems, but interpretability can be improved with eXplainable Artificial Intelligence (XAI) methods. We classify and review XAI methods based on transparency, the scope of interpretability (global vs. local), and model specificity (model-specific vs. model-agnostic); in marketing research, transparent, local, and model-agnostic methods are most common. The third section proposes three promising future research directions related to model interpretability: the economic value of augmented reality in 3D product tracking and visualization, field experiments to compare human judgments with the outputs of machine vision systems, and XAI methods to test strategies for mitigating algorithmic bias.