Exploring perceived face similarity and its relation to image-based spaces: an effect of familiarity

One challenge in exploring the internal representation of faces is the lack of controlled stimuli changes. Researchers are often limited to verbalizable changes in the creation of a dataset. An alternative approach to verbalization for interpretability is finding image-based measures that allow us to quantify image manipulation. In this study, we explore whether PCA could be used to create controlled changes to a face by testing the effect of these changes on human perceived similarity and on computational differences in Gabor, Pixel and DNN spaces. In Experiment 1, the effect of single dimensional (PCA) colour or shape changes in unfamiliar faces was explored. We found that perceived similarity and the three image-based spaces are linearly related, almost perfectly in the case of the DNN, with a correlation of 0.94. This provides a controlled way to alter the appearance of a face. In experiment 2, the effect of familiarity on the perception of multidimensional changes was explored. Our findings show that there is a positive relationship between the number of components changed and both the perceived similarity and the same three image-based spaces used in experiment 1. We found that familiar faces are rated more similar overall than unfamiliar faces. That is, a change to a familiar face is perceived as making less difference than the exact same change to an unfamiliar face. The ability to quantify, and thus control, these changes is a powerful tool in exploring the factors that mediate a change in perceived identity.