The effect of induced emotional states on the magnitude of cross-modal correspondence effect

Cross-modal correspondence effect (i.e., facilitated processing of congruent stimuli from different modalities) occurs not only when simple multi-modal sensory stimuli are processed together, but also during their simultaneous processing with words with emotional and spatial connotations. We tested a hypothesis that the magnitude of cross-modal correspondence effect, arising from concurrent processing of basic sensory and verbal stimuli, is differentially modulated by individual's emotional state. Thirty-six volunteers (26 females, 18-34 years old) watched videos that evoked positive, negative, or neutral emotional states. This was followed by the main task in which they were presented with sounds of different pitch (low: 1000 Hz; high: 2000 Hz) simultaneously with words that differed in their emotional valence and were associated with different parts of space (low/high). The participant's task was to identify the pitch (low/high) of the non-verbal sound stimuli. Two-way mixed ANOVA and subsequent pairwise comparisons (Student's t-test for dependent samples) were used to compare both mean reaction times and estimated parameters of the ex-Gaussian distribution. The results showed that the audiovisual correspondence effect became manifested in faster responses to congruent stimulus combinations compared with non-congruent ones (t(35) = -3.20, p = 0.005, d(z) = -0.53, 95% CI [-0.89, -0.18]). However, we did not find a large size effect of the induced emotional state on the magnitude of this correspondence effect (F(4, 68) = 0.49, p = 0.744, h(G)(2) = 0.001). This result may be explained either by robustness of cross-modal correspondence effect and its resilience to emotional influence or by specific limitations of present study design.