Linking audiovisual integration to audiovisual speech recognition in noise

Especially in challenging listening conditions, listeners can benefit from the audiovisual nature of speech by using visual information. Yet there exists great inter-individual variability, not only in understanding speech in noise, but also in the benefit obtained from additional visual cues. First empirical evidence suggests that the ability to integrate auditory and visual input, i.e. audiovisual integration, is altered in hearing impairment and is, at the same time, relevant for audiovisual speech intelligibility. The distinct role of mild hearing loss on audiovisual integration and the significance of these changes for speech intelligibility, however, need further scrutiny. Thus, here we investigated differences in audiovisual integration capacities between elderly, normal-hearing and hearing-impaired individuals using two tests of audiovisual integration (sound-induced flash illusion, McGurk task). To explore whether potential differences in audiovisual integration are meaningful for natural speech intelligibility, we then linked audiovisual integration capacities to speech-in-noise recognition using an audiovisual speech-reception threshold test, expecting this to reflect a more realistic listening scenario. Our results indicate that audiovisual integration abilities are already altered in mild hearing impairment, while the magnitude and direction of the effect depend on the specific test used. At the same time, audiovisual integration capacities seem relevant for predicting audiovisual speech intelligibility in noise, especially in those individuals with a hearing loss. We conclude that audiovisual integration abilities should therefore be considered for future predictions of speech recognition outcomes, which – in turn – should be assessed audiovisually, to account for the multisensory nature of speech and communication.