McGurk effect and audiovisual speech perception in students with Learning Disabilities exposed to online teaching during the COVID-19 pandemic

Abstract

Speech perception is primarily considered as an auditory process. In face-to-face interaction, though, speech perception involves the integration of auditory (sounds) and visual (lip and face movements) information at some stage in the process. This integration occurs before phonetic evaluation [8] or at a later stage suggesting that the acoustic and visual signals are evaluated separately and then integrated [2]. The integration of acoustic and visual signals could give rise to the McGurk effect [3]. In this effect, audiovisual speech information integrates to the extent visual information influences what the hearers report listening [5]. The McGurk effect has been investigated in children and adults with normal hearing [7] but less often among children and adults with learning disabilities (LD) [1]. The question that arises is whether young adults with LD attending higher education could integrate auditory and visual information for speech perception in an online classroom context. These students typically face difficulties in basic auditory perception, attention and memory that may affect more complex cognitive processing. In traditional face-to-face classrooms, students with LD encounter several speech perception difficulties due to noisy open-classrooms and distance from the lecturer. On the other hand, online teaching may provide a more suitable alternative since it incorporates the use of audio-visual information. Students could attend a lecture away from the noisy classroom background focusing on the lecturer’s face and voice. We hypothesise that via online teaching, lecturers’ hyper-articulated speech could lead to clearer speech perception marked with exaggerated articulatory movements that facilitate visual perception. With the use of computer-based material such as online sessions and video streaming, we assume that students’ engagement increases while they take control over the pace of learning and, as a result, learn more effectively since they could review sessions repeatedly. Online teaching may be more beneficial since even though numerous computer-aided language learning systems (CALL) are available, the feedback that these provide to students stem from a speech processing software and, as a result, are very different to a human speaker [6], [4]. Conflicts of Interest Statement We know of no conflicts of interest associated to this publication.