Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion

Abstract

The ventriloquism effect is the tendency to underestimate the spatial separation between synchronous auditory and visual signals moderately separated in space. If, as it is thought, this effect is pre-attentive, it could modulate the mismatch negativity (MMN) that indexes the automatic, pre-attentive detection of deviant auditory stimuli rarely occurring in a sequence of standard stimuli. We assessed the existence of an MMN evoked by auditory and visual signals made up of standard sounds coming from the same location as the visual signal and deviant sounds coming from lateral deviations (20 or 60 degrees). As first observed in a behavioral study, a ventriloquism effect occurred for 20 degrees spatial separation but not for 60 degrees. Cortical evoked potentials were recorded using the oddball paradigm on 8 adults in auditory alone and audiovisual conditions. In the auditory alone condition, each spatial localization contrast elicited a significant MMN. In the audiovisual condition, a significant MMN was only evoked for the 60 degrees contrasts. The MMN evoked by spatial separation contrasts (20 and 60 degrees) in the auditory alone condition was suppressed by the corresponding audiovisual condition only when the latter yielded a ventriloquism effect, suggesting that this effect occurs at an early perceptual level.