EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort

Abstract

Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. A least‐squares, electroencephalography (EEG)‐based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5‐s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5‐s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ $p=0.034$] and minimum parietal alpha power measure [ $p=0.016$]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( $p=0.0428$). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech‐based features.

Abstract

We implemented two variants of attention switching during a “cocktail party” listening task that consisted of two simultaneously played audiobooks. Attention switching was studied under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial‐audio cues. Pupil and alpha measures of effort indicate that listeners expend more effort in the at‐will switch condition than in a sustained‐attention control condition.