Individuals reporting high levels of distractibility in everyday life show impaired performance in standard laboratory tasks measuring selective attention and inhibitory processes. Similarly, increasing cognitive load leads to more errors/distraction in a variety of cognitive tasks. How these two factors interact is currently unclear; highly distractible individuals may be affected more when their cognitive resources are taxed, or load may linearly affect performance for all individuals. We investigated the relationship between self-reported levels of cognitive failures (CF) in daily life and performance in the antisaccade task, a widely used tool examining attentional control. Levels of concurrent cognitive demand were manipulated using a secondary auditory discrimination task. We found that both levels of self-reported CF and task load increased antisaccade latencies while having no effect on prosaccade eye-movements. However individuals rating themselves as suffering few daily life distractions showed a comparable load cost to those who experience many. These findings suggest that the likelihood of distraction is governed by the addition of both internal susceptibility and the external current load placed on working memory.