Supervising and Controlling Unmanned Systems: A Multi-Phase Study with Subject Matter Experts

Everyday life requires frequent shifts between cognitive tasks. Research reviewed in this article probes the control processes that reconfigure mental resources for a change of task by requiring subjects to switch frequently among a small set of simple tasks. Subjects' responses are substantially slower and, usually, more error-prone immediately after a task switch. This 'switch cost' is reduced, but not eliminated, by an opportunity for preparation. It seems to result from both transient and long-term carry-over of 'task-set' activation and inhibition as well as time consumed by task-set reconfiguration processes. Neuroimaging studies of task switching have revealed extra activation in numerous brain regions when subjects prepare to change tasks and when they perform a changed task, but we cannot yet separate 'controlling' from 'controlled' regions.

We show that psychophysiologically driven real-time adaptive aiding significantly enhances performance in a complex aviation task. A further goal was to assess the importance of individual operator capabilities when providing adaptive aiding. Psychophysiological measures are useful for monitoring cognitive workload in laboratory and real-world settings. They can be recorded without intruding into task performance and can be analyzed in real time, making them candidates for providing operator functional state estimates. These estimates could be used to determine if and when system intervention should be provided to assist the operator to improve system performance. Adaptive automation was implemented while operators performed an uninhabited aerial vehicle task. Psychophysiological data were collected and an artificial neural network was used to detect periods of high and low mental workload in real time. The high-difficulty task levels used to initiate the adaptive automation were determined separately for each operator, and a group-derived mean difficulty level was also used. Psychophysiologically determined aiding significantly improved performance when compared with the no-aiding conditions. Improvement was greater when adaptive aiding was provided based on individualized criteria rather than on group-derived criteria. The improvements were significantly greater than when the aiding was randomly provided. These results show that psychophysiologically determined operator functional state assessment in real time led to performance improvement when included in closed loop adaptive automation with a complex task. Potential future applications of this research include enhanced workstations using adaptive aiding that would be driven by operator functional state.