An Intelligent Man-Machine Interface—Multi-Robot Control Adapted for Task Engagement Based on Single-Trial Detectability of P300

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Advanced man-machine interfaces (MMIs) are being developed for teleoperating robots at remote and hardly accessible places. Such MMIs make use of a virtual environment and can therefore make the operator immerse him-/herself into the environment of the robot. In this paper, we present our developed MMI for multi-robot control. Our MMI can adapt to changes in task load and task engagement online. Applying our approach of embedded Brain Reading we improve user support and efficiency of interaction. The level of task engagement was inferred from the single-trial detectability of P300-related brain activity that was naturally evoked during interaction. With our approach no secondary task is needed to measure task load. It is based on research results on the single-stimulus paradigm, distribution of brain resources and its effect on the P300 event-related component. It further considers effects of the modulation caused by a delayed reaction time on the P300 component evoked by complex responses to task-relevant messages. We prove our concept using single-trial based machine learning analysis, analysis of averaged event-related potentials and behavioral analysis. As main results we show (1) a significant improvement of runtime needed to perform the interaction tasks compared to a setting in which all subjects could easily perform the tasks. We show that (2) the single-trial detectability of the event-related potential P300 can be used to measure the changes in task load and task engagement during complex interaction while also being sensitive to the level of experience of the operator and (3) can be used to adapt the MMI individually to the different needs of users without increasing total workload. Our online adaptation of the proposed MMI is based on a continuous supervision of the operator's cognitive resources by means of embedded Brain Reading. Operators with different qualifications or capabilities receive only as many tasks as they can perform to avoid mental overload as well as mental underload.

Related collections

Most cited references 29

Record: found
Abstract: found
Article: not found

On the utility of P3 amplitude as a measure of processing capacity.

ALBERT KOK (2001)

The present review focuses on the utility of the amplitude of P3 of as a measure of processing capacity and mental workload. The paper starts with a brief outline of the conceptual framework underlying the relationship between P3 amplitude and task demands, and the cognitive task manipulations that determine demands on capacity. P3 amplitude results are then discussed on the basis of an extensive review of the relevant literature. It is concluded that although it has often been assumed that P3 amplitude depends on the capacity for processing task relevant stimuli, the utility of P3 amplitude as a sensitive and diagnostic measure of processing capacity remains limited. The major factor that prompts this conclusion is that the two principal task variables that have been used to manipulate capacity allocation, namely task difficulty and task emphasis, have opposite effects on the amplitude of P3. I suggest that this is because, in many tasks, an increase in difficulty transforms the structure or actual content of the flow of information in the processing systems, thereby interfering with the very processes that underlie P3 generation. Finally, in an attempt to theoretically integrate the results of the reviewed studies, it is proposed that P3 amplitude reflects activation of elements in a event-categorization network that is controlled by the joint operation of attention and working memory.

0 comments Cited 405 times – based on 0 reviews      Review now

Record: found
Abstract: not found
Article: not found

On quantifying surprise: the variation of event-related potentials with subjective probability.

Connie Duncan-Johnson, Emanuel Donchin (1977)

0 comments Cited 189 times – based on 0 reviews      Review now

Record: found
Abstract: found
Article: not found

Multiple resources and mental workload.

Christopher D Wickens (2008)

The objective is to lay out the rationale for multiple resource theory and the particular 4-D multiple resource model, as well as to show how the model is useful both as a design tool and as a means of predicting multitask workload overload. I describe the discoveries and developments regarding multiple resource theory that have emerged over the past 50 years that contribute to performance and workload prediction. The article presents a history of the multiple resource concept, a computational version of the multiple resource model applied to multitask driving simulation data, and the relation of multiple resources to workload. Research revealed the importance of the four dimensions in accounting for task interference and the association of resources with brain structure. Multiple resource models yielded high correlations between model predictions and data. Lower correlations also identified the existence of additional resources. The model was shown to be partially relevant to the concept of mental workload, with greatest relevance to performance breakdowns related to dual-task overload. Future challenges are identified. The most important application of the multiple resource model is to recommend design changes when conditions of multitask resource overload exist.

0 comments Cited 128 times – based on 0 reviews      Review now

All references

Author and article information

Contributors

Elsa A. Kirchner: URI : http://loop.frontiersin.org/people/128000/overview

Su K. Kim: URI : http://loop.frontiersin.org/people/314883/overview

Marc Tabie: URI : http://loop.frontiersin.org/people/314119/overview

Hendrik Wöhrle: URI : http://loop.frontiersin.org/people/127778/overview

Frank Kirchner: URI : http://loop.frontiersin.org/people/102557/overview

Journal

Journal ID (nlm-ta): Front Hum Neurosci

Journal ID (iso-abbrev): Front Hum Neurosci

Journal ID (publisher-id): Front. Hum. Neurosci.

Title: Frontiers in Human Neuroscience

Publisher: Frontiers Media S.A.

ISSN (Electronic): 1662-5161

Publication date (Electronic): 21 June 2016

Publication date Collection: 2016

Volume: 10

Electronic Location Identifier: 291

Affiliations

[1] ¹Research Group Robotics, Mathematic and Computer Science, University of Bremen Bremen, Germany

[2] ²Robotics Innovation Center (RIC), German Research Center for Artificial Intelligence (DFKI GmbH) Bremen, Germany

Author notes

Edited by: Klaus Gramann, Berlin Institute of Technology, Germany

Reviewed by: Tamer Demiralp, Istanbul University, Turkey; Pieter-Jan Kindermans, TU-Berlin, Germany

*Correspondence: Elsa A. Kirchner ekir@ 123456informatik.uni-bremen.de

Article

DOI: 10.3389/fnhum.2016.00291

PMC ID: 4914506

PubMed ID: 27445742

SO-VID: 4eecc751-0ef7-4fa8-a211-e081d0394f18

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

History

Date received : 05 December 2015

Date accepted : 31 May 2016

Page count

Figures: 10, Tables: 5, Equations: 0, References: 57, Pages: 21, Words: 15066

Comments

Comment on this article

Cited by 6

See all cited by

Most referenced authors 345

See all reference authors