+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Crossmodal Pattern Discrimination in Humans and Robots: A Visuo-Tactile Case Study

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          The quality of crossmodal perception hinges on two factors: The accuracy of the independent unimodal perception and the ability to integrate information from different sensory systems. In humans, the ability for cognitively demanding crossmodal perception diminishes from young to old age. Here, we propose a new approach to research to which degree the different factors contribute to crossmodal processing and the age-related decline by replicating a medical study on visuo-tactile crossmodal pattern discrimination utilizing state-of-the-art tactile sensing technology and artificial neural networks (ANN). We implemented two ANN models to specifically focus on the relevance of early integration of sensory information during the crossmodal processing stream as a mechanism proposed for efficient processing in the human brain. Applying an adaptive staircase procedure, we approached comparable unimodal classification performance for both modalities in the human participants as well as the ANN. This allowed us to compare crossmodal performance between and within the systems, independent of the underlying unimodal processes. Our data show that unimodal classification accuracies of the tactile sensing technology are comparable to humans. For crossmodal discrimination of the ANN the integration of high-level unimodal features on earlier stages of the crossmodal processing stream shows higher accuracies compared to the late integration of independent unimodal classifications. In comparison to humans, the ANN show higher accuracies than older participants in the unimodal as well as the crossmodal condition, but lower accuracies than younger participants in the crossmodal task. Taken together, we can show that state-of-the-art tactile sensing technology is able to perform a complex tactile recognition task at levels comparable to humans. For crossmodal processing, human inspired early sensory integration seems to improve the performance of artificial neural networks. Still, younger participants seem to employ more efficient crossmodal integration mechanisms than modeled in the proposed ANN. Our work demonstrates how collaborative research in neuroscience and embodied artificial neurocognitive models can help to derive models to inform the design of future neurocomputational architectures.

          Related collections

          Most cited references 49

          • Record: found
          • Abstract: found
          • Article: not found

          Is neocortex essentially multisensory?

          Although sensory perception and neurobiology are traditionally investigated one modality at a time, real world behaviour and perception are driven by the integration of information from multiple sensory sources. Mounting evidence suggests that the neural underpinnings of multisensory integration extend into early sensory processing. This article examines the notion that neocortical operations are essentially multisensory. We first review what is known about multisensory processing in higher-order association cortices and then discuss recent anatomical and physiological findings in presumptive unimodal sensory areas. The pervasiveness of multisensory influences on all levels of cortical processing compels us to reconsider thinking about neural processing in unisensory terms. Indeed, the multisensory nature of most, possibly all, of the neocortex forces us to abandon the notion that the senses ever operate independently during real-world cognition.
            • Record: found
            • Abstract: found
            • Article: not found

            Crossmodal processing in the human brain: insights from functional neuroimaging studies.

             Jay Calvert (2001)
            Modern brain imaging techniques have now made it possible to study the neural sites and mechanisms underlying crossmodal processing in the human brain. This paper reviews positron emission tomography, functional magnetic resonance imaging (fMRI), event-related potential and magnetoencephalographic studies of crossmodal matching, the crossmodal integration of content and spatial information, and crossmodal learning. These investigations are beginning to produce some consistent findings regarding the neuronal networks involved in these distinct crossmodal operations. Increasingly, specific roles are being defined for the superior temporal sulcus, the inferior parietal sulcus, regions of frontal cortex, the insula cortex and claustrum. The precise network of brain areas implicated in any one study, however, seems to be heavily dependent on the experimental paradigms used, the nature of the information being combined and the particular combination of modalities under investigation. The different analytic strategies adopted by different groups may also be a significant factor contributing to the variability in findings. In this paper, we demonstrate the impact of computing intersections, conjunctions and interaction effects on the identification of audiovisual integration sites using existing fMRI data from our own laboratory. This exercise highlights the potential value of using statistical interaction effects to model electrophysiological responses to crossmodal stimuli in order to identify possible sites of multisensory integration in the human brain.
              • Record: found
              • Abstract: found
              • Article: not found

              A Brain-Wide Study of Age-Related Changes in Functional Connectivity.

              Aging affects functional connectivity between brain areas, however, a complete picture of how aging affects integration of information within and between functional networks is missing. We used complex network measures, derived from a brain-wide graph, to provide a comprehensive overview of age-related changes in functional connectivity. Functional connectivity in young and older participants was assessed during resting-state fMRI. The results show that aging has a large impact, not only on connectivity within functional networks but also on connectivity between the different functional networks in the brain. Brain networks in the elderly showed decreased modularity (less distinct functional networks) and decreased local efficiency. Connectivity decreased with age within networks supporting higher level cognitive functions, that is, within the default mode, cingulo-opercular and fronto-parietal control networks. Conversely, no changes in connectivity within the somatomotor and visual networks, networks implicated in primary information processing, were observed. Connectivity between these networks even increased with age. A brain-wide analysis approach of functional connectivity in the aging brain thus seems fundamental in understanding how age affects integration of information. © The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.

                Author and article information

                Front Robot AI
                Front Robot AI
                Front. Robot. AI
                Frontiers in Robotics and AI
                Frontiers Media S.A.
                23 December 2020
                : 7
                1Department of Neurology, University Medical Center Hamburg-Eppendorf , Hamburg, Germany
                2Department of Informatics, Universität Hamburg , Hamburg, Germany
                Author notes

                Edited by: Igor Farkaš, Comenius University, Slovakia

                Reviewed by: Michal Vavrecka, Czech Technical University in Prague, Czechia; Pablo Lanillos, Radboud University Nijmegen, Netherlands

                *Correspondence: Focko L. Higgen f.higgen@ 123456uke.de

                This article was submitted to Sensor Fusion and Machine Perception, a section of the journal Frontiers in Robotics and AI

                †These authors have contributed equally to this work

                Copyright © 2020 Higgen, Ruppel, Görner, Kerzel, Hendrich, Feldheim, Wermter, Zhang and Gerloff.

                This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

                Page count
                Figures: 5, Tables: 2, Equations: 0, References: 52, Pages: 12, Words: 8413
                Funded by: Deutsche Forschungsgemeinschaft 10.13039/501100001659
                Funded by: National Natural Science Foundation of China 10.13039/501100001809
                Robotics and AI
                Original Research

                aging, braille, multisensory integration, neural networks, ann


                Comment on this article