Blog
About
Search
Advanced search

283
views
48
references
 Top references
0
cited by
 Most cited
    
0 reviews
 Review
0
comments
 Comment
0
recommends
+1 Recommend
1 collections
    9
    shares
      231
      similar
       All similar
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      The Parallels between the Study of Crossmodal Correspondence and the Design of Cross-Sensory Mappings

      Author(s):

      Publication date (Print):

      Conference name: Electronic Visualisation and the Arts (EVA 2017) (EVA)

      Conference theme: Electronic Visualisation and the Arts

      Conference date: 11 – 13 July 2017

      Keywords: Display and interaction design, Mappings, Perceptual learning, Crossmodal correspondence

      Read this article at

      ScienceOpen
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The aim of this paper is to examine how recent research findings and methodologies from the field of cognitive science could be utilised to inform the design and evaluation of cross-modal mappings for multimodal user interaction. In this paper we argue that by using empirical methods to enact embodied knowledge about cross-modal correspondences, we can form an adequate empirical framework for the design of multimodal mappings that successfully align with prior perceptual knowledge. This alignment can significantly improve the human computer dialogue and the analytical, creative and pedagogical value of user interfaces.

          Related collections

          Most cited references 48

          • Record: found
          • Abstract: found
          • Article: not found

          Benefits of multisensory learning.

           Ladan Shams,  Aaron R. Seitz (2008)
          Studies of learning, and in particular perceptual learning, have focused on learning of stimuli consisting of a single sensory modality. However, our experience in the world involves constant multisensory stimulation. For instance, visual and auditory information are integrated in performing many tasks that involve localizing and tracking moving objects. Therefore, it is likely that the human brain has evolved to develop, learn and operate optimally in multisensory environments. We suggest that training protocols that employ unisensory stimulus regimes do not engage multisensory learning mechanisms and, therefore, might not be optimal for learning. However, multisensory-training protocols can better approximate natural settings and are more effective for learning.
          Article 0 comments Cited 110 times – based on 0 reviews     Review now
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans.

             Vera U. Ludwig,  Ikuma Adachi,  Tetsuro Matsuzawa (2011)
            Humans share implicit preferences for certain cross-sensory combinations; for example, they consistently associate higher-pitched sounds with lighter colors, smaller size, and spikier shapes. In the condition of synesthesia, people may experience such cross-modal correspondences to a perceptual degree (e.g., literally seeing sounds). So far, no study has addressed the question whether nonhuman animals share cross-modal correspondences as well. To establish the evolutionary origins of cross-modal mappings, we tested whether chimpanzees (Pan troglodytes) also associate higher pitch with higher luminance. Thirty-three humans and six chimpanzees were required to classify black and white squares according to their color while hearing irrelevant background sounds that were either high-pitched or low-pitched. Both species performed better when the background sound was congruent (high-pitched for white, low-pitched for black) than when it was incongruent (low-pitched for white, high-pitched for black). An inherent tendency to pair high pitch with high luminance hence evolved before the human lineage split from that of chimpanzees. Rather than being a culturally learned or a linguistic phenomenon, this mapping constitutes a basic feature of the primate sensory system.
            Article 0 comments Cited 37 times – based on 0 reviews     Review now
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              The development of audiovisual multisensory integration across childhood and early adolescence: a high-density electrical mapping study.

               Amy Russo,  John J. Foxe,  Ted S Altschuler … (2011)
              The integration of multisensory information is essential to forming meaningful representations of the environment. Adults benefit from related multisensory stimuli but the extent to which the ability to optimally integrate multisensory inputs for functional purposes is present in children has not been extensively examined. Using a cross-sectional approach, high-density electrical mapping of event-related potentials (ERPs) was combined with behavioral measures to characterize neurodevelopmental changes in basic audiovisual (AV) integration from middle childhood through early adulthood. The data indicated a gradual fine-tuning of multisensory facilitation of performance on an AV simple reaction time task (as indexed by race model violation), which reaches mature levels by about 14 years of age. They also revealed a systematic relationship between age and the brain processes underlying multisensory integration (MSI) in the time frame of the auditory N1 ERP component (∼ 120 ms). A significant positive correlation between behavioral and neurophysiological measures of MSI suggested that the underlying brain processes contributed to the fine-tuning of multisensory facilitation of behavior that was observed over middle childhood. These findings are consistent with protracted plasticity in a dynamic system and provide a starting point from which future studies can begin to examine the developmental course of multisensory processing in clinical populations.
              Article 0 comments Cited 30 times – based on 0 reviews     Review now
                Bookmark
                All references

                Author and article information

                Contributors
                Augoustinos Tsiros
                Conference
                Publication date: July 2017
                Publication date (Print): July 2017
                Pages: 175-182
                Affiliations
                Centre for Interaction Design

                Edinburgh Napier University

                10 Colinton Road, EH10 5DT

                Scotland
                Article
                DOI: 10.14236/ewic/EVA2017.39
                Copyright statement: © Tsiros. Published by BCS Learning and Development Ltd. Proceedings of EVA London 2017, UK
                License:

                This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

                Conference name: Electronic Visualisation and the Arts (EVA 2017)
                Conference acronym: EVA
                Conference number:
                Conference location: London, UK
                Conference date: 11 – 13 July 2017
                Conference sponsor: Electronic Workshops in Computing (eWiC)
                Conference theme: Electronic Visualisation and the Arts
                Product
                Product Information: 1477-9358BCS Learning & Development
                Self URI (journal page): https://ewic.bcs.org/
                Categories
                Series title: Electronic Workshops in Computing

                Data availability:

                ScienceOpen disciplines: Applied computer science, Computer science, Security & Cryptology, Graphics & Multimedia design, General computer science, Human-computer-interaction

                Keywords: Crossmodal correspondence, Display and interaction design, Mappings, Perceptual learning

                Comments

                Comment on this article