+1 Recommend
1 collections
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Approaches to Visualising the Spatial Position of ‘Sound-objects’

      1 , 1
      Electronic Visualisation and the Arts (EVA)
      Electronic Visualisation and the Arts
      12 - 14 July 2016
      Digital art, Mobile applications, Music, Performing arts, Technologies


            In this paper we present the rationale and design for two systems (developed by the Integra Lab research group at Birmingham Conservatoire) implementing a common approach to interactive visualisation of the spatial position of ‘sound-objects’. The first system forms part of the AHRC-funded project ‘Transforming Transformation: 3D Models for Interactive Sound Design’, which entails the development of a new interaction model for audio processing whereby sound can be manipulated through grasp as if it were an invisible 3D object. The second system concerns the spatial manipulation of ‘beatboxer’ vocal sound using handheld mobile devices through already-learned physical movement. In both cases a means to visualise the spatial position of multiple sound sources within a 3D ‘stereo image’ is central to the system design, so a common model for this task was therefore developed. This paper describes the ways in which sound and spatial information are implemented to meet the practical demands of these systems, whilst relating this to the wider context of extant, and potential future methods for spatial audio visualisation.


            Author and article information

            July 2016
            July 2016
            : 15-22
            [1 ] Birmingham Conservatoire

            Birmingham, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Electronic Visualisation and the Arts
            London, UK
            12 - 14 July 2016
            Electronic Workshops in Computing (eWiC)
            Electronic Visualisation and the Arts
            Product Information: 1477-9358BCS Learning & Development
            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2016.4
            Self URI (journal page): https://ewic.bcs.org/
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Digital art,Performing arts,Mobile applications,Technologies,Music


            1. 2015 Comparison of five time series EMG features extractions using Myo Armband 2015 International Electronics Symposium (IES), Surabaya, Indonesia 29-30 September 2015 11 14 IEEE

            2. 2000 Guidelines for using multiple views in information visualization Working Conference on Advanced Visual Interfaces, 110 119 ACM

            3. 1993 Charade: remote control of objects using free-hand gestures Communications of the ACM 36 7 28 35

            4. 2004 Visuo-vestibular interaction in the reconstruction of travelled trajectories Experimental Brain Research 154 1 11 21

            5. 2002 Visual cues and virtual touch: Role of visual stimuli and intersensory integration in cross-modal haptic illusions and the sense of presence Presence 2002 Porto, Portugal October 9-11 2002 410 428

            6. 2009 Visual trajectory perception in humans: Is it lateralized? Clues from online rTMS of the middle-temporal complex (MT/V5) Behavioral brain research 197 2 481 486

            7. 1994 Auditory Scene Analysis: The Perceptual Organization of Sound Bradford Books MIT Press

            8. 1893 A Statistical Study of Pseudo-Chromesthesia and of Mental-Forms The American Journal of Psychology 5 4 439 464

            9. 2015 Hearing in Color; Seeing in Sound: Chromesthesia and Its Influences on Audio-Visual Work

            10. 1992 Minimum audible movement angle in the horizontal plane as a function of stimulus frequency and bandwidth, source azimuth, and velocity The Journal of the Acoustical Society of America 91 3 1624 1636

            11. EA Sports ext-link-type="uri" xlink:href="http://nautil.us/issue/26/color/what-color-is-this-song">http://nautil.us/issue/26/color/what-color-is-this-song 15 March 2016

            12. 2004 Multisensory integration: how visual experience shapes spatial perception Current biology 14 3 R115 R117

            13. 2002 Enhancement of visual perception by crossmodal visuo-auditory interaction Experimental Brain Research 147 3 332 343

            14. 2000 Auditory-visual associations for music compositional processes: A survey International Computer Music Conference ICMC2000, Berlin, Germany

            15. 2009 Seeing Sounds and Hearing Colors: An Event-related Potential Study of Auditory-Visual Synesthesia Journal of Cognitive Neuroscience 21 10 1869 1881

            16. 2015 Myopoint: Pointing and clicking using forearm mounted electromyography and inertial motion sensors 33rd Annual aCm Conference on Human Factors in Computing Systems, 3653 3656 ACM

            17. 2013 A Survey of Interaction Techniques for Interactive 3D Environments Eurographics2013, Girona, Spain 65 93 Eurographics Association

            18. 1999 Internal models for motor control and trajectory planning Current opinion in neurobiology 9 6 718 727

            19. 2009 Simulating Haptic Feedback Using Vision: A Survey of Research and Applications of Pseudo-Haptic Feedback Presence 18 1 39 53

            20. 2010 Seeing Sound: Hans Jenny and the Cymatic Atlas, University of Pittsburgh Pennsylvania, USA

            21. 1975 On Colored-Hearing Synesthesia: Cross-Modal Translations of Sensory Dimensions Psychological Bulletin 82 3 303 31

            22. 2013 Principles and Techniques of Schlieren Imaging Systems Columbia University Computer Science Technical Reports.

            23. MyoMapper ext-link-type="uri" xlink:href="https://github.com/balandinodidonato/MyoMapper">https://github.com/balandinodidonato/MyoMapper 22 March 2016

            24. 2013 High-precision pointing on large wall displays using small handheld devices Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris 27 April 2 May 2013 831 840 ACM

            25. 2015 MuMYO-Evaluating and Exploring the MYO Armband for Musical Interaction Proceedings of the International Conference on New Interfaces for Musical Expression, Baton Rouge, LA, USA 31 May 31-3 June 2015 NIME’15

            26. 2000 A gesture-based interface for seamless communication between real and virtual worlds 6th ERCIM Workshop on User Interfaces for All.

            27. 2014 Color and emotion caused by auditory stimuli Ann Arbor, MI Michigan Publishing, University of Michigan Library

            28. 2012 When the eye listens: A meta-analysis of how audio-visual presentation enhances the appreciation of music performance Music Perception: An Interdisciplinary Journal 30 1 71 83

            29. 1987 Chromesthetic responses to music: Replication and extension Perceptual and Motor Skills 65 2 439 443

            30. Pure Data ext-link-type="uri" xlink:href="https://puredata.info/">https://puredata.info/ 22 March 2016

            31. Processing ext-link-type="uri" xlink:href="https://processing.org/">https://processing.org/ 22 March 2016

            32. 1990 Minimum audible movement angles as a function of sound source trajectory The Journal of the Acoustical Society of America 88 6 2639 2644

            33. 1966 Traité des objets musicaux Editions du Seuil, Paris

            34. Sega Amusements ext-link-type="uri" xlink:href="https://www.voutube.com/watch?v=WiORqfFk64s">https://www.voutube.com/watch?v=WiORqfFk64s 15 March 2016

            35. 2012 Schlieren and shadowgraph techniques: visualizing phenomena in transparent media. Springer Science & Business Media

            36. 1997 Direct manipulation for comprehensible, predictable and controllable user interfaces Proceedings of the 2nd international conference on Intelligent user interfaces Orlando, FL, USA 06-09 January 1997 33 39 ACM

            37. 1993 Auditory distance perception by translating observers Virtual Reality, IEEE Annual International Symposium, San Jose, CA, USA 16-23 March 1993 92 99 IEEE Computer Society

            38. 2000 Attracting attention to the illusory location of a sound: reflexive cross modal orienting and ventriloquism Neuroreport 11 9 2057 2061

            39. 2015 An Exploratory Evaluation of User Interfaces for 3D Audio Mixing Audio Engineering Society Convention 138.

            40. 1991 Studie II Multimediale 2

            41. 1999 Cognitive design elements to support the construction of a mental model during software exploration Journal of Systems and Software 44 3 171 185

            42. 1998 Auditory-Visual Cross-Modal Perception Phenomena Naval Postgraduate School Monterey CA, USA

            43. 1933 Improvements in the schlieren method Journal of Scientific Instruments 10 12 378

            44. Thalmic ext-link-type="uri" xlink:href="https://www.myo.com/">https://www.myo.com/ 22 March 2016

            45. 2012 Architecture and Design Procedure of a Generic SWB Antenna with Superb Performances for Tactical Commands and Ubiquitous Communications Ultra Wideband - Current Status and Future Trends.

            46. 2007 Auralization: fundamentals of acoustics, modelling, simulation, algorithms and acoustic virtual reality. Springer Science & Business Media

            47. WB Kids ext-link-type="uri" xlink:href="https://youtu.be/XqJqZKL8uEU">https://youtu.be/XqJqZKL8uEU 15 March 2016

            48. 3Dception ext-link-type="uri" xlink:href="http://www.twobigears.com/3dception.php">http://www.twobigears.com/3dception.php 12 February 2016


            Comment on this article