1,702
views
0
recommends
+1 Recommend
1 collections
    4
    shares

      Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Beyond the Audience Seat: The recording and production of immersive opera and interactive concerto programmes for VR experiences

      proceedings-article
      ,
      Proceedings of EVA London 2019 (EVA 2019)
      Electronic Visualisation and the Arts
      8 - 11 July 2019
      Opera, Virtual reality, 6DoF, Volumetric, Ambisonics, Recording
      Bookmark

            Abstract

            New modes of expression have always defined a central attribute within the ambitions of classical music. The extended realities (XR) hold the potential to expand this expressive repertoire by transporting audiences into the middle of the action within an immersive music performance. We argue that implementing these innovative formats in contemporary opera performance, composition, production, and recording opens up new aesthetical paradigms that can reciprocally inform the artistic disciplines and reach a wider demographic of audiences. To our knowledge, this paper presents the first ever interactive virtual reality (VR) production of an entire opera, “She” (2017) and “Croak” (2018). The recordings devise 3-dimensional 360-degree high-resolution video and 3rd-order Ambisonics audio from multiple perspectives within a bespoke software interface. Following a discussion of both operas from the composer’s and engineer’s perspective, the paper will present an enhanced rendition of “CroakVR”, a 6-degrees-of-freedom (6DoF) volumetric simulation of its corresponding opera, that raises the bar of interactivity by enabling audiences to freely explore various mimetic and diegetic perspectives of its staging. Furthermore, we discuss on-going work on “Climb!VR”, a 6DoF volumetric VR production devising multiple Ambisonics recordings of an interactive electro-acoustic piano concerto “Climb!” based on ludonarrative structures. It is argued that VR recordings can amend the life span of musical compositions by offering listeners a concert experience that can extend beyond the walls of the physical concert hall.

            Content

            Author and article information

            Contributors
            Conference
            July 2019
            July 2019
            : 409-416
            Affiliations
            [0001]Aalborg University

            Dept. of Communication & Psychology

            Musikkens Plads 1

            9000 Aalborg, Denmark
            [0002]Hong Kong Baptist University

            Rm AST801, Sing Tao Building

            Hong Kong Baptist University

            Kowloon Tong, Kowloon, Hong Kong
            Article
            10.14236/ewic/EVA2019.75
            ef74ab17-b76c-4a61-9ea5-0b13fbeb6b5c
            © Gasselseder et al. Published by BCS Learning and Development Ltd. Proceedings of EVA London 2019, UK

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Proceedings of EVA London 2019
            EVA 2019
            London, UK
            8 - 11 July 2019
            Electronic Workshops in Computing (eWiC)
            Electronic Visualisation and the Arts
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2019.75
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Virtual reality,Opera,6DoF,Recording,Ambisonics,Volumetric

            REFERENCES

            1. Audionamix 2019 ADX Trax Pro 3 Audio Spectral Editing Tool https://audionamix.com/technology/adx-trax-pro/ retrieved 19 March 2019 )

            2. 2015 The Role of Agency in Ludoacoustic Immersion: Experiencing Recorded Sound and Music in Situational Context Audio Mostly Thessaloniki, Greece 7-9 October 2015 ACM New York

            3. 2018 Stereo2Depth http://pseudoscience.pictures/stereo2depth/pseudo_sciencestereo2depth1.8.5.zip retrieved 19 March 2019 )

            4. 2018 Use depth maps to create 6DoF in Unity Immersive Shooter https://www.immersiveshooter.com/2018/01/23/use-depth-maps-create-6dof-unity retrieved 19 March 2019 )

            5. Insta360 2017 Insta360 Pro 360° 3D Camera for VR https://www.insta360.com/product/insta360-pro retrieved 19 March 2019 )

            6. 2017 Composing and Realising a Game-like Performance for Disklavier and Electronics NIME Copenhagen, Denmark 15-18 May 2017

            7. 2018 Under Construction – Contemporary Opera in the Crossroads Between New Aesthetics, Techniques, and Technologies Audio Mostly Wrexham, UK 12-14 September 2018 ACM New York

            8. Kodak 2015 Kodak SP360-4K 360° Camera https://kodakpixpro.com/Europe/cameras/vrcamera/sp3604k retrieved 19 March 2019 )

            9. 2018 Linear (Live Generated Interface and Notation Environment in Augmented Reality) International Conference on Technologies for Music Notation and Representation Tenor 18 Montreal 24-26 May 2018 33 42

            10. 2018 New Realities in Audio Boca Raton, FL CRC Press

            11. 2016 Pianolens: An Augmented Reality Interface for Piano Instruction https://cs.stanford.edu/people/estrasni/otherprojects/pianolens.html retrieved 19 March 2019 )

            12. 2018 “Eight’ description https://www.vanderaa.net/eight retrieved 19 March 2019 )

            13. Zylia 2019 Zylia ZM-1 Ambisonics Microphone Array http://www.zylia.co/zylia-pro.html retrieved 19 March 2019 )

            Comments

            Comment on this article