A Framework for Hybrid Multimodal Performances

The boundaries between physical and virtual spaces are becoming more blurred in our everyday lives with the advent of wearable action cameras, virtual reality technologies, and algorithmicallyled audience interactions. Artists and creatives have been at the forefront of exploring these new technologies; however, little literature exists that reflects on best practices for navigating this new complex space. We reflect on our experiences in creating and participating in what we refer to as ‘hybrid multimodal performances’ or performances that blend physical and virtual spaces. We propose a framework of aesthetic choices we have implemented to seamlessly blend physical and virtual entities. We consider aspects such as the interaction between the camera and performer, the integration of multiple conceptual spaces, the changing relation between the artist and the work, and the multiple transformations of shape that occur when transitioning between physical and virtual spaces.

Content

Author and article information

Contributors

Shannon Cuykendall

Prophecy Sun

Reese Muntean

Thecla Schiphorst

Steve DiPaola

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 159-166

Affiliations

[0001]School of Interactive Arts + Technology, Simon Fraser University

250-13450 102 Avenue Surrey, British Columbia V3T 0A3, Canada

Article

DOI: 10.14236/ewic/EVA2017.37

SO-VID: 1b80f6d4-54ae-4d13-a936-f03e968efbf6

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts (EVA 2017)

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 11 – 13 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2017.37

Self URI (journal page): https://ewic.bcs.org/

7. REFERENCES

P. Ausländer 2007 Motional Abstraction: Alwin Nikolais’ Formalism C. GitelmanR. Martin The Returns of Alwin Nikolais: Bodies, boundaries and the dance canon. Wesleyan University Press Middletown, Connecticut
W. Benjamin 2008 The Work of Art in the Age of Mechanical Reproduction. Penguin UK
E. Brannigan 2011 Dance Film: Choreography and the Moving Image. Oxford University Press New York
J. P. BowenG. DiproseN. Lambert 2016 EVA London 2016: Electronic Visualisation and the Arts. Electronic Workshops in Computing, BCS. http://www.bcs.org/ewic/eva2016 3 May 2017
K. CarlsonS. CuykendallP. Sun 2016 Presence in a Box: Crossing Liminal Spaces. Scores + Traces: Exposing the Body through Computation. Exhibition, 10-12 March
S. CuykendallS. DiPaolaR. MunteanP. Sun 2017 Fractured Perspectives: Movement and Light. J. P. BowenG. DiproseN. Lambert EVA London 2017: Electronic Visualisation and the Arts. Electronic Workshops in Computing, BCS. (This volume.)
J. ChoT. H. LeeJ. OgdenA. StewartT. Y. TsaiJ. ChenR. Vituccio 2016 Imago: presence and emotion in virtual reality ACM SiGgRAPH 2016 VR Village Anaheim, California, USA 24-28 article 6 ACM
S. DiPaola 2014a Using a contextual focus model for an automatic creativity algorithm to generate art work Procedía Computer Science 41 212 219
S. DiPaola 2014b Computer modelling fine art painting using a cognitive correlative heuristics approach Proc: Biologically Inspired Cognitive Architectures. MIT, MA, USA
S. DiPaolaG. McCaig 2016 Using artificial intelligence techniques to emulate the creativity of a portrait painter Bowen et al 2016 158 165 DOI:10.14236/ewic/EVA2016.32
G. FauconnierM. Turner 1998 Conceptual integration networks Cognitive Science 22 2 133 187
A. GoldbergT. Schiphorst 2015 Glass Dance. Vancouver, BC, Canada http://www.peppersghost.org/?pageid=45 3 May 2017
A. GoldbergT. SchiphorstM. LantinT. Landry 2015 Worlds. Vancouver, BC, Canada. http://www.peppersghost.org/?pageid=43 3 May 2017
K. E. Jennings 2010 Developing creativity: Artificial barriers in artificial intelligence Minds and Machines 20 4 489 501
C. Lévi-Strauss 1966 The Savage Mind. Chicago, IL University of Chicago Press
A. NikolaisM. Louis 2005 The Nikolais/Louis Dance Technique: A Philosophy and Method of Modern Dance. Routledge New York
D. Reynolds 2013 Empathy, contagion, and affect: The role of kinesthesia in watching dance G. BrandstetterG. EgertS. Zubarik Touching and being touched: Kinesthesia and empathy in dance and movement. Walter de Gruyter Berlin/Boston
S. SalevatiS DiPaola 2015 A creative artificial intelligence system to investigate user experience, affect, emotion and creativity K. NgJ. P. BowenN. Lambert EVA London 2015: Electronic Visualisation and the Arts 140 152 Electronic Workshops in Computing, BCS DOI:10.14236/ewic/eva2015.13
S. SalevatiS. DiPaolaK. CarlsonT. Schiphorst 2016 Movement awareness through emotion based aesthetic visualisation Bowen et al 2016 124 132 DOI:10.14236/ewic/EVA2016.27
A. VallgårdaY. Fernaeus 2015 Interaction design as a bricolage practice Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction January 173 180 ACM
S. Zeki 2001 Artistic creativity and the brain Science 293 5527 51 52
R. Zimmer 2003 Abstraction in art with implications for perception Philosophical Transactions of the Royal Society of London B: Biological Sciences 358 1435 1285 1291

Comments

Comment on this article

[1] P. Ausländer 2007 Motional Abstraction: Alwin Nikolais’ Formalism C. GitelmanR. Martin The Returns of Alwin Nikolais: Bodies, boundaries and the dance canon. Wesleyan University Press Middletown, Connecticut

[2] W. Benjamin 2008 The Work of Art in the Age of Mechanical Reproduction. Penguin UK

[3] E. Brannigan 2011 Dance Film: Choreography and the Moving Image. Oxford University Press New York

[4] J. P. BowenG. DiproseN. Lambert 2016 EVA London 2016: Electronic Visualisation and the Arts. Electronic Workshops in Computing, BCS. http://www.bcs.org/ewic/eva2016 3 May 2017

[5] K. CarlsonS. CuykendallP. Sun 2016 Presence in a Box: Crossing Liminal Spaces. Scores + Traces: Exposing the Body through Computation. Exhibition, 10-12 March

[6] S. CuykendallS. DiPaolaR. MunteanP. Sun 2017 Fractured Perspectives: Movement and Light. J. P. BowenG. DiproseN. Lambert EVA London 2017: Electronic Visualisation and the Arts. Electronic Workshops in Computing, BCS. (This volume.)

[7] J. ChoT. H. LeeJ. OgdenA. StewartT. Y. TsaiJ. ChenR. Vituccio 2016 Imago: presence and emotion in virtual reality ACM SiGgRAPH 2016 VR Village Anaheim, California, USA 24-28 article 6 ACM

[8] S. DiPaola 2014a Using a contextual focus model for an automatic creativity algorithm to generate art work Procedía Computer Science 41 212 219

[9] S. DiPaola 2014b Computer modelling fine art painting using a cognitive correlative heuristics approach Proc: Biologically Inspired Cognitive Architectures. MIT, MA, USA

[10] S. DiPaolaG. McCaig 2016 Using artificial intelligence techniques to emulate the creativity of a portrait painter Bowen et al 2016 158 165 DOI:10.14236/ewic/EVA2016.32

[11] G. FauconnierM. Turner 1998 Conceptual integration networks Cognitive Science 22 2 133 187

[12] A. GoldbergT. Schiphorst 2015 Glass Dance. Vancouver, BC, Canada http://www.peppersghost.org/?pageid=45 3 May 2017

[13] A. GoldbergT. SchiphorstM. LantinT. Landry 2015 Worlds. Vancouver, BC, Canada. http://www.peppersghost.org/?pageid=43 3 May 2017

[14] K. E. Jennings 2010 Developing creativity: Artificial barriers in artificial intelligence Minds and Machines 20 4 489 501

[15] C. Lévi-Strauss 1966 The Savage Mind. Chicago, IL University of Chicago Press

[16] A. NikolaisM. Louis 2005 The Nikolais/Louis Dance Technique: A Philosophy and Method of Modern Dance. Routledge New York

[17] D. Reynolds 2013 Empathy, contagion, and affect: The role of kinesthesia in watching dance G. BrandstetterG. EgertS. Zubarik Touching and being touched: Kinesthesia and empathy in dance and movement. Walter de Gruyter Berlin/Boston

[18] S. SalevatiS DiPaola 2015 A creative artificial intelligence system to investigate user experience, affect, emotion and creativity K. NgJ. P. BowenN. Lambert EVA London 2015: Electronic Visualisation and the Arts 140 152 Electronic Workshops in Computing, BCS DOI:10.14236/ewic/eva2015.13

[19] S. SalevatiS. DiPaolaK. CarlsonT. Schiphorst 2016 Movement awareness through emotion based aesthetic visualisation Bowen et al 2016 124 132 DOI:10.14236/ewic/EVA2016.27

[20] A. VallgårdaY. Fernaeus 2015 Interaction design as a bricolage practice Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction January 173 180 ACM

[21] S. Zeki 2001 Artistic creativity and the brain Science 293 5527 51 52

[22] R. Zimmer 2003 Abstraction in art with implications for perception Philosophical Transactions of the Royal Society of London B: Biological Sciences 358 1435 1285 1291

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65