Framework for a Bio-Responsive VR for Interactive Real-time Environments and Interactives

In this paper, we propose a framework for a bio-responsive Virtual Reality (VR) system that evolves and reflects the immersant’s physiological data such as gestures and movements to biofeedback measures over time into a tightly coupled real-virtual immersive space. This paper will present current prototypes in the various application and experiential spaces we see our work being applied to from therapeutic applications to meditative spaces and artistic installations.

Content

Author and article information

Contributors

Meehae Song

Steve DiPaola

Conference

Publication date: July 2017

Publication date (Print): July 2017

Pages: 377-384

Affiliations

[0001]Simon Fraser University

250-13450 102 Avenue

Surrey, BC V3T 0A3, Canada

Article

DOI: 10.14236/ewic/EVA2017.75

SO-VID: 63c579f6-a460-4496-a794-04dbfc13d01c

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts (EVA 2017)

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 11 – 13 July 2017

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2017.75

Self URI (journal page): https://ewic.bcs.org/

7. REFERENCES

ADMTUK 2002 Arts Therapy: Benchmark statement Gloucester. Quality Assurance Agency for Higher Education. http://www.qaa.ac.uk/en/Publications/Documents/Subiect-benchmark-statement-Health-care-programmes---Arts-Therapy.pdf 22 March 2017
American Art Therapy Association 2017 https://www.arttherapy.org/ 20 March 2017
S. Buckley May 19 2015 This Is How Valve’s Amazing Lighthouse Tracking Technology Works. http://gizmodo.com/this-is-how-valve-s-amazing-lighthouse-tracking-technol-1705356768 22 March 2017
R. Darnley-SmithH. M. Patey 2003 Creative Therapies in Practice series: Music Therapy (1). SAGE Publications Ltd. London
M. S. El-NasrJ. F. MorieA. Drachen 2011 A Scientific Look at the Design of Aesthetically and Emotionally Engaging Interactive Entertainment Experiences D. GôkçayG. Yildirim Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives 281 307
A. Ebrahimi 15 December 2016 EditorVR Experimental Build Available Today. https://blogs.unity3d.com/2016/12/15/editorvrexperimental-build-available-today/ 25 March 2017
R. Ellis 2001 Movement metaphor as mediator: A model for the dance/movement therapy process The Arts in Psychotherapy 28 3 181 190
P. Ford-Martin 2001 Art therapy. Gale Encyclopaedia of Psychology 2nd Ed. 48 49 Gale Group New York
D. Freeman 2008 Studying and treating schizophrenia using virtual reality: A new paradigm Schizophrenia bulletin 34 4 605 610
D. Halprin 2003 Expressive Body in Life, Art and Therapy: Working with Movement, Metaphor and Meaning. Jessica Kingsley Publishers London
H. G. HoffmanA. Garcia-PalaciosA. CarlinT. A. Furness IIIC. Botella-Arbona 2003 Interfaces that heal: Coupling real and virtual objects to cure spider phobia International Journal of Human-Computer Interaction 16 2 283 300
G. J. Kim 2005 A SWOT analysis of the field of virtual reality rehabilitation and therapy Presence: Teleoperators and Virtual Environments 14 2 119 146
D. Kret 11 April 2014 What is Rhythmic Entrainment? and Why is it Important in Music Therapy? http://mindfullmusic.com/what-isrhythmic-entrainment-and-why-is-it-important-in-music-therapy-part-1-of-4/ 18 March 2017
C.A. Malchiodi 2003 Handbook of Art Therapy. Guilford Press New York
Mayo Clinic 14 January 2016 Biofeedback http://www.mayoclinic.org/testsprocedures/biofeedback/home/ovc-20169724 20 March 2017
R. McCraty 2015 Heart-brain neurodynamics: The making of emotions M. DahlitzG. Hall Heart: The Neuropsychotherapist Special Issue 76 110 Dahlitz Media Brisbane
S. McNiff 1981 The Arts and Psychotherapy. Charles C. Thomas Springfield
B. Meekums 2002 Dance Movement Therapy: A Creative Psychotherapeutic Approach. SAGE Publications London
S. MitraT. Acharya 2007 Gesture recognition: A survey IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37 3 311 324
G. OptaleS. CapodieciP. PinelliD. ZaraL. GamberiniG. Riva 2001 Music-enhanced immersive virtual reality in the rehabilitation of memory related cognitive processes and functional abilities: A case report Presence: Teleoperators and Virtual Environments 10 4 450 462
G. PapagiannakisE. ArgentoE. BakaM. ManiadakisP. Trahanias 2015 A virtual reality brainwave entrainment method for human augmentation applications Technical Report FORTH-ICS/TR-458
J. H. ParkJ. H. Park 2016 The effects of game-based virtual reality movement therapy plus mental practice on upper extremity function in chronic stroke patients with hemiparesis: A randomized controlled trial Journal of Physical Therapy Science 28 3 811 815
V. I. PavlovicR. SharmaT. S. Huang 1997 Visual interpretation of hand gestures for human computer interaction IEEE Trans. Pattern Anal. Mach. Intell. 19 7 677 695
J. Raab (Producer) 2016 Virtual Reality is for Artists. http://time.com/vr-is-for-artists/ 17 March 2017
L. Rosenbaum 1989 Biofeedback Frontiers: Selfregulation of Stress Reactivity. AMS Press New York
S. SalevatiS. DiPaola 2015 A creative artificial intelligence system to investigate user experience, affect, emotion and creativity Electronic Visualisation and the Arts (EVA2015) 140 147
M. SongS. DiPaola 2015 Exploring different ways of navigating emotionally-responsive artwork in immersive virtual environments Electronic Visualisation and the Arts (EVA2015) 232 239
R. M. SternW. J. RayK. S. Quigley 2001 Psychophysiological Recording. Oxford University Press New York
M. H. ThautG. P. KenyonM. L. SchauerG. C. McIntosh 1999 The connection between rhythmicity and brain function IEEE Eng. Med. Biol. Mag. 18 2 101 108 doi:10.1109/51.752991
M. H. ThautG. C. McIntoshV. Hoemberg 2014 Neurobiological foundations of neurologic music therapy: Rhythmic entrainment and the motor system Frontiers in Psychology 5 1185
URDesign 4 May 2016 http://www.urdesignmag.com/art/2016/05/04/miguel-chevaliers-vr-installation-for-art-brussels-2016/ 30 March 2017
Vive Team 27 March 2017 Get Creative with MakeVR. https://blog.vive.com/us/2017/03/27/get-creative-with-makevr/ 27 March 2017
J. Wang 2012 A whole body, kinaesthetic digital drawing tool for art therapy. Master’s thesis, Simon Fraser University Burnaby, Canada http://summit.sfu.ca/item/12214 23 May 2017
B. Warren 2003 Using the Creative Arts in Therapy: A Practical Introduction 2nd Edition Taylor and Francis London
T. WigramI.N. PedersenL. O. Bonde 2002 A comprehensive guide to music therapy: Theory, clinical practice, research and training. Jessica Kingsley Publishers Philadelphia
M. Zamagni 2017 Nature Abstraction. Times Square Arts 1-31 March http://www.timessquarenyc.org/times-square-arts/projects/midnight-moment/nature-abstraction/index.aspx 23 May 2017

Comments

Comment on this article

[1] ADMTUK 2002 Arts Therapy: Benchmark statement Gloucester. Quality Assurance Agency for Higher Education. http://www.qaa.ac.uk/en/Publications/Documents/Subiect-benchmark-statement-Health-care-programmes---Arts-Therapy.pdf 22 March 2017

[2] American Art Therapy Association 2017 https://www.arttherapy.org/ 20 March 2017

[3] S. Buckley May 19 2015 This Is How Valve’s Amazing Lighthouse Tracking Technology Works. http://gizmodo.com/this-is-how-valve-s-amazing-lighthouse-tracking-technol-1705356768 22 March 2017

[4] R. Darnley-SmithH. M. Patey 2003 Creative Therapies in Practice series: Music Therapy (1). SAGE Publications Ltd. London

[5] M. S. El-NasrJ. F. MorieA. Drachen 2011 A Scientific Look at the Design of Aesthetically and Emotionally Engaging Interactive Entertainment Experiences D. GôkçayG. Yildirim Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives 281 307

[6] A. Ebrahimi 15 December 2016 EditorVR Experimental Build Available Today. https://blogs.unity3d.com/2016/12/15/editorvrexperimental-build-available-today/ 25 March 2017

[7] R. Ellis 2001 Movement metaphor as mediator: A model for the dance/movement therapy process The Arts in Psychotherapy 28 3 181 190

[8] P. Ford-Martin 2001 Art therapy. Gale Encyclopaedia of Psychology 2nd Ed. 48 49 Gale Group New York

[9] D. Freeman 2008 Studying and treating schizophrenia using virtual reality: A new paradigm Schizophrenia bulletin 34 4 605 610

[10] D. Halprin 2003 Expressive Body in Life, Art and Therapy: Working with Movement, Metaphor and Meaning. Jessica Kingsley Publishers London

[11] H. G. HoffmanA. Garcia-PalaciosA. CarlinT. A. Furness IIIC. Botella-Arbona 2003 Interfaces that heal: Coupling real and virtual objects to cure spider phobia International Journal of Human-Computer Interaction 16 2 283 300

[12] G. J. Kim 2005 A SWOT analysis of the field of virtual reality rehabilitation and therapy Presence: Teleoperators and Virtual Environments 14 2 119 146

[13] D. Kret 11 April 2014 What is Rhythmic Entrainment? and Why is it Important in Music Therapy? http://mindfullmusic.com/what-isrhythmic-entrainment-and-why-is-it-important-in-music-therapy-part-1-of-4/ 18 March 2017

[14] C.A. Malchiodi 2003 Handbook of Art Therapy. Guilford Press New York

[15] Mayo Clinic 14 January 2016 Biofeedback http://www.mayoclinic.org/testsprocedures/biofeedback/home/ovc-20169724 20 March 2017

[16] R. McCraty 2015 Heart-brain neurodynamics: The making of emotions M. DahlitzG. Hall Heart: The Neuropsychotherapist Special Issue 76 110 Dahlitz Media Brisbane

[17] S. McNiff 1981 The Arts and Psychotherapy. Charles C. Thomas Springfield

[18] B. Meekums 2002 Dance Movement Therapy: A Creative Psychotherapeutic Approach. SAGE Publications London

[19] S. MitraT. Acharya 2007 Gesture recognition: A survey IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 37 3 311 324

[20] G. OptaleS. CapodieciP. PinelliD. ZaraL. GamberiniG. Riva 2001 Music-enhanced immersive virtual reality in the rehabilitation of memory related cognitive processes and functional abilities: A case report Presence: Teleoperators and Virtual Environments 10 4 450 462

[21] G. PapagiannakisE. ArgentoE. BakaM. ManiadakisP. Trahanias 2015 A virtual reality brainwave entrainment method for human augmentation applications Technical Report FORTH-ICS/TR-458

[22] J. H. ParkJ. H. Park 2016 The effects of game-based virtual reality movement therapy plus mental practice on upper extremity function in chronic stroke patients with hemiparesis: A randomized controlled trial Journal of Physical Therapy Science 28 3 811 815

[23] V. I. PavlovicR. SharmaT. S. Huang 1997 Visual interpretation of hand gestures for human computer interaction IEEE Trans. Pattern Anal. Mach. Intell. 19 7 677 695

[24] J. Raab (Producer) 2016 Virtual Reality is for Artists. http://time.com/vr-is-for-artists/ 17 March 2017

[25] L. Rosenbaum 1989 Biofeedback Frontiers: Selfregulation of Stress Reactivity. AMS Press New York

[26] S. SalevatiS. DiPaola 2015 A creative artificial intelligence system to investigate user experience, affect, emotion and creativity Electronic Visualisation and the Arts (EVA2015) 140 147

[27] M. SongS. DiPaola 2015 Exploring different ways of navigating emotionally-responsive artwork in immersive virtual environments Electronic Visualisation and the Arts (EVA2015) 232 239

[28] R. M. SternW. J. RayK. S. Quigley 2001 Psychophysiological Recording. Oxford University Press New York

[29] M. H. ThautG. P. KenyonM. L. SchauerG. C. McIntosh 1999 The connection between rhythmicity and brain function IEEE Eng. Med. Biol. Mag. 18 2 101 108 doi:10.1109/51.752991

[30] M. H. ThautG. C. McIntoshV. Hoemberg 2014 Neurobiological foundations of neurologic music therapy: Rhythmic entrainment and the motor system Frontiers in Psychology 5 1185

[31] URDesign 4 May 2016 http://www.urdesignmag.com/art/2016/05/04/miguel-chevaliers-vr-installation-for-art-brussels-2016/ 30 March 2017

[32] Vive Team 27 March 2017 Get Creative with MakeVR. https://blog.vive.com/us/2017/03/27/get-creative-with-makevr/ 27 March 2017

[33] J. Wang 2012 A whole body, kinaesthetic digital drawing tool for art therapy. Master’s thesis, Simon Fraser University Burnaby, Canada http://summit.sfu.ca/item/12214 23 May 2017

[34] B. Warren 2003 Using the Creative Arts in Therapy: A Practical Introduction 2nd Edition Taylor and Francis London

[35] T. WigramI.N. PedersenL. O. Bonde 2002 A comprehensive guide to music therapy: Theory, clinical practice, research and training. Jessica Kingsley Publishers Philadelphia

[36] M. Zamagni 2017 Nature Abstraction. Times Square Arts 1-31 March http://www.timessquarenyc.org/times-square-arts/projects/midnight-moment/nature-abstraction/index.aspx 23 May 2017

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65