Experimental Affect: Gesture controlled audio-visual performance in VR

This work is in alignment with notions presented by Erika Fischer-Lichte and Max Hermann, regarding theatre and performance as a spatial, embodied event; something which has energy and sensation (Carlson, p. 6). I draw on Mark Hansen’s ideas around bodily affectivity and the affective capacity of the human subject by way of digital content in “tingeing or flavouring the embodied perceptual present” (Hansen 2004b, p.605; Clough, p. 212). This work also proceeds with an awareness of Karan Barad’s Agential Realism and Eleanor Dare’s manifesto and ideas around co-emergent possibilities, accountability, and the role of subjectivity in VR, in relation to Bertold Brecht’s AEffect.


INTRODUCTION
This practice-based research examines audience experience in Virtual Reality (VR) and draws upon experimental audio-visuals generated live and performed to a group in VR. The work is experimental in terms of generating new forms of performance and includes an improvisational element.
This study investigates the ways in which this type of art performance in VR affects and impacts an audience, and to what degree the audience is aware of the live setting whilst immersed in the virtual space. The 'performance ' consists of a performer controlling audio-visuals live through gesture, which are networked to audience headsets in real-time.
The intention of this project is to create an altered state experience and to discern if the audience have an awareness that 'the instrument', as it is understood, to be sonified and VR responsive gestures, is controlled by a human.
I focus on case studies and pilot tests to investigate these areas. This paper analyses five case studies, focusing on key demonstrations, peer one-on-one demonstrations and feedback through self-report questionnaire development.
This work is in alignment with notions presented by Erika Fischer-Lichte and Max Hermann, regarding theatre and performance as a spatial, embodied event; something which has energy and sensation (Carlson,p. 6). I draw on Mark Hansen's ideas around bodily affectivity and the affective capacity of the human subject by way of digital content in "tingeing or flavouring the embodied perceptual present" (Hansen 2004b, p.605;Clough, p. 212). This work also proceeds with an awareness of Karan Barad's Agential Realism and Eleanor Dare's manifesto and ideas around co-emergent possibilities, accountability, and the role of subjectivity in VR, in relation to Bertold Brecht's A-Effect.

FEEDBACK
Utilising gaming technology for creative experimental purposes lends itself to audience interpretation, which I am directing through aesthetic choices. The aim of this work is to create a meditative altered state experience. Receiving feedback of a Science Fiction ' Marsy 'or ' Gamey ' feel for instance lead me to orient the VR environment in an abstract direction. To provide a container for the participants, there is a need here either for a strong narrative, which would position the work in a VR film or gaming framework, or for an abstract approach -one which aims to create sensory reactions to colour, sounds and crucially to fellow participants as well.
Working with minimal forms allows me to veer away from figurative, literal representations of realworld objects and their interpretations. Transitioning between scenes presents a break in the virtual experience and testing has been done on versions with a hard switch, a seven second fade and a shorter fade out. Participants are very aware of particle trail behaviour. There is an obvious 'feeling of presence and connection' when the trails are anchored towards them in VR, and this was preferred over trails which did not do this. It adds to a sense of feeling 'more like you're there' and it feels more present than dislocated.
Another reactive response from participants is that of passive versus active viewing or listening. From the pilot tests thus far I have come to understand that gamer participants generally want to be more active, potentially controlling the environment, the story line or the situation. The inclusion of natural elements such as air or water do create a visceral expectation as well. 'The brain wants to connect things' in relation to this issue of presence and immersion. The staging of sound is critical for immersion and one particular participant felt the visual element of the piece is 'all around' so the sound needs to be as well.
During initial case studies another aspect of this work emerged, that of witnessing the performer move, creating gesture and controlling the audiovisuals in VR, whilst not in VR themselves. On a practical level this situation arose due to the amount of VR equipment available during initial pilot testing, but it has opened up a new and very interesting audience feedback loop, which I had not intended. In particular one participant said this had a 'ceremonial' or 'ritualistic' essence. He thought it was very interesting to explore the divide between being in and out of VR.
It opens up questions about performance, communal experience and what people do in VR. It raises the question -what does it mean to be part of an audience? He also saw a link between secular group experiences and non-secular group experiences, specifically related to blurred lines between being in VR and not in VR. For instance in a larger exhibition or performance setting, everyone in the room would be a part of the 'religious experience' but only the ones with the headsets on would be having the 'trance-like' experience.
This particular mixed reality audience, which combines those in VR and those not in VR who are able to gaze at the proceedings, brought up ideas around extended performance in four distinct ways: alternative ways of seeing performance, new ways of engaging with it and sitting amongst it and new forms of dancing or performative movement. The ceremonial attributes were mentioned by one particular participant. This instigated further ideas for public exhibition of the work, in that those wanting to take part in the more frontline aspects of 'a ceremony' could be in full VR, and those that choose a more observational role could be audience out of VR. This feedback will be explored during public pilot tests further this year, purposely including audience both in and out of VR. Through advice from Dr Luke, we have communicated about the 5D-ASC and the 11D-ASC, which David et al used for the experiment on Mirza's Chamber. In addition to this, Luke has introduced me to the MODTAS, which is used to measure the susceptibility of people to be hypnotised. There are lines of questioning in the 5D-ASC which have a negative framework in that there is a focus on topics such as 'feeling threatened' or 'no longer having a will of one's own'. It is the most commonly used method of measuring altered states of consciousness, including meditation, daydreaming, hypnagogic and hypnopompic states. To clarify, the 5D-ASC and the 11D-ASC are essentially the same test, but the way in which they are evaluated is different and the 11D-ASC has further markers and a greater number of data points.

CONCLUSION
This research consists of case studies that have concentrated on small groups and one-on-one demonstrations. The next phase will produce another cycle of pilot tests, focusing on group experience. This will include four participants and one performer.
Findings thus far indicate the experience does garner a calming affect. Also, at present audience members do not perceive a human performer or conductor. All but one case study showed this was not obvious to the participants. Interestingly, the one case study which did perceive a connection, is a professional musician, and much of the feedback focus in this instance was referential to the psychoacoustics of the piece and symbolic meanings of 'actioning' the work.
For this peer, there was an obvious link that the changes and progressions in the work were being controlled by the performer. It is necessary to state here that this particular participant has key knowledge of my practise, in particular my gestural AV performative work, which utilises a Kinect camera and was a precursor to my current work in VR.