Analogue and Digital Immersive Experiences : What should digital creators learn from live theatre makers ?

In every century, there are different apparatuses that are redefining and refining the notion of immersion. After experiencing panoramas, planetariums or even IMAX movies, we are now longing to see whether VR can really fulfil our expectations. We are ready to offer all our senses and to experience new situations and to be present somewhere else. But is the virtual reality’s medium prepared for these expectations? Meanwhile the immersive theatre’s genre is spreading and engaging the audience for couple of hours into their very peculiar environments. The aim of my research is to point out different features of live theatre that, when combined with or added to VRexperiences, can enhance the immersion and create more realistic illusions. My investigation is related to narratology and reception studies, and is based on recent findings related to VR. First I’m going to present through some examples how 360-degree fictional movies are exploring their specialties by transferring the narrative literally into the viewers head (e.g. Alteration by Jerome Blanquet, or Rhizomat VR by Mona el Gammal) as a way to capture the viewers empathy more. In immersive performances the audience is considered a participant with a given role. This way the narrative environment is constructed in such a way that allows the participants to unfold the story by directing their focus to carefully planned details. This does not make the participants the story’s protagonist, but they can become the protagonist of the experience itself. They can have a variety of experiences that are directly addressing all of their senses creating more suspense. Based on different examples, I will point out various tactics that VR-movie creators can use to draft their own language and offer more inputs for their viewers. I will also point out how to create such environments for using the VR HMD that can dramatically be built into a viewing situation, without having the forced isolation feeling. My research is based on an audience survey made in June 2017 in Mannheim following a performance by SIGNA, on several interviews conducted with immersive theatre makers, and also on hermeneutical analysis of different VR creations.


INTRODUCTION
"Immersion" and "getting immersed" are the new buzzwords and aesthetical taglines for the artworks that promise to "transport" their audience into the world depicted by the artwork.This "transportation", which is not a bodily transportation but a virtual transportation with the help of our senses, is many times an abrupt experience.The medium of performing arts with it characteristics (as defined by Fischer-Lichte (2008) in her book "The Transformative Power of Performance") like bodily co-presence, liveness is already offering a certain interactive experience for its audience, but the medium of performance art merged with an immersive environment can serve as a new ground for liminal experiences.Getting immersed in a virtual reality production means to create a liminal experience for the audience, which is the key for entering an immersive environment.In this environment the audience is actually encountering the characteristics of virtual reality.
In virtual reality, a panoramic view is joined by sensorimotor exploration of an image space that gives the impression of a ''living'' environment.Interactive media have changed our idea of the image into one of a multisensory interactive space of experience with a time frame.In a virtual space, the parameters of time and space can be modified at will, allowing the space to be used for modelling and experiment.The possibility of access to such spaces and communication worldwide via data networks, together with the technique of telepresence opens up a range of new options.Images of the natural world are merged with artificial images in ''mixed realities,'' where it is often impossible to distinguish between original and simulacrum (Grau 2004, p.7).
The process of distinguishing reality from the artwork (which I will call 'production' from now on)by being transported in such a responsive environment is what creates stress and disorientation in the viewer.This disorientation is a liminal experience, and by overcoming this disorientation the viewer can get immersed.In this paper, I will analyse how 360 degree movies are aiming at an immediate absorption of the audience into the production, especially by using a special dramaturgical form of 360 degree movies (and of course the 360 degree panoramic view) that is offering new roles for the audience, as proved by Dolan and Parets (2016).I will point out what dramaturgical strategies these productions are using.Comparing to these productions, I will analyse a performance installation, which, in my approach, is a production that comes closest to the idealistic notion of an immersive performance.In June 2017, I conducted a survey in collaboration with City Theatre of Mannheim, where we asked 201 people participating at the performance installation Das Heuvolk (Strawpeople) by the Danish company SIGNA, during the Schillertage festival.I will provide the results of this survey, where the participants were asked questions such as what was their expectations regarding the performance, whether they had the feeling that they could control the actions in the performance; whether did they have a comparative view of the immersive effect of the performance by comparing it to the medium of video game and VRexperiences, and what was their particular bodily experiences (their "bodily participation experiences" to borrow a term from Griffiths 2008).At the end of my paper I will propose techniques used in immersive performances (as well as game design) that could be integrated into VR-productions in order to smooth the abrupt liminal experience of VR-productions and also to enhance their immersive effect by using liveness.

AUDIENCE AS GHOST PROTAGONIST IN 360-DEGREE VIDEOS
360-degree videos haven't so far found the most suitable design forms that would enhance the characteristics of 360-degree videos.The 360 degree videos cannot offer what immersion actually promises, because, immersion, as Grau puts it, is "mentally absorbing and a process, a change, a passage from one mental state to another... characterised by diminishing critical distance to what is shown and increasing emotional involvement."(Grau 2004, p.12)These videos, though, cannot fully provide this diminished critical distance, because they cannot actually interact with their viewers because of the characteristics of the medium, as their responsiveness is limited to a response to one's head movements.I will analyse two 360-degree VR videos, both of them with an enhanced special dramaturgical characteristic.Usually, the 360-degree videos offer what the traditional 2D films can offer too, to give the audiences the feeling of the passive observer status.Dolan and Parets (2016) identify 4-types of story consumers: Active observant, passive observant, active participant and passive participant.While the passive observant totally belongs to the non-interactive movie characteristics, the passive participant is that is where "The viewer functions as a silent bystander, or pure-recipient of action."(Dolan & Parets 2016).The other two belong merely to Virtual Reality experiences or video games.In what follows, I will present two 360-degree videos that help immersing the passive participant in their story by partly or totally offering them the role of passive protagonist.

How to behave as a passive participant?
Both of my examples, Alteration by Jerome Blanquet and RhizomatVR by Mona el Gammal, are situating the syuzhet of their production into the viewers' mind or imagination.
Mona el Gammal's production RhizomatVR is a futuristic story about a fictional institute called "Institute Für Methode" (IFM); which scans the thoughts and emotions of the people in order to remove their free will.While the viewer as a subject is going through rooms where these scanning procedures are going on, suddenly the underground group Rhizomat intervenes, their members appearing in the form of inner voices and holographic figures, trying to stop the process and free the subject.The viewer, as a passive participant, is going through this process while passing through the narrative environment created by the director of the film (that originally served as an installation), become the passive protagonist of a sci-fi scenario, enduring different procedures of thought reading.The viewer is provided with a context, with an initiation into the situation and explanation of the reason why the viewer is there, and this smooths out the first reactions of the viewer.But as opposed to the smooth introduction, the 360-degree video also uses smash cuts later, which actually re-creates the surprise liminality characteristic of a first-scene.
The production is also challenging the viewer's position, by constantly changing the "setting" and using inner voice effects.This way the production is tackling the boring dramaturgical issue of the special, medium-sized content and using the metaphor of the viewer's head as the actual spot where the syuhzet is taking place.Also, the story's narrative and constant changes of scene, as well as the inner voice, is used to confuse the vieweras the statement that the viewer is subjected to in the beginning: "We are going to connect ourselves to your thoughts".
In a similar way Alteration is also using the inner voice effect, but here the viewer remains more as a passive observant than a participant, and the dramaturgy is connected more to the protagonist's dreams, and the protagonist is not the viewer in this case.While the latter example makes us of the perspective of first person shooters, Alteration is the third person shooter type of movie (if we analyse it from a game design perspective).The production presents an experiment that is set in 2020, and aims, with the presence of an AI robot called Elsa, at mapping the pattern of human behaviour.The movie deals with the issue of dreams, psychological patterns and the emotions of an AI character, and is continuously confusing the viewer, disturbing her expectations.The video uses the panoramic and motion-sensitive feature of VR, so in its case sometimes-unexpected things can happen outside the visual field of the viewer while someone is interacting with her (an effect considered characteristic of the medium of VR).This confusing effect continues as the video jumps in time sequences, sometimes to the past of the protagonist and sometimes to his present, until the protagonist dies.The viewer becomes omniscient as she also hears the thoughts of the main protagonist, Alejandro, whose thoughts we continuously hear and whose dreams and imaginings we always see.The viewer is set there as an invisible ghost, who can witness the experiment, including the scenes where the protagonist is not present.Even though its dramaturgy is making the viewer feel more passive, the above mentioned third person view provides the viewer with more chances to reflect on the happenings, and to stay more focused (Isbister 2016, p.40) compared to RhizomatVR.

Passive participant in first person and third person shooter mood: how to create the feeling of personalised story with a hint of interactivity?
While Alteration confuses its viewer by its third person perspective and also with distraction effects, the viewer is treated as an omniscient witness, and therefore the events do not touch her that deeply.In RhizomatVR the storyline is affects the viewer as she has the role of the central protagonist of the story.The environments in the productions can be conceived as narrative environments, therefore there is a huge visual difference between the scenes, such as the scene where the viewer is led through the IFM rooms and when Rhizomat is attacking, the former scene taking place in environments with clear, white surfaces, and the latter one in more subterranean, chaotic places.Even though the production does not use special effects of VR productions, the characteristics of 360-degree movie is enhanced content-wise, making it tend more towards interactivity.

IMMERSING INTO A PERFORMANCE
The tag "immersive theatre performance" started to be widespread among theatre creators, but can actually create a truly immersive environment, that makes the viewer completely forget about the outside world.An exception to this is the Danish company SIGNA, which creates hyper realistic productions in multi-storey buildings with usually more than 30 actors.One of their latest productions is Das Heuvolk (Strawpeople), which was presented in 2017 at Schillertage festival in Mannheim, Germany.Their productions are famous for taking place in big houses for approximately one month, and by their site-specificcharacteristics they can create a sense of being in a parallel universe, where their audience can wonder around from room to room for a whole evening.During this time they can chat with the numerous cast of actors, find out the rules of their system, eat, drink and get immersed in their world.
Here the audience had the chance to spend 6 hours with a sect whose members are Himmelfahrers (Ascensioners, those who can go to heaven) and whose leader died not long ago.Their daily life is busy: They are continually making rituals to incorporate their gods.According to their mythology they already found ten out of twelve gods and in order to find all of them before the world is ending, they are struggling in different means to search for the remaining two.If they find them in time, they won't burn like "the rest" (Heuvolk meaning Straw People, those who are not initiated into this cult).Meanwhile they are also looking for new members for their sect, so they are very happy to have visitors (audience) as some of them might turn out to be chosen Himmelfahrers.
Can this performance by SIGNA considered an immersive performance?By closely analysing the characteristics of immersive performances we can list the following features that make a performance immersive:  Panoramic physical environment (however limited), which the members of the audience can explore in very fine detail.
 Involves all senses of the audience members. The medium of immersive performance bears no self-reference (for example it is not based on an existing well-known theatre performance). Anything that independently happens in the vicinity of the performance (such as the ambient sound from the surrounding environment) can be integrated into the story world of the performance. The performers of an immersive performance don't have a fixed text, but rather a character, that they express through improvisation and interacting with the members of the audience. The audience can have the sense that they have the control of the plot (even when they don't.) SIGNA has all these above listed characteristics essential to an immersive theatre performance.
Here, the actors don't have a fixed text through which they control the narrative path, rather playing out the syuzhet through their actions and interactions, and it is important to mention that as the performance progresses the performers also memorise the names of the members of the audience, learn a bit about their background, and keep in mind the interactions with the members throughout the performance and recall these interactions when they meet the same audience members when they come across later in the performance, as if they have become acquaintances.Also, the performers give small tasks to the audience: To bring food from one floor to the other one, to feed someone, to light up a cigarette, to help someone else wake up, etc.This is also a tool that helps the audience to feel that they are being an active part of the events that unfold throughout the performance.In SIGNA's performance the costume and set design create a unique and characteristic world that is somehow also leading the audience through all the spaces of the venue.(It is important to mention though that Mona el Gammal, the creator of RhizomatVR, was also previously working as a set designer for SIGNA, which might explain some elements in her productions.)Deliberately or not, SIGNA also makes use of what I would like to call the "checklist effect" which makes the audience more curious, creating thematic spots (either thematic rooms or special small communities within the fictional community) which makes the audience want to go through these thematic spots exhaustively and not miss anything.This might be enhancing factor for the immersive effect in relation to the members of the audience being able to explore the performance environment in fine detail.

Measuring the level of immersion
During June 2017 in cooperation with the Mannheim City Theatre I conducted an audience research with the audience of SIGNA's production.Almost half of those who answered the survey said they knew what to expect.As for the question of how they would define the genre of this production, besides providing definitions such as "performance installation" (as the company itself defines their productions) or "immersive theatre" many of them used the concept of "horror" and "psycho thriller", and even a made-up notion like "controlled video game".On the question how many of them let themselves to be totally immersed into the world of the performance, more than 60% answered that they let themselves to be totally immersed or to flow together with the events, while around 20% didn't want to let themselves go or partially let so.Almost 70% considered themselves as an active participant and 18% considered themselves as partially active.Here is important to mention that also those who answered the question of how much they let themselves immersed into the story world of the performance with "partially" or "not at all" admitted here that they were active participants.Another question asked the participants how much did they feel that they can control the happenings/events and whether they felt that they could hack the system of the performance, and from 201 participants 40 answered that they felt the events under their control and also that they can hack the system of the performance; 60 sometimes felt that they lost control or wanted to lose the control; 72 totally let themselves go and let anything happen to them.Many, more than 65%, answered that they would come back to the performance one more time and they would be a more active participant, as much as the performance's system allows, or they would visit more rooms in the performance.On the question of whether they had any novel bodily sensation, almost 60% answered with yes.Among these were both positive and negative sensations: Some felt excitement, or even anger, and also there were participants who felt nausea, disgust or insecurity.
In order to analyse the level of immersion from the point of view of other mediums, I introduced two questions into the survey, one related to video games and the other related to VR-experiences, in order to test the video game-literacy or VR-literacy of the audience members and to measure the level of their immersion from this reference point, as video games and VR-experiences are considered as mediums with a big potential for immersion.Even though we can assume that many of them had some experience with video games, from the total 201 answerers only 36 answered that there was (but a limited) resemblance between the performance and video games medium, with many participants also mentioning Live Action Role Playing games as well.In the case of the VRexperiences, 65% haven't yet tried out the goggles, and from those who have tried only 18 said that there might be a resemblance but the performance was a much more intensive experience.

CONCLUSION
Besides the phenomenological aspect of bodily copresence, what creates the sense of a more intense experience for the members of the audience of an immersive performance compared to a VR production?One of the most important features of immersion is semantic responsiveness, and it should be a must for VR-productions that aim to be intense, for 360-degree videos as well as VR experiences.This technique is already used in some interactive videos, such as in the works of Blast Theory.This technique of semantic responsiveness can have many forms, some more transparent for the users and some less, and its mediums can be various, as the transmedia story world can allow it.
Also it is importantalso related to the story world tooto design certain mechanics that are smoothing out the liminality level of the immersion, especially the first encounter with the immersive environment.As we could see in the case of SIGNA's production, the members of the audience were given an explanation why they stepped into that world on that specific time and what is the audience's role or task while visiting the story world of the performance.(Of course all these are stated by the performance in reference to the fictional world of the performance.)But, at the end, it seems that the most crucial aspect that can be imported from among the characteristics of the medium of performance art is liveness.This feature is hard to establish in virtual reality, but it is an aspect that can create a higher level of immersion for the VR-production viewer.Many participants of SIGNA's performance compared it to a horror movie, and an effect similar to the effect of shock typical to a horror movie or a thriller can be created in VR productions as well, with the help of the sense of liveness, and the sudden co-presence of an unfamiliar body.The emerging technologies offer new ways of experimenting with such effects, and some productions are already beginning to take up such experiments.