ShakesVR : Redefining narrative strategies for linear storytelling in a fully immersive environment

The author posits that the author-driven approach of storytelling, resulting in a linear narrative, remains a valid approach for VR. Therefore, the paper investigates the question how a traditional storyline can be told in VR, while simultaneously addressing the specific issues that arise from the audience becoming part of the story. For instance, how might the actions of the viewer (who in traditional film would have passively consumed the story) affect the way in which the story is being experienced? In a fully immersive environment, the user can freely choose where to look and this can affect the experience of the story. How can we avoid that he/she misses essential parts of a linear storyline? Our research project therefore investigated how VR can accommodate the needs of traditional narrative forms. In this essay the author will demonstrate how the integration of interactive elements and audio-visual storytelling strategies can actually support the understanding of a linear narrative in a fully immersive environment.


INTRODUCTION
Over the last three years (2017-2020) the interdisciplinary research project From Print to Digital: Re-Defining Narrativity for Interactive Digital Media, was funded by the Ministry of Education in Singapore (Tier 2 grant). It explored a Shakespeare adaptation for storytelling in immersive environments (VR). This investigation was carried out in close collaboration with the Shakespeare Institute in Stratford-upon-Avon, namely its Director, Prof. Michael Dobson.
Our research method built from initial scholarly research towards the completion of a practical component that served as an experiment to test our theories. Three major Shakespeare plays (Macbeth, The Tempest; A Midsummer Night's Dream) were adapted as abridged versions for animated VR, completed in January 2020. Three different worlds were created that corresponded with the respective plays. The user may discover these worlds through a narrative that connects plays and worlds. In this paper, it is proposed that the author-driven approach of storytelling, resulting in a linear narrative, remains a valid approach for VR. Therefore, this paper investigates the question how a traditional storyline can be told in VR, while simultaneously addressing the specific issues that arise from the audience becoming part of the story. For instance, how might the actions of the viewer (who in traditional film would have passively consumed the story) affect the way in which the story is being experienced? In a fully immersive environment, the user can freely choose where to look and this can affect the experience of the story.
However, the question arises of how can we avoid the audience missing essential parts of a linear storyline.
Our research project therefore investigated how VR can accommodate the needs of traditional narrative forms. In this essay the author will demonstrate how the integration of interactive elements and audio-visual storytelling strategies can actually support the understanding of a linear narrative in a fully immersive environment.
To address the first question, one might ask why the author aimed to use traditional (linear) storytelling within a fully immersive world. Would this environment not naturally lead itself more towards a non-linear approach that would allow the user to drive the narrative? This would obviously have the advantage that the user is in full control of the story and his attention would be naturally directed where it is supposed to be (by and large, whereas limitations in practice might apply). In other words: why not simply adapt a more 'gamified' approach instead of 'sticking' with seemingly outdated narrative concepts as established in traditional filmmaking? Yet, there are strong arguments for an authordriven approach that still favours a traditional sequence of events. Moreover, the paper will also address the question how cultural heritage such as Shakespeare's classical literary canon can be adapted adequately for virtual media. It will unpack the connection between scholarly analysis of source material, the writing of the abridged adaptations, and the development of the corresponding visual adaptation strategies.

WHY A LINEAR NARRATIVE IN A FULLY IMMERSIVE ENVIRONMENT?
A wide variety of professional and academic sources attest to the fact that audiences are still interested to experience a story that emerges from an author-driven vision. This is an opinion that is shared throughout state-of-the-art studios in VR and digital animation. Oculus Story Studio was a VR production company led by creative director Saschka Unseld and technical director Max Planck. It was founded in 2014 but had to close after three creatively successful productions in 2017. Rall (2019) notes that: Since their first foray into VR they have been among the most frequently quoted and prominently discussed artists in the field, as a high number of related online publications attests to. Thill (2015) also quotes Planck accordingly: The reason we currently love film is that we can identify with the director's taste and vision.
That's what you're buying into.
The point being made here is that there is a fundamental audience (or user) interest in the authorial expression of an artist, instead of 'merely' driving or even inventing the story him/herself. This can be demonstrated by the fact that (apart from in the games world) linear storytelling has prevailed as the pre-dominant choice for narrative method. Movies in theatrical distribution and VOD distribution rely almost completely on linear storytelling, interactive narratives never made it beyond the odd experiment in commercial distribution. The same can be said for TV and home video. But why should such considerations bear any significance towards the fully immersive medium of VR? Should not all such experiences be using a fully gamified, non/linear narrative? When referring to storytelling in VR, Pixar animation cofounder Ed Catmull (as quoted by Dredge 2015) states that: Linear narrative is an artfully directed telling of a story, where the lighting and the sound is all for a very clear purpose. You're not just wandering around in the world... The fact that you've changed the technology, and people are excited about it, doesn't change the underlying difficulty of the compelling narrative story.
A good case in point for this theory is the fact that the long-time world-leading studio for VRproductions, Google Spotlight Stories (2011-2019), relied heavily on linear storytelling as is evidenced by several of their productions. These were also released as traditional theatrical releases, most prominently the Oscar-nominated Pearl (2016)  The next section will explore how this problem was addressed in the project ShakesVR.

STRATEGIES FOR DIRECTING ATTENTION IN VR TOWARDS THE NARRATIVE
For a start, it will be useful to reintroduce the major practical problem for delivering a linear narrative in VR. If the user is free to look wherever he/she chooses at any given point in time, how can it be avoided that they miss any crucial part of storytelling? To use an example: If story-relevant action takes place in location A of the VR-world, but the users choose to look at point B instead, they might not be able to understand the story anymore. ShakesVR was facing precisely this problem, because an interconnected story is told that unfolds largely on a continuous timeline, with only few instances where interactivity allows the user to specifically trigger events.

Solutions through multi-modal storydelivery
The solution our team chose to opt was to address this challenge by offering the story through a multimodal approach. But what does that mean in practice? Simply put, it means that it is possible to follow and understand the story through multiple mediations and locations. If an event is represented through audio and video, there is a high likeliness that the user can at least here it, if not see it. A concrete example from our project can demonstrate this aptly. In our version of Macbeth, the original story is transposed to the setting of a 19 th century British naval vessel. King Duncan has become Captain Duncan, Macbeth his first officer, Macduff is another officer (third in command). A central story point is that the MacDuff is washed overboard by a gigantic wave. At the Annual Conference of the Society for Animation Studies in Lisbon 2019, ShakesVR was tested by a total of 27 participants of the conference who were allowed to spend as much time as they needed to experience the story in full.
In the case of the aforementioned story-point, all users were able to 'get it'. Even if they did not see the event, they heard the sound of the wave washing Macduff overboard and his subsequent scream. Subsequently the VR world changed entirely to a beach where all the users saw a shipwrecked Macduff lying. The combination of video and audio with multi-location-based information made it virtually impossible for the audience to miss out on this crucial story point.
Another strategy to enable better storycommunication in a fully immersive environment is to offer the narrative content simultaneously in multiple locations. When in our story Captain Duncan is murdered in his cabin by the mutineer Macbeth, the user does not have to be there to learn of the gruesome deed. Even if he is still below deck, he will hear the (muffled) sound of the murder, and see blood dripping from the ceiling. This provides two different locations, and two different ways to audio-visually experience the same story. But what will happen, if the user is actually on the upper deck, but does not choose not to walk into the cabin where the murder takes place?

Interactive elements to direct attention
Paradoxical as it may seem, interactivity offers the solution here to help the understanding of the linear storytelling. By defining a localised threshold (the entrance to the cabin), the murder itself will only take place if the user actually enters the captain's cabin.
To achieve a satisfactory delivery of the narrative as a whole, the coding must accommodate all the above-mentioned solutions accordingly: • In case the user remains under the deck, the murder must unfold on a fixed timeline, and must be experienceable in any location below deck (which is not really a problem, as this is one big open space). • If the user decides to move upstairs, the coding must implement the thresholdsolution to make sure the murder only unfolds when the user enters into the cabin.
By combining all of these methods, our tests in Lisbon 2019 attested to a successful delivery of the narrative content even in (primarily) linear story mode. As such it can provide the narrative thread that connects the three fundamentally different worlds that were created to represent the three different plays: Macbeth, The Tempest and A Midsummer Night's Dream. After the previous examination of the underlying storytelling mechanics, the following investigation will probe into the connection with the source-material: the original Shakespeare-texts. This will address the initial question of how cultural heritage such as Shakespeare's classical literary canon can be adapted adequately for virtual media.

THREE WORLDS FOR VR: MASHING UP THREE SHAKESPEARE-PLAYS
Preceding the actual production of the VR experience, the research team benefitted from the input of one of its major collaborators: Prof. Michael Dobson, the Director of the Shakespeare Institute in Stratford-upon-Avon (Shakespeare's birth and death place). A 2-hour interview with Prof. Dobson in Stratford, December 2016, provided the actual starting point for the concept of the VR-adaptation. As one of the world's leading Shakespeare scholars Dobson was able to offer advice and encouragement towards our approach to 'mash-up' three major Shakespeare-plays in a fully immersive environment The plays were selected according to the advice of Prof. Dobson (2016) based on their capacity for VR and their potential for worldbuilding. All three of the chosen plays include supernatural elements in their storytelling, a quality that distinguishes them for an adaptation in animated VR. Animation has particularly strong capabilities for the depiction of the supernatural. Digital-Shakespeare expert Dr. Erin Sullivan (2016) and renowned animation scholar John Canemaker (2017) similarly attest to the supernatural elements in specific Shakespeare plays, as strong candidates for portrayal through animation in general, especially VR.
Each play also offers rich opportunities for creating fully immersive environments in VR. Besides the naval setting for Macbeth, which was already introduced, two more VR worlds were created for the two other plays. A sunny island that resembles a late 60s Southern California beach-setting is provided for The Tempest. Also, a magical Chinese forest was created for A Midsummer Night's Dream, which quotes different periods of Chinese art history to achieve a quality of timelessness (Figures 2-3). The motivation for such complexity and diversity in the worldbuilding was chosen very deliberately. In doing so our team intend to represent the universality of Shakespeare beyond temporal and cultural borders. Moreover, the 'mash-up' of the chosen stories also allows an overview of the classical canon. In a nutshell, historical plays, comedies and tragedies are all represented and connected through our approach. Mixing storyelements of all three categories provides a microcosmic insight into Shakespeare's narrative macrocosm, where dramatic and the comedic coexist, often in the same play. In the final section, the consequences for implementing these ideas through the storytelling approaches will be discussed.

SUMMARY: WORLD-DISCOVERY AND LINEAR STORYTELLING RECONCILED
As explained previously, the representation of the diversity and richness of Shakespeare's canon through three very different worlds was considered important for this project. At the same time, linear narrative brings these worlds together and gives an idea of the multiplicity of tonal approaches (historic, comedic, tragic). The narrative adaptation-concept for ShakesVR therefore provides the possibility for the users to fully discover the richness of all three worlds, but also enables them to understand the story at all times.
To resolve this seeming dichotomy, the research team developed a toolset that sought to make this possible. As has been demonstrated, this was mainly achieved through a multi-modal and multilocal approach that communicates story independently from the virtual location of the user. Looking forward, the research term intends to apply further perception-studies that will help to refine the initial elements of the toolset and work towards further sophistication for linear and non-linear VRnarratives. In doing the aim of this project is to make a relevant contribution towards the existing body of scholarship that engages with storytelling in VR.