Narrating the Past: Virtual Environments and Narrative

This paper explores how traditional narrative language used in film and theatre can be adapted to create interactivity and a greater sense of presence in the virtual heritage environment. It focuses on the fundamental principles of narrative required to create immersion and presence and investigates methods of embedding intangible social histories into these environments. These issues are explored in a case study of Greens Mill in the 1830's, interweaving the story of the reform bill riots in Nottingham with the life of George Green, mathematician and proprietor of the Mill.


INTRODUCTION "Like a good narrative plot, VR systems should instill an element of surprise in the fulfillment of expectation."
Marie-Laure Ryan In the last decade there has been a growth in the use of Virtual Reality technologies for reconstructing heritage sites.It has reached such an advanced stage that accurate reconstruction in high fidelity has become commonplace.Viewers can now 'walk' or 'fly through' virtual models and have something of an experience of 'being there'.However despite these advances in technology, 3D reconstructions still lack a sense of presence and are often sterile, clean spaces where the viewer's experience is limited by the architecture of the site.These virtual models satisfy the desire for architectural description of a site but when they are exhibited in museums and heritage sites, the same attention given to contextualising real artefacts needs to be considered for these virtual artefacts.Incorporating aspects of social and cultural history by embedding audiovisual narrative into a model can assist in generating a deeper understanding of the historical site hence, more accessible.
Traditional museums and heritage sites have evolved from the 'cabinets of curiosity' focusing on the authority of the voice organising content.Contemporary museums and heritage sites tend to try to break down the division between audience and museum.Heritage sites in particular often recreate and re-interpret the site's history through thematic trails and guided tours.In these tours the social and cultural history of the site is re-enacted through a unique interpretation of historical facts mixed in with memorable stories.Film and theatrical techniques are often used to transport visitors back in time with evocative sound effects and captivating narrative."[1] A good storyteller can conjure up and animate the spirit of history.The power of these stories should not be underestimated -they are significant in enriching the visitors' experience of historical memories and culture.They have been demonstrated to be engaging and entertaining as well as educational, whether they are live re-enactments or self guided audio tours.Cameron and Gate wood's [2] study into what the public wants from historical sites found that people value, "developing a feeling for x,.., connecting with the past… or using the mind to experience.Revealing that what people want is to engage their minds and emotions with what they are viewing." It is within this context that this research project is positioned.It aims to investigate devices intended to immerse the audience in virtual heritage environments and to promote a greater sense of social engagement with historical narrative.The authors of the paper are multidisciplinary multimedia practitioners, specialising in the disciplines of scenography and film making, so the project is also concerned with redeploying the devices of traditional narratives and the conventions of cinema and theatre to promote engagement within real-time virtual heritage environments

The Sites
The project interconnects three of Nottingham's heritage sites, -Green's Mill, Nottingham Castle and the Galleries of Justice -with a narrative based around the events of the Reform bill riots of 1831.The sites are all currently open to visitors and are about 25 minutes walk from each other and firmly embedded in the City's tourist trail.
So why reconstruct these sites virtually?The initial meetings with the site mangers and curators revealed that some of the key areas within the sites were inaccessible and could not be adapted due to building restrictions.On a purely pragmatic level virtual models of the sites will fulfil an immediate need.After several visits a clearer perspective on how the sites interpret their social history was obtained.Each site offers visitors guided tours and re-enactments.Green Mills is a fully functioning flour mill and the tour invites visitors to investigate the windmill and its cogs, to weigh and measure grain and flour.As well as making bread, listening to a story and meeting the mill's characters.The 'crime and punishment tour' around the Galleries of Justice, re-enacts a trial in the original Victorian Courtroom where friends and family can be put in the dock before being sentenced, and `sent down` to the prison cells, laundry, medieval caves and prison exercise yard.Prisoners and gaolers act as guides.Beneath Nottingham Castle are manmade caves and tunnels, some of which date back to medieval times.Visitors can take a guided tour to see Mortimer's Hole, King David's dungeon and the old wine cellar hearing on the way gruesome tales of Roger Mortimer and King David of Scotland.
It seemed essential that the virtual model should not be perceived as a limited version of the real tours but be regarded with the same objectivity.Therefore it should be in real-time, interactive with embedded narratives that are just as compelling and engaging as the real tours.In an observational study of virtual visitors within 3D web-based museum environments, Lin, Higget, and Baines, [3] found that 'attraction levels were highest for the exhibits which employed multiple media formats and 3D models combined with rich information content; holding power was highest for the exhibits which used games or a video with high levels of interaction.' The Narratives It was decided to set the narrative in the period of the Reform Bill riots (1831) because it linked one historical incident with the three sites.The rioters came knocking at Green's Mill but they were scared off by George Green; however they were successful in setting fire to parts of the Nottingham Castle.In 1832 some of them had been caught and imprisoned, tried and hung at the Galleries of Justice.
The initial phase of the research concentrated on the narrative for the Green's Mill site.Mr Green started building his mill in 1807 and it was one of the most powerful and advanced mills in and around Nottingham at the time.It is situated in the Sneinton district of the city which in 1831 was described as a small village just outside of Nottingham.By the 1880's with the growth in new industrial machinery and steam power the mill became non-profitable and was left in disrepair until 1985 when it was restored as a memorial to George Green, the mathematician and son of the original owner.The Mill has had a chequered history and there are many viewpoints from which the narrative can be told for example, as a working flourmill, the story of a remarkable mathematician who remained a miller all his life or as an example of the technology of the past.For the first phase of this research the narratives concentrate on Green and the riots.
Much of the history is intangible, particular that pre-dating modern recording methods.Finding material was difficult, in some cases there were only snippets of information, which we reinterpreted from maps, newspaper clippings and the books of local historian.In order to tell a compelling and dramatic story we procured the help of the writer Michael Eaton [4] who further developed our initial story lines into a well structured narrative.

THE NARRATIVE ARCHITECTURE
Our approach to the narrative architecture in a virtual environment (VE) can be compared to promenade theatre, where the audience inhabit the space, rather than just watching.They moves around to view the action and sometimes interact with it; such as in Punch Drunk's site-specific promenade performance of Faust. .[5] Set in a large warehouse, the spectators encountered scenographic spaces and actors performing simultaneously through the building.As a result everyone's journey differed.Bruno [6] describes the act of wandering as one of connecting."She who wanders through a building or a site acts precisely like a film spectator absorbing and connecting visual spaces.The changing position of a body in space creates both architectural and cinematic grounds.The consumer of architectural (viewing) space is the prototype of the film spectator." In the Placeholder, virtual reality (VR) installation, the participants wandered around the three locations -a cave, a waterfall, and a river valley.It used videographic scene elements, spatialised sounds, words and simple character animation.It was developed in mid 90's and was one of the first VR projects to grappled with the issues of creating narrative spaces that were not connected to a plot or quest.The makers Laurel, Strickland and Tow [7] were interested in creating a 'sense of place' and saw the virtual environment "imbued with narrative potential -places that could be experienced and marked through narrative activity.When a person visits a place, the stories that are told about it -by companions, by rock art or graffiti, or even by oneself through memories or fantasies -become part of the character of the place." Likewise in the Green's Mill VE, we are interested in creating a 'sense of place' as the participants wander through the environment they encounters several video and audio narratives that are designed to immerse them further into the virtual experience .The narratives are small "story-worlds" [8] as seen in figure1; they do not follow a linear story or are related to a macro plot.However a narrative coherence still exists as all the stories have the Reform bill riots as their common the thread.Nowadays, designers of computer games like Oblivion are realising the need to design spaces that are not only related to the quest but give their participants the freedom to explore and wander around.It should be noted that we chose not to use avatars for the narratives, the participant's point of view (POV) is the virtual camera and the video that augments the space is of real actors that we filmed.Film techniques and conventions are becoming more prevalent in current video games.The addition of interactivity, the placement of the "viewer" into the "players" position, and the VE as a "place" that the "player" occupies, offers opportunities for developing narrative spaces and techniques for telling stories.In particular the use of the POV in VE allows the user to feel part of the action.The POV works in a similar way as in films, putting the viewer into the story space.Film is a time based medium and the editing techniques used allow us to compress time, flash backwards and forwards.The limitations of POV in VE centre on it's non-editable 'real-timeness' -in film these shots are used by the narrator in specific situations and not continuously as in the case of VE.(The Blair Witch project used POV but also had the advantage of cutting when the video camera was switched off) There is a long and well documented history of techniques used by film and theatre that creates scenarios for the "willing suspension of disbelief" the need to make an experience "real" from the audience point of view.As disbelief is a conscious decision made by the participant, creating such a situation in a VE could be problematic.However Marie-Laure Ryan suggests in her book Narrative as Virtual Reality that some VR projects found that there is a tendency for the participants to suspend his/her disbelief in an interactive and physically immersive environment.

Mise en scène
Creating engaging narrative is not limited to the multimedia elements encountered.The mise en scène is just as important to consider within virtual environments as it is in film and theatre.The nature of the POV means that the VE designer cannot direct the narrative as can be done in film e.g. a close up allows a director to point the viewer to objects in a scene.This limitation of control, over the participant's view means the VE designer needs to rely greatly upon the mise en scène to direct the narrative.(Clarke& Mitchell 2001) [9] Mise en scène is defined by film theorist Robert Kolker [10] as: "the use of space within the frame: the placement of actors and props, the relationship of the camera to the space in front of it, camera movement, the use of colour or black and white, lighting, the size of the screen frame itself".Mise en scène can be used to set the "mood" of the narrative, provide a perspective upon the story and set the scene.David Dernie, [11] in his book Exhibition Design refers to narrative as a central element in exhibition design today.He says, "Narrative space can be about a simple relationship between a single object and its setting in space, a question of light and shadow, reflections and material configuration which evoke visual correspondences and engagement" The angle of view allows the director to use perspective so that say, a hand held shot at head height will put the viewer into the scene as though they are looking through the protagonist's eyes.Alternatively a view of a person from below suggests the POV of a smaller person thus making the person in shot seem bigger and dominant to the viewer.When using this particular reference of perspective (the average height of a person) the audience identify with this POV as in real life (Silverman 1989) [12].Unlike film, the point of view (POV) of participants cannot be controlled by the authors, therefore the mise en scène can be more difficult to frame In The Castle Burns narrative for Green's Mill we have attempted to recreate at fast dramatic scene around the participant to hold his/her attention to the story (see figure 2).The narrative is set around a copy of the Nottingham Herald newspaper on a table in the top room of the Mill.The newspaper's front page shows an article and lithograph of the rioter's attack on Nottingham Castle.A sound of a crackling fire is placed around the area of the newspaper; the participant approaches, as he does so he/she triggers a second narrative which is 'set' on the front page of the newspaper gradually becomes immersed into the narrative space.The lithograph's image is animated and slowly takes the form of a 3d model of the castle in flames.At the same time the participant hears a sound bite of the Town Crier telling the story of the riot.
As the story progresses the VE environment around the participant turns dark, the lanterns in the top room provide dim light and the room is swathed in flickering fire effects (as if a fire burns outside the mill).The animated image in the newspaper emits smoke as if it appearing to come from the castle scene; this fills the top room of the mill creating a dim smoky environment.Once the narrative is completed the VE is transformed back to the daytime and the animation and audio ends.
This scene is similar to a flash back in film and the devising process and timing of the effects were cued as it is done in theatre.By creating 'spatial simultaneity' and considering the mise en scène; special effects, animation, triggers, lighting and proximity sound the participant is more likely to become immersed and experience a sense of presence.In Oblivion when a narrative needs to be conveyed to the players they are frozen on the spot (they can still adjust their POV) until it is completed.This reduces the interactivity of the players but forces them to view the narrative content.

Transitions
One of the problems facing designers of VE is the handling of transitions, if they are not done well the participant can lose their focus and a sense of immersion.Transitions are devices used to join image and sound together in sequence.Frequently a film director will aim to make the join "seamless".The simple cut is made more seamless by using shots that are not too dissimilar in composition or too different in tone and quality -a cut will be used to change the view of the scene suggesting simultaneous action.The change in sound and image made when cutting from shot to shot can be made smoother by staggering the changes -bringing the sound of the next shot in before the vision changes helps to divert attention away from the cut.A mix transition where image and sound cross fade, will translate as "some time later" or "meanwhile somewhere else" (depending on the narrative content).To make this transition more seamless the director can match the compositions of the shots that end the first scene and the beginning of the second.This is referred to as match editing-the focus of the eye is not disturbed as it holds the two similar forms in focus, diverting the eye and ear away from the distinct changes that exist elsewhere in the shots, e.g.changes in location and changes in atmospheric sound.In the Hearson and Beck trail narrative we tried to use some of these transition techniques to seamlessly travel from the outside of Green's Mill to the caves at the Galleries of Justice.This narrative is presented through a poster nailed to a tree situated in the grounds of the mill.The poster (Figure 3) is historically accurate, sourced from an 1832 poster announcing the hanging of Hearson and Beck who were convicted for their part in the riots.The narrative is triggered by the participant's proximity to the poster.It starts an animation of an ink drawing of the hangings.A dramatic audio sequence relates the story of the trial through four different voices.Simultaneously the 3D virtual environment gradually transforms around the participant from Green's Mill to the caves at the Galleries of Justice, acting as a "portal" to another scene.To keep the participant's focus, the only object which teleports with him is the poster.(Figure 4) Computer games have various methods to deal with transitions from freezing the players to teleporting them in the Sims 2, Bon Voyage the characters have a choice of when to teleport by selecting the onscreen menu.This new feature allows the player to skip long journeys within the environment and to immediately teleport to their desired location.This has become an acceptable way to travel within many game environments.

THE ENVIRONMENT Embedded Audiovisual Narrative
Two of the narratives are experienced through video embedded in the VE.Much like video projections are used to augment the theatrical stage.In Greens Monologue as the participant approaches the window opposite Green's desk, on the top floor of the mill he/she triggers an audiovisual narrative delivered by George Green.The video is semitransparent and made to appear as if. it is reflection in the window.This effect was used so that the participant can also have a view the environment outside the widow and see the castle burning.Whiles in the William and Jane narrative the participant is drawn to a conversation but can only see the movements of shadows on cobbles.In this narrative we are investigating whether a sense of presence can be felt by the using audio pronominally as part of scene.
Sound in VE can be very evocative; the use of soundscapes can create a greater sense of spatial immersion and an additional source of information which can help enhance the VE for the participant.Embedded sound effects and proximity sound triggers can also play a role in directing the participant's attention to a particular narrative space.

Building the Virtual Environment
Constructing the Green's Mill model was fairly straight forward as the mill had not structurally change since 1830.Even though it was in disrepair for a long time, most of the mill was restored in the 1980's, except for the miller's cottage which we reconstructed in VE using paintings of the mill and photographs of similar cottages.Still prevalent today on one side of the mill are white paint marks where the cottage was once situated.The reconstruction of the landscape around the mill proved to be a bit more difficult as the earliest map we found was dated 1881 and it was not an ordinance survey (OS) map.(figure5).Therefore we modelled the landscape using a current OS together with the 1881 map.The real-time interaction was done in Quest 3D, (Figure 6) a real time software package which allow non programmers to develop complex interaction by using the pre-designed channels.The experience of designing and putting together the interactive elements feels a bit like a theatre technical rehearsal, as we worked with the digital assets live, testing and retesting different levels of, visual, sound, video and light effects.

CONCLUSIONS
With the current popularity of computer games, the experience of virtual environments have become common place for the younger museum visitor but an older generation may find them difficult to use; so careful consideration of the users' needs have to be facilitated.Extensive testing of the VE with museum visitors will be carried out in the future.We work closely with the sites, our initial presentation of the virtual environments (Figure 7) to the staff at the Galleries of Justice was very fruitful and they understood the goals of the project much better.Finding the best user interface, space and hardware to display the virtual environments could play a part, in increasing the virtual experience of a sense of presence and immersion.Museums have limited space to display VE and they are often confined to a small screen.One of the other collaborators on this project is the Broadway Cinema and Media Centre, situated in the heart of the city; they are interested in experimenting with new ways of utilising their smaller screens.Therefore we are also investigating how the cinema could be linked to the museum sites, e.g. a visitor to the cinema or museum could book a ticket to experience a virtual journey of the sites projected on a large screen in a cinema with immersive surround sound.
We are still in the early stages of this project; but from our initial research we believe that embedded narrative in VE can play significant role in enhancing the museum visitor experience.There are many historical sites already modelled in 3D, these digital assets could be shared and potentially be utilise for embedded narratives.For this to occur we need more academics collaborating in the fields of archaeology, history and art and design.

Figure 1 .
Figure 1.Narrative structure, participants wandering and 'story worlds.'B illustrates the non linear journeys the participant can take.

Figure 2 .
Figure 2. The Castle Burns, the participant hears a sound and approaches the table.

Figure 3 .
Figure 3. Photo of the original 1832 poster announcing the Hearson and Beck trail.

Figure 4 .
Figure 4. Hearson and Beck trail, the participant is transported from one VE to another.

Figure 5 .
Figure 5.A photograph of Green's Mill today and a map of Sneinton area in 1881.

Figure 6 .
Figure 6.Screen shots of an early model of Green's Mill in the real-time environment