Let me understand the poetry : embedding interactive storytelling within panoramic virtual environments

This paper presents a method for the merging of poetry into interactive storytelling that is based on still and video panoramas. This non-linear approach aims to give a young audience a new understanding of poetry, by exploring the poet Charles Causley’s house and the town of Launceston in England where he spent most of his life. His poetry mentions a number of locations from this town and artefacts located in his house. The user of the interactive application based on Adobe Flash plug-in and Lucid Viewer (panoramic viewer), take a number of narrative journeys in order to search for hidden poems, voice marks or trails signs that have references to the town. Still panoramas localised at decision-making points were linked by using video panoramas (360-degree video), which were recorded with a spherical video camera – Ladybug2 mounted on a motorised wheelchair. Still and video panoramas are elements that create a branching narrative. The aim of this application is to develop the interest not only in the Causley’s biography but also in literary output of the poet.


INTRODUCTION
Poetry is not always easy to understand, especially for young people.This paper discusses a new approach to merging interactive narrative with poetry in order to make poems more accessible for this group of readers and moreover to encourage them to write their own poems on the basis of collected objects they find during their adventure within this experimental application.This non-linear project aims to render a new understanding of poetry, where users, after finding a final piece of a puzzle, will be able to consider poetry as both motivating and appealing.Combining still and video panoramas is a new method of visualising sites which was applied to this research project (previously only still panoramas were used for the generation of virtual tours), because they are appropriate for the concept of a branching narrative (interactive storytelling) that is discussed in this paper.
The Causley poems that refer to specific places in the town of Launceston (Figure 1) were visualised using graphical illustrations to enable the search for hidden poems, voice marks or trail signs that are crucial to puzzle solving process.

Figure 1: Causley's poem refers to a specific place in Launceston
To achieve the goal of interactive storytelling, the researchers did not author the story, but provided an environment and context, in which puzzles and hidden poems might be discovered (Meadows, 2003, p. 54).Still and video panoramas are a perfect visualisation tools if opened in a panoramic viewer that allows additional programming.Lucid Viewer (Villmer, 2010) not only displays interactive still panoramas, but also video panoramas and simple 3d models (Figure 2).This viewer is the basis for this educational project.

Figure 2: 3D object inside virtual panoramic environment -a clock with a hidden message at the back
The researchers' approach was to generate location-based panoramic interactive narratives that consist, at the time of writing of 37 still panoramas and 42 video panoramas.In order to allow the user to unfold the story and moreover give the enjoyment and privilage of making choices (Morse, 1996, p. 199) a number of links had to be created in XML programming language.By unfolding the narrative, the user learns about the poet's life and his literary output.
The process of recording video panoramas that link still panoramas is discussed in the third section of this paper, whereas the fourth section discusses the project in more detail.Possible outputs of this narrative (Blu-ray, 360-degree screen, Internet) are also presented.

CHARLES CAUSLEY
Charles Causley (1917Causley ( -2003) ) (Figure 3) was an internationally known and respected English poet who spent most of his life in Launceston, Cornwall, UK.His poetry and the Charles Causley Society's website (Charles Causley Society, 2008) provided the researchers with an inspiration to merge poetry with interactive storytelling using both digital tools and the latest imaging technology.There are many references that link Causley's poetry and objects or sites that are encountered in his home town.Moreover, there are also links between his literary output and artefacts that are in his house.Causley was a teacher in Launceston's National Primary School until 1976, and then he became a full-time poet (Lewis, 2008).He published many collections of poems and Who's Who mentions that his literary output was 'the rediscovery of his native town' (Trewin, 2003).
Causley spent many years in Launceston occupying different houses.For instance, the house where he was born, the church in which he was baptised and the graveyard in which he was buried are situated in the same street.Most local stories and legends are now included in his poetry.These stories are included as part of the interactive narrative, where voiceless objects (a sculpture of Mary Magdalene, sculptures of eagles near Eagle House Hotel, ZigZag stairs) describe narratives in the form of poetry when user of the application clicks the right icon.
Panoramic images and video enable the user to experience the real town, because the images are created photographically using DSLR and a spherical video camera.
Causley's house was visualised by using high resolution panoramic photography to allow for the exploration of objects and artefacts hidden in different spaces of the location.As an example, there is a Causley's poem that relates to a small figure of a bear, this bear is visible within the panoramic environment; after mouse clicking on this object the poem appears and can be used read by the user.
My mother saw a dancing bear By the schoolyard, a day in June.
The keeper stood with chain and bar And whistle-pipe, and played a tune.
The process by which the key elements of the branching narrative are generated in described in the next session.

STILL AND VIDEO PANORAMAS
Still panoramas were created using a DSLR camera (Nikon D90) and Manfrotto 303 SPH panoramic head.360-degree images constitute decision points (marked red on the map -Figure 5) in the interactive storytelling, whereas video panoramas link these points and enable smooth journeys through the town while also exploring the digital narrative.Ladybug2 (Figure 4) -a spherical video camera was used to record the journeys between decision points, these journeys are marked as yellow lines on the map (Figure 5).The locations of these points are dictated by the plan of the streets in Launceston.Most of the main crossroads were visualised using high resolution still panoramas to provide the user with a number of choices at each crossing.As the number of decision points grows, the number of narrative journeys has to grow (Riedl and Young, 2006, p. 24).The map (Figure 5) presents positions of the still panoramas and links between them.In most cases, 360-degree video was recorded in both directions to enable smooth exploration of the interactive narrative.There are also some narrow streets that are recorded only in one direction, due to the length of the streets or the danger imposed by driving a motorised wheelchair upstream.The researchers aimed to record all the links in both directions to achieve branching narrative almost without limitations.

Figure 5: The map of Launceston with marked decision points (still panoramas -red dots) and links between them (video panoramas -yellow lines) -based on the map produced by John Fenn
The application of a motorised scooter and a special helmet was crucial in the realisation of the project (Kwiatek and Woolner, 2009, p. 201).The researchers plan to re-record video panoramas using an improved version of the motorised scooter (Figure 6).This wheelchair was adjusted for Ladybug2 camera in the workshops at the University of Plymouth.It has a special pole at the back that holds the camera above the head of the driver.The camera has to be located above the equipment and the driver's head, because it 'collects video from more than 75% of a sphere' (Point Grey Research Inc, 2010).The next session of recordings will improve the image quality and footage stabilisation.
Video panoramas enable the user to take quick journeys between locations in the town, normally switching between still panoramas can be disorientating.Moreover, video panoramas facilitate individual journeys and the listening to linear narratives between decision points.In this way the user can perceive that they are in the story world and have the potential to pursue their own goals (Riedl and Young, 2006, p. 24).
Large amounts of panoramic content virtually guarantee the users engagement with a branching narrative.Story-graphs (Riedl and Young, 2006, p. 27;Ryan, 2001, p. 246) help the creator of the interactive narrative to generate the best environment for the user.The goal of the interactive narrative is to give the user a welldesigned environment, and not author the story.
The researchers applied Lucid Viewer (Villmer, 2010) to this project because it is a panoramic viewer that displays video panoramas while also allowing the programming the 360-degree content using XML.The concept of using XML for programming panoramic virtual tours became popular when the panoramic viewer SPi-V (Hoeben, 2009) appeared on the market in 2004.Individual actions are linked with buttons (or hotspots) and they allow for the switching from a still to a video panorama and the other way around.Other panoramic viewers KrPano(krpano, 2009), Flash Panorama Player (Chumakov, 2008) or Pano2VR (Rauscher, 2009) are also based on Flash Player (which is mostly installed on every computer (Adobe, 2009)), but the researchers have chosen Lucid Viewer, because it enables the display of 3D objects and handles the display of video panoramas very well.

Interactive Storytelling
Aristotle and his writing Poetics has an impact on the understanding of media, he described narratives that have beginning, middle and end (Hageboelling, 2004, p. 1).The first form of interactivity applied to stories was introduced many years ago by storytellers who changed and adjusted the narrative according to audience's responses (de Sena Caires, 2007, p. 3).This function almost disappeared because of the invention of books and other media, where the spectator has to follow a linear route in the story from the beginning to the end.A number of researchers defined interactive storytelling (Cavazza et al., 2008, p. 14), but Manovich(2002, p. 227) definition seems to be the closest to expectations in this project.Manovich considers an interactive narrative as a database that the user is traversing, where links between input data were established by the creator of that database.Consequently, Manovich(2002, p. 227) defines the concept of interactive storytelling as 'the sum of multiple trajectories through a database'.Manovich's theory seems to fit to this project and a list of still and video panoramas is considered a directory with files within a database.The user has an ability not only to view the visual data in this database, but also traverse a journey prepared by the creator of the interactive storytelling (Manovich, 2002, p. 228) in a sequence that is not chronological.

Description of the project
The opportunity for video panoramas to be embedded in interactive storytelling has not been fully investigated.To create an educational and entertaining product for young audiences, a number of trajectories for branching narrative were recorded.Causley's poetry provides clues and encourages further investigation.This encouragement starts from Causley's house where the user is given their first task.A selected poem provides clues to help find the right answer and at the same time encourages the user to visit other rooms within the house.Once the user of the panoramic application is outside Causley's house, they are introduced to video panoramas for the first time.A puzzle solving process starts and from time to time, while the user is travelling between decision points, animated characters from Causley's poetry emerge to supplement textual content displayed during the presentation.When the 'reader' of this interactive narrative interacts with poems, they have an opportunity to collect objects that appear within the text and place them in a virtual rucksack, the content of which might be printed out at the end of the exploration.For example, the first part of poem As I went down ZigZag consists of four objects that are transferable (draggable) to the virtual rucksack (Figure 7).This printout then acts as the basis of young person's own writing of new poetry that might be submitted for a competition related to Causley's poetry or used as a basis for workshops in schools.
The overall aim of the interactive story is to find the 'Paradise' that Causley describes in on one of his poems: We called it Paradise: a plat of grass, […] See you in Paradise, we'd say.For here Was entrance to another land (Causley, 2000, p. 397) Animated characters (Mr Pennycomequick (Figure 8), Charlie Chaplin, Britannia, Mary Magdalene) are located in different locations in the town, and once the user meets them, they provide clues about the possible position of the Paradise.These characters as well as numerous voiceless objects are gateways for further stories, sometimes with additional clues and quizzes.Interactivity gives the users an encouragement to look for clues that are hidden in, under or behind 3D objects and artefacts and in this way enhances their experience (Danks et al., 2007, p. 112).The researchers created a panoramic environment available for exploration, but did not author the interactive narrative.All data presented in the application are photographic, excluding the illustrations of characters from Causley's poetry and a few graphics located in the interface.After finding the 'Paradise' the user should 'return renewed' (Morse, 1996, p. 209) with a new understanding of poetry and also with a list of objects to be printed out that are the basis for the creation of a new poem.

Interface
The design of the interface of this application has to follow a number of rules (Glassner, 2004, pp. 301-302).It needs to be effective and intuitive for young users.A map was included to help explore the town and find missing information and in this method, to unfold the narrative.The application of icons with tool tips helps to understand the interface of the program, and for example, a small graphic of Mrs Busybody works as a help button (Figure 7).This lady was described by Causley in one of his poems.
Mrs Bessie Busybody, I declare, Knows all the news, And some to spare.(Causley, 1996, p. 320) This character will always support the user of the interactive narrative and give important information whenever it is necessary.Mrs Busybody appears on the overlay layer of the application and she is always visible, whereas for example Mr Pennycomequick is attached to a specific point in one of the still panoramas.His location is at the top of a gate (Figure 7) that leads to the castle in Launceston.
Mr Hector Pennycomequick, Stood on the castle keep, Opened up a carriage-umbrella, And took a mighty leap.(Causley, 1996, p. 307) Once the user reaches this decision point in the branching narrative a voice over is played with a part of the poem to enable the user to find the animated character that has an important message about the location of 'Paradise'.
As the user clicks the animation of Mr Pennycomequick, they will be provided with a clue that will help them to decide where to go in branching narrative (go back or go forward by clicking a hotspot in the form of an arrow).Two interfaces were generated for this project; one for still panoramas and the other for video panoramas.Video panoramas provide buttons to play and pause, whereas they do not appear when exploring still 360-degree images.

Further research
In the future, a new poem created by a young person on the basis of a printout from a rucksack, could also appear on the on-line version of the application.New functions of adding/tagging new information to the existing still and video panoramas will be added, but this needs further investigation and collaboration with other researchers.
The aim is to publish this application on a Blu-ray disk due to the large size of the database with media files.Video panoramas, even compressed using H.264 codec are large files (each file is about 100-500 MB) and only Blu-ray disks give the potential of presenting good quality images on a computer.
The other option of publishing this database of interactive storytelling is the Internet, but files, mainly video panoramas need to be resized to 1024x512 pixels to enable the fast streaming of larger files.
Another alternative is to prepare the content for a 360-degree screen in which the user will change to the audience and interactivity with the content has to be programmed for the purposes of the wraparound screen.The first test of projecting 360degree interactive narratives on a 360-degree screen (Figure 9) took place during an experimental week at the University of Plymouth in February 2010 (Kwiatek, 2010).The interactivity for a 360-degree screen needs further researching, but the second trial will be performed during icci360 festival in September 2010 in Plymouth (Innovation for the Creative and Cultural Industries, 2010).The audience of up to 1000 people would interact either, by clicking a device button with the option of where to go or, they might move into the specific space within panoramic environment to make one decision together.

CONCLUSION
The intention of the researchers was to compile a comprehensive encyclopaedia of data in the form of narrational journeys experienced between decision points.This method may be preferential for young audiences as it allows them to gather a large amount of information once they are interacting with the application (Nofrontiere Design, 2004, p. 59).The sequence of these linear narratives, which consists of biographical data about Causley's life and information about his literary output, is not important because they all are separate linear narratives and do not have to be listened or experienced in a chronological order.Manovich's concept to treat a number of data as a database seems to suit the researchers' approach with a number of still and video panoramas.
This paper aims to present an experimental application where interactive storytelling and still and video panoramas were applied in one project.The user should not only improve the understanding of Causley's poetry, but also develop further interest in one of the topics, not always related to the poet.This educational project is a pioneer method of merging branching narratives with panoramic content.360-degree video is the basis for creation of the environment in which the user can discover a narrative.This project aims to be presented in three different forms (Blu-ray disk, Internet and 360-degree screen).Experiments continue and this leads to promoting not only the poet and his literary output, but also his town of Launceston.This method might be applied to other towns or cities in the world.

ACKNOWLEDGMENTS
The researchers would like to thank Charles Causley Society, Charles Causley Trust, Kent Stanton, Arthur Wills, John Hurst and Robert Tilling for their support and also three creative students from Plymouth -Paul Lloyd, Ben Phelps and Matthew Allen -for their work in the generation of music and illustrations for this project.

Figure 3 :
Figure 3: Charles Causley in 1979 -Photograph by Robert Tilling/ National Portrait Gallery, London

Figure 4 :
Figure 4: Ladybug2 -Spherical Digital Video Camera from Point Grey Research

Figure 6 :
Figure 6: A motorised scooter provided by More Than Mobility from Plymouth was adjusted to be used with Ladybug2 camera

Figure 7 :
Figure 7: The user can transfer items that appear in Causley's poem to the virtual rucksack

Figure 8 :
Figure 8: Application of graphics from Causley's poetrye.g .Mrs Busybody is situated in the bottom left corner; Mr Pennycomequick in the middle

Figure 9 :
Figure 9: 360-degree screen provided by IglooVision as a medium for location-based panoramic interactive narrative