3d Spatiotemporal Reconstruction of Places and Events for Digital Heritage

This paper discusses the capabilities of the Vist3D system which enables non-technical users, specifically users without knowledge of 3D modelling techniques, interface design, data manipulation and creation of multimedia artefacts to build historical 3D spatiotemporal visualisations from natural language narratives. The user (e.g. museum curator, people involved with cultural heritage) need only supply a narrative of the event and a specification of the artefacts, e.g. buildings, ships. Specifically this paper demonstrates the application of the Vist3D system in the development of a visualisation of Cape Town Harbour during the 1890s. It shows how experts (e.g. in history, cultural heritage, archaeology) can more easily create 3D visualisations without the need for an in depth understanding of complex software or data gathering technologies. The system allows for models of artefacts and visualisations to be created that could enhance static museum displays.


INTRODUCTION
Computing technologies, particularly graphics and multimedia, have potential to aid the understanding of history and heritage.3D graphics, digital media and interaction provide the means to illustrate and augment historical artefacts so that aspects of life and historical events that would otherwise be difficult to re-enact are possible 3D graphics programming employing 3D models provide an information-rich, interactive environment.Their use in CGI supported films, such as Avatar, demonstrate their ability to create a compelling audience experience (Ardito et al. 2010, Adabala et al. 2010, Caffey et al. 2010).More often than not, however, the amount of time and effort required for novice users to use these tools and applications to produce effective 3D models is considerable.3DVisa report (Bentkowska-Kafel 2007) suggests that amongst the academic community there is a requirement for 3D visualisations of heritage for research purposes but that the subject experts (lecturers and students) did not always have the necessary technical skills.
The creation of animated visualisations of real and imagined events has significant advantages in many fields.The fields include the study of historical events and the demonstration of the working of complex historical devices and operations.There is much interest in these areas as is evidenced by the popularity of social and economic history, military history and industrial heritage and archaeology.A significant contribution to the UK economy is made by the heritage industry.A fundamental need when exhibiting these areas in books and museums is trying to explain what is actually going on.Visualisations have a major role to play in meeting this need.Many museums and historical societies use reenactors and re-enactments to visualise historical events and situations, for example to recreate train and bus timetables using vintage trains and buses to provide a visualisation of such timetables, or to re-enact a battle.
The current problem with visualisations by recreation/restoration or by physical modelling is that they are expensive and time consuming to make and often only give a sample of the real thing.
Through the use of Cape Town Harbour as a case study this paper demonstrates that there are benefits for using the Vist3D system including: • creation of interactive 3D spatiotemporal visualisations from user generated natural language narratives • ease of use in that it allows building of 3D spatiotemporal simulations of events and locations by non-technical experts • applicability to domains (e.g.reconstructive archaeology, historical reconstruction (Caffey et al. 2010) where other approaches for digital heritage techniques or situations (e.g.geographical location, time scales, cost) are impractical or more complex • flexibility i.e. the Vist3D visualisation may be static or include interactions.

RELATED WORK
The investigation and exploration of cultural heritage creates some major challenges for the visualisation and organisation of data gathering.In recent years there has been an increasing interest and adoption of the use of technology amongst cultural heritage areas.This interest originated as a means for the digitisation and cataloguing of digital artefacts.However as the concepts of social computing and the building of online communities developed, cultural heritage groups such as museums and galleries have realised that this technology provides the potential to enhance their collections and make them more accessible (both onsite and offsite).One way of doing this is by providing immersive, interactive 3D visualisations.These visualisations can enhance the exhibits and displays and aid in the transfer of knowledge.
Approaches range from the modelling and viewing of artefacts (Ardito et al. 2010) to systems that attempt to combine all resources gathered into an interactive framework for visitors (Adabala et al. 2010, Caffey et al. 2010).The major difficulties with such systems are the complex IT framework and the reliance on subject experts to "repackage" findings to fit the technologies being used.
The digital representation of heritage has been gaining momentum.It is also a diverse area which includes digital archiving and providing a web presence, for example, for museums and interactive visualisations.Interactive visualisations have the benefit of not only allowing researchers to explore history and artefacts in new ways but allows a level of engagement that is expected in an audience in a more technological aware society.
Research suggests that the approach to digital representation of artefacts and heritage is varied.Mendes et al. (2010) suggest that the use of interactive 3D visualisation of artefacts provides benefits for researchers and visitors to the museum but that the technological aspects (need for high detailed models, storage implementation) is complex.Similarly other work done demonstrates the need for not only subject knowledge but specific visualisation and programming skills (Bryan 2009, Caffey 2010).Going beyond the consideration of individual artefacts history and heritage is also concerned with interpretation and understanding -artefacts, places and events investigated holistically.In the context of the digital representation of heritage some researchers have considered how to combine the wealth of resources available into a digital application, for example,.in the form of 2D multimedia applications (Ardito, 2010, Alisi et al. 2010) or 3D visualisation (Adabala 2010, Molony 2010).These try to draw together not only the physical evidence in digital form but try to link them with their historical narrative to provide a context.
The key issues with some of approaches above is that they are difficult to set up and maintain, not easy to correct or extend and requires not only subject experts but technical experts.For example, Adabla et al. (2010) propose an interactive framework for presenting heritage narratives which requires a formal planning restructuring and abstracting of resources into databases.Similarly Alisi et al. (2010) presents a 2D multimedia system with spatiotemporal elements but is also dependent on technical knowledge of touch screen frameworks for implementation.
The Vist3D system eliminates many of these issues as it only requires the user to supply narrative, textual descriptions and data that is within the realm of their subject expertise.For example in the Cape Town Harbour case study the user supplies information such as ship movements within the harbour, ship specifications and weather conditions as textual narrative or data.The subject experts would make use of primary sources such as shipping logs and harbour records to access this information.The Vist3D system then processes and utilises this information to generate the appropriate models and environment for the visualisation.

THE VIST3D SYSTEM OVERVIEW
Vist3D represents the further development of the TMap3D system (Presland et al. 2010).It generates interactive 3D spatiotemporal visualisations of places and events and currently allows these to be visualised in either Panda3D or VRML.The Vist3D visualisation uses several sources of information that can be generated by interested parties e.g.historians, hobbyists.The information includes the historical narratives, models (such as ships) and terrain.The system is designed in such a way that users require no knowledge of 3D modelling or 3D graphics.The user need only supply a textual narrative and input information into the Vist3D components such as ship builder to generate 3D models.The Vist3D system architecture is illustrated in Figure 1.The system comprises of three main sections; the Narrative Component, the Analyser and the Viewer.

The Narrative Component
The narrative component of the Vist3D system comprises of three main elements the Narrative Editor, the Narrative Parser and the Temporal Database.The Narrative Editor allows a user to supply a textual narrative or to select an existing narrative from the Temporal Database.
The Narrative Parser has been developed in PHP and makes use of regular expressions.This parses any narrative supplied by the user and populates the Temporal Database.The Narrative is written a sequence of sentences terminated by full stops (see Figure 2).Each of these sentences is parsed in turn.It identifies time stamped data, objects their associated actions, locations and environmental features.The Narrative Parser then uses SQL commands to store the parsed data in the Temporal Database.
"The schooner Compass Rose departs berth 17 at 11:30 am on its way to Rio de Janeiro.At 13.00 the Compass Rose turns to heading 080"

Figure 2: Cape Town Harbour Narrative Extract
A temporal data model has been designed to allow the storage of time-stamped facts.The data model captures the temporal data content from the narrative.The system is designed to be flexible and store any attributes specified in the narrative as it is impossible to predict what attributes the user may include in the narrative.

The Analyser
The Analyser creates the scenario file by extracting the time stamped data from the Temporal database along with other information such as model files and terrain files to create visualisation files (scene graph and keyframes) to be processed by the Viewer.The Analyser generates its visualisation files in the following way: • By the creation of models (ships, buildings, structures) and terrain to be used in the visualisation.This is currently done using Vist3D tools such as ship builder.Models are held in a model library.• The Vist3D Analyser processes the scenario file, terrain, model and other resource files to create a scene graph (the 3D scene).It also generates the keyframes and action information used to animate the scene over time and to provide environmental conditions and other effects.
The output can be previewed as a VRML file or run in Panda3D (using the viewer)

The Viewer
The Visualisation viewer displays the scenario in 3D -currently using Panda3D or VRML.Controls display the current scenario time and provide buttons to run the simulation in real time.Buttons may be added to speed up or slow down the simulation.A horizontal scroll bar -the time-sliderenables the user to move quickly to any chosen time.The arrows at the end of the time slider can be used to increment or decrement time by fixed amounts.Viewpoints are added to the individual objects as well as the overall scene.These allow the user to examine the visualisation from different perspectives, for example, from the Compass Rose or from the harbour.
The Vist3D Viewer has been designed for flexibility.The 3D scene is expressed in terms of a keyframes XML file and a scene graph XML file.This means that through the use of Vist3D objectifiers that the visualisation could be targeted to other formats such as 3DsMax, C++ or X3D.

IMPLEMENTATION OF CAPE TOWN HARBOUR
In order to demonstrate the capabilities of the Vist3D system a case study relating to the visualisation and reconstruction of Cape Town Harbour and the passage of ships in and out of it during the 1890s was created.This scenario was chosen to demonstrate how easy it is to generate the spatiotemporal aspects, such as the movement of the ships in the harbour and the evolution of the harbour during that time period.
The Vist3D system requires the user (subject expert) to provide two types of information in order to generate the visualisations.Firstly, the user must supply a narrative which is parsed and stored in the Temporal Database which is then used to generate a scenario input file.The following fragment from the scenario input file illustrates a fragment of the scenario used to generate the 3D visualisation.
"The schooner Compass Rose departs berth 17 at 11:30 am on its way to Rio de Janeiro." The second stage is for the user to supply data and information that will be used to generate objects that will be used in the visualisation such as the 3D models of ships and the harbour.This will come from their subject expertise and will be taken from appropriate primary and secondary sources.
The ships for the Cape Town Harbour case study were generated within the Vist3D software utilising the ship specifications.The ship specifications were input in the form of XML: The data for the specification files in this case were obtained from ship drawings on the Internet.An example of the type of data that the user must supply can be seen in the fragment of XML illustrated in Figure 3.It is important to note that the XML is a template and the user only has to enter details such as ship measurements, colour and the number of sails.<stern> <sternband id="0" colour="dark green" lines="10 8" thickness="3.0"/><sternband id="1" colour="dark green" lines="8 0" thickness="3.0"/><sternband id="2" colour="dark green" lines="0 7" thickness="3.0"/><sternband id="3" colour="dark red" lines="7 1" thickness="3.0"/><sternband id="4" colour="copper" lines="1 2" /> ... </stern>

Figure 3: Modelling the stern section of a ship in XML
The 3D terrain was also generated within the Vist3D software.As with the models the user needed only supply land or terrain data gathered from an appropriate source, e.g.digital elevation maps (DEM), shuttle radar typography mission data (SRTM) or data from one's own surveys.The Cape Town Harbour case study used SRTM data to generate the landscape surrounding the harbour and to generate Table Mountain.The process of making a terrain matrix was straight forward.Using software called 3DEM data was processed and converted to UTM (Universal Transverse Mercator Projection).This allowed Vist3D to accurately visualise and position objects in the 3D world later.The projection was then converted to a terrain matrix and a texture.The file names for these elements were typed in to the scenario visualisation to allow Vist3D to generate the landscape.
The screen shots in Figures 4 and 5 show rendered images generated by the Vist3D system using the VRML previewer.It is important to note that these only show a snap shot of an interactive 3D spatiotemporal visualisation that has been generated.

EVALUATION
The Cape Town Harbour case study demonstrates the ability of the Vist3D system to create realistic detailed models and environments including ships and harbour buildings with very limited technical knowhow.It shows how the system can be used as a powerful interactive 3D visualisation tool to support the realisation and reconstruction of events and places.This case study demonstrates the ease at which the 3D models can be generated via textual specifications.To physically model this scenario accurately for a heritage display would be time consuming and costly and would only be able to show a snapshot of the period.Any updates would involve remodelling.These models would also require a lot of space and require maintenance.In addition an in depth knowledge of a 3D modelling application and tools would be required in order to convert data such as terrain data, ship specifications and movement patterns to be used in the modelling application.
The system allows a level of interaction that is not possible with many passive resources such as diagrams, static museum displays and even websites.The Vist3D system can enable the visualisation of the story of an historical event in an interactive and visual way.For example it can easily be used to visualise spatiotemporal data showing users how places or other objects move, change or interact over time.The user can interact with the timeline and views to examine the events and evolution of the harbour from different perspectives.The system also allows for additional multimedia resources to enhance the visualisation such as sound The interface by default allows the user to enter English like narratives, allowing the user to generate 3D models and visualisations without any prior 3D modelling expertise.The Vist3D functionality allows users not only to view the historical scenario but also to analyse strategy and tactics.The Vist3D system allows the user to examine the historical events from different perspectives, e.g. from the perspective of different ships.
The Vist3D system provides an opportunity for users to create animations of historical events, to a considerable level of detail and with multiple selectable viewpoints with no prior knowledge of 3D modelling tools or techniques.
As discussed the Case Study demonstrates many advantages and benefits of the Vist3D system.However as the system is still at the proof on concept stage it also highlighted areas for improvement in terms of usability.Whilst it is true that the user only has to supply data regarding ship specifications to generate the models of ships, the system makes the user input this into XML files as the forms are not yet complete.The interface requires further work to enhance usability.

CONCLUSIONS AND FUTURE WORK
This paper has highlighted the importance of 3D visualisations in enhancing and supporting culture and heritage.This paper has demonstrated the capability of the Vist3D system to provide a tool that enables non-technical users to generate engaging spatiotemporal 3D visualisations of historical events and to reconstruct places and environments.
The Vist3D system proof of concept application discussed in this paper provides the user with the ability to create interactive, accurate and engaging 3D spatiotemporal visualisations.It is a powerful tool that can support the digitisation of culture and heritage and the realisation of history and is one that depends on very little technical knowledge or support.The visualisations that are generated by the system provides a quicker way of providing "live" simulations of the evolution of places and reenactment of events and may enhance or replace static displays, models and images which due to their nature only offer a snapshot of an event or period.
The team have identified several areas for future work.Firstly, there is the usability and interface design.This must be addressed to improve usability particularly as the system is intended for non-technical users such as curators, scholars, and historians.As can be seen some of the stages that a user must go through are simple but crude (e.g. the need to do external pre-processing using the 3DEM application, entering the textual data into XML files instead of forms) but these are aspects that will be refined once as software moves beyond proof of concept stage.A further consideration is the usability and design in terms of generating the narrative and the data files.
Secondly, the project team wish to enhance the realism of the 3D visualisation.This will include consideration of environmental conditions such as weather and sea, realistic movement of objects such as ships and trains.The system currently includes sound but can also be extended to include multimedia to provide a more information rich visualisation of the historical event.
Vist3D can generate accurate, interactive spatiotemporal simulations.It offers a tool to enable the reconstruction and representation of historical events and places in order to bring to life culture and heritage.This is an ideal tool for an area which has now recognised the importance of digitisation but has limited technical skills in the field of 3D modelling and graphics programming.

Figure 1 :
Figure 1: Overview of the Vist3D System

Figure 5 :
Figure 5: A ship leaving Cape Town Harbour