Experiencing Personalised Heritage Exhibitions Through Multimodal Mixed Reality Interfaces

This paper presents solutions for both museum exhibitions and mobile guides using multimodal mixed reality interfaces based on open standards that can be highly customisable and intuitive to attract and engage a broad spectrum of users. For indoor exhibitions, some of the most characteristic technologies will be presented to demonstrate how multi-modal presentations and interactions of three-dimensional heritage environments can provide enhanced support in practice. For mobile personalised applications like on-site visits, a highly customisable mobile framework illustrates how personalized visits to open-air heritage sites can be performed. The multimodal mixed reality interfaces take into account the diverse needs of visitors to heritage and mobile guide exhibitions allowing for multimedia representations of the same content but using diverse interfaces including a web, a map, a virtual reality and an augmented reality domain. Different case-studies demonstrate the capabilities of the multimodal mixed reality interfaces when applied to museum and open-air environments as well as how personalisation and customisation can be performed to meet a wide range of user needs.


INTRODUCTION
Museums and other heritage exhibitions try to attract the attention of visitors by presenting audio-visual information in a number of different ways.As a result, heritage exhibitions have evolved from passive viewing to interactive applications to use several different input devices.Although during the past few years technological advancements in computing have advanced rapidly the current generation of museum exhibitions and mobile guides offer simplistic applications with restricted capabilities and content.Occasionally, experimental prototypes have emerged from universities and research institutions but there is still need to provide enhanced support to cultural heritage exhibitions mainly because the resulted prototypes are too complex for the visitors and thus cannot meet their needs.The most characteristic example includes user selected mobile audio guides and interactive touch screen kiosks installed at predefined locations.However, advanced 3D graphics technologies have not actually applied in practice even if they exist for many years now.One of main reasons behind that is because there are no multimodal and user-friendly interfaces capable of supporting a big variation of visualization and interaction technologies.
Online communities have evolved together with the progression of fast communication technologies.One of the most important representatives is Web3D communities.It is worth mentioning here that several other approaches that are not using Web3D have been proposed [1], [2] but they are not so widely acceptable by museum curators and archaeologists.Research in multimodal heritage systems based on the use of Web3D technologies is growing [3].The great advantage of Web3D technologies is that it offers tools for creating virtual environments using technologies like VRML and X3D and with some customization with more advanced technologies, such as mixed reality.In 1994, Milgram and Kishino [4] tried to depict the relationship between VR and AR by introducing two new terms called mixed reality (MR), which has similarities with VR and AR, and Augmented Virtuality (AV).The basic theoretical principle of AR is to superimpose digital information directly into a user's sensory perception [5], rather than replacing it with a completely synthetic environment as VR systems do.However, both technologies process and display the same digital information and often make use of the same dedicated hardware.For example, both a VR and an AR system may be equipped with a HMD to visualize the same 3D computer generated model.
On the other hand, VR systems can completely immerse a user inside a synthetic environment by blocking the signals of the real world.In a typical VR system the user's natural sensory information is completely replaced with digital information.Most AR systems use more complex software approaches compared to VR systems and there are some major differences that make the two technologies distinct.An AR system uses the real world instead of trying to replace it.On the other hand, in a VR system the whole environment is synthetic.The user is completely immersed within a virtual world trying to mimic reality.A virtual reality simulated world does not always obey all laws of nature.Furthermore, the most common problems of VR systems are of emotional and psychological nature including motion sickness, nausea, and other symptoms, which are created by the high degree of immersiveness of the users.Although AR systems are influenced by the same factors the amount of influence is much less than in VR, since only a portion of the environment is virtual.demonstrate how multi-modal presentations and interactions of three-dimensional heritage environments can provide enhanced support in practice.For mobile personalized applications like on-site visits, a highly customizable mobile framework illustrates how personalized visits to open-air heritage sites can be performed.The multimodal mixed reality interfaces take into account the diverse needs of visitors to heritage and mobile guide exhibitions allowing for multimedia representations of the same content but using diverse interfaces including a web, a map, a virtual reality and an augmented reality domain.Different case-studies demonstrate the capabilities of the multimodal mixed reality interfaces when applied to museum and open-air environments as well as how personalization and customization can be performed to meet a wide range of user needs.

CULTURAL HERITAGE EXHIBITIONS 2.1 Museum exhibitions
There are a number of researches that have explored the potential of personalization in virtual museums.The M-PIRO project used the concept of personalized information objects, which are entities capable of responding to requests for information by taking into account the user's previous knowledge and interests.The technology used allows textual and spoken exhibits' descriptions to be generated automatically from existing free-text descriptions and from an underlying language-neutral database [6].The MUSEpad project created user profiles via focus groups and visitor studies techniques addressing issues of mobility, dexterity, visual and auditory acuity, cognitive abilities and technology acceptance.In addition, it developed a mobile computing tool that enabled visitors with disabilities to optimize their learning and entertainment museum experience through the technology of WorldBoard that utilizes wireless connectivity, as well as positioning and Internet technologies and customize the system to their needs and interests [7].The Museum Wearable, developed for the MIT gathered information about the museum visitor's preferences from their physical path and length of stops along the museum exhibition.It used infrared location sensors distributed in the museum space, and statistical mathematical modeling, in order to create a user model and to deliver a personalized audiovisual narration to the visitor [8].
The Carrara Marble Museum's website provided personalised access to the collection.Its visitors were grouped into three categories: tourists, art students and experts.Each user type was defined by the level of detail and the mode of access to the information provided.Concerning people that do not associate themselves with any of the three types, it is offered a userdefined/customisable option that provides the opportunity to the users to create their own profile by defining the type of access to the virtual collection, the type of information they prefer and the layout they choose [9].The Telebuddy, an avatar equipped with camera, microphone and speakers in a museum's environment was connected with an online group of visitors [10].In another case, an interactive conversational character established a customized person-to-person dialogue while guiding the user through an online museum exhibition, by performing engaging storytelling and by customizing the tour information to user's interests [11].
The PEACH (Personal Experience with Active Cultural Heritage) project [12] explored the personalized request of information about museum exhibits by the visitors providing a variety of information sources and media types (museum servers, online remote servers, etc.), and presenting a variety of clients, such as tabloid computers and PDAs [13].The CHIP (Cultural Heritage Information Personalization) project provided personalized access to cultural heritage content using semantic-aware adaptation strategies and semantic based reasoning user model elicitation.Data about artifacts from the Rijksmuseum Amsterdam were presented on various devices (e.g.web application, PDA) and information about museum visitors in terms of their interests and preferences were collected and stored in user profiles.Then, the information was used to recommend routes through the museum and to guide the users towards artifacts related to their interests and preferences [14].
The problem of a cost effective way to customize and distribute content for virtual museum websites, like the 24 Hour Museum is the UK's National Virtual Museum [15], a cultural portal that promotes UK museums, galleries, archives and cultural tourism sites, was solved by adopting the RSS (Rich Site Summary) technology that facilitates automated content syndication and information personalization [16].Another research examined the use of mobile technology that creates personalized learning trails during visits to museums, botanic gardens and cultural heritage sites.Children and adults visited botanic gardens and art museums and studied horticulture and artworks.Visitors can capture, edit and share information in virtual galleries, in the form of audio, photos and text, according to their preferences and needs [17].
The VIMU project developed by four institutions, funded by EU and concerns the Danish-German history.It traced the various virtual museum target groups and categorized them into six target groups, according to their interests and motives that activate a visit to the virtual museum website.These categories were the general users, students, teachers, researchers, tourists, and kids.Firstly, the content of the website was displayed taking into account the various profiles of the visitors.Then, the visualisation of the content can be customized to various user needs, since the size of the fonts and the background colour can be changed and animations can be replaced by static elements.There were various options for navigating the site, in order to enable the users to choose the most convenient option for them and several information layers with various depths, so as the users can select how much they want to learn [18].

Mobile exhibitions
In the past a number of commercial and research mobile guides have been developed targeting a diverse range of users but none of them can fully satisfy one the most important information needs, which is to access information from everywhere, anywhere and at anytime using the most appropriate presentation interface.Nevertheless a representative list of the most similar systems is briefly presented below.One of the earliest mobile guides is the HyperAudio project [19], a context-sensitive adaptive and mobile guide for museum environments taking into account the visitor's needs and the exhibit outline.Based on the system, museum visitors can explore a certain subject, have instructions for finding the objects of interest, receive descriptions with references and have instructions on alternative routes, thereby giving a personalised guide for visiting the physical space.Another early system, called the Cyberguide, was developed by [20] and aimed in both indoor and outdoor environments.The system provided simplistic schematic black and white maps and information services about predefined indoor locations.Localisation in indoor environments relied on infrared beacons, whereas for outdoors a GPS was used.A major disadvantage of the Cyberguide system is that no client-server architecture was used and all content (maps and other information) were stored on the mobile device thus can work only on a limited area.
The GUIDE project [21] developed to provide hand-held computerbased tourist guides for the city of Lancaster.Based on client-server architectures and wireless communication protocols the mobile guide approximates its location and then provides guidance and information services through a browser-based interface.This could include walking tours or directions on how to get from one location to another.The GUIDE project differs to other computer-based tourist projects because was one the first systems to obtain all information wirelessly allowing support for interactive services and access to the full resources of the Internet.However, no orientation information was provided which is essential for navigation and wayfinding mobile systems.
The REAL project [22] presents a hybrid navigation system that adapts the presentation of route directions to different output devices and modalities.The resource-adaptive navigation system consists of three components: an information booth, an indoor system that uses PDAs as presentation devices and an outdoor system that uses position and orientation information on a laptop combined with a head-mounted display.The system assists users to find locations by generating route descriptions and can adapt the graphical representations according to various cognitive and other navigation issues.The HIPS system [23] uses a radio back channel for downloading multimedia content on laptop computers.The main aspect of the system is that takes into account the absolute position of the user and the relative distances to objects in the exhibition.Furthermore, the MOBIS system [24] is a situational guide based on a PDA that provides a visitor with information about the museum exhibits.The PDA localises itself using IR beacons distributed in the environment and uses this position as a pointer to a specific content that is stored in a database.
In terms of VR and AR mobile exhibitions, the LAMP3D system [25] was designed for the location-aware presentation of VR content on mobile devices, applied in tourist mobile guides.Although the system provides tourists with a 3D visualization of the environment they are exploring and synchronized with the physical world through the use of GPS data, there is no orientation information available.An example of a multi-modal mobile interaction platform which supports both indoor and outdoor pedestrian navigation by combining 3D graphics with synthesised speed generation was developed by [26].Indoor tracking was achieved through infrared beacon communication while outdoor tracking via GPS.However, the system does not use geo-referenced or accurate virtual representations of the real environment, neither report on any evaluation studies.
MARS [27] is a characteristic example of a wireless mobile interface system for indoor and outdoor applications because it was one of the first outdoor AR systems.MARS was developed to aid navigation or to deliver location-based information to tourists in a city.The user is standing in a university campus wearing a prototype wearable system.The user is holding a handheld computer and stylus, in order to interact while a tracked see-through head worn display is driven by a computer contained in the backpack.However, MARS, like most mobile AR guides, can only operate within a restricted area.Archeoguide [28] exploited the use of mobile technologies, AR, 3D visualization and archiving to present visitors with on-site and online tours of the physical site according to their profile and behaviour during the tour.The system used differential GPS and orientation tracking on wearable computers and provides personalized electronic guides to assist users navigating in the outdoor heritage environments.The major disadvantage of Archeoguide is that the experimental set-up was too heavy for the users to carry around addressing safety issues.Finally, Thum et al. [29] presented a prototype regarding the adaptation of the user interface of an existing mobile AR system for cultural heritage to multimodal interaction.Speech and capturing of hand movements were achieved using an inertial tracker but evaluation is under way.

MULTIMODAL KIOSK SOLUTIONS
To make the heritage information accessible to users all over the world, it needs to be distributed in an Internet-compliant way [30].Clearly, one of the most important goals of modern archaeological applications is to present museum artifacts in an attractive manner.In this way, museum visitors and especially small children could become more interested in cultural heritage.In terms of presentation, multimodal kiosk solutions may offer: web domains in the traditional way, web-3D, web-VR and web-AR domains.This allows users to select relevant information and combine it into the same presentation environment.The web client is a customized web-browser interface, which communicates with a database and can then present the requested information in one of the above four visualization domains.In other words, the retrieved digital content can be rendered with the aid of web technologies initially on the web browser and then on the web3D, VR and AR environments [3].For the visualization, touch-screens, plasma screens, projection based systems, head mounted displays (HMDs) and stereo glasses may be used.HMDs can provide an immersive video see-through AR visualization as well as immersive VR experiences.However, HMD-based systems are still expensive and introduce complications such as nausea and motion sickness.
For the pure web domain, using web technologies (like XSL and XML technologies) it is easy to create fast and accurately new presentations that could serve different user needs.To achieve effective customization, when the client is initiated, it must be handled by internal data structures to prepare the data for visualization.As a result, all multimedia objects associated with the visualization are first downloaded into the host computer's cache (kiosk environment) and then stored into a predefined directory.The visualization client retrieves all media objects and presents them to the kiosk visitors.For the web-AR solution, again different presentation scenarios can be achieved.Augmentation may vary from simple 3D objects to complete galleries and exhibitions.Users can manipulate freely the superimposed information to receive a different perception of the visualization.The customization of the AR also relies in either computer vision, or hardware based tracking.Computer vision is much cheaper and provides very accurate results in controlled environments such as museums and galleries and as a result it is usually the preferred medium.
The VR domain may contain different feel as well: one using pure VR and another using web-based VR.The pure VR solution can operate as a stand-alone component and can work within a museum environment in a dedicated PowerStation.On the other hand, the web-based solution takes advantage of the client-server functionality and can access heritage information not only from a museum environment but from user's homes.The differences between the pure VR solution and the web-based solution are in realism, accessibility and personalization.The latter can take advantage of the latest communication technologies (i.e.broadband, WiFi, etc) and provide a web-based VR presentation that can be accessed from anywhere and at anytime.In addition it is easy to customize and personalize the content of the VR-based presentation since everything is controlled via server technologies.However, the underlying technology used (i.e.Web3D) for rendering lacks of advanced rendering capabilities and thus realism.On the contrary, the pure VR solution offers a much better representation of the virtual artifacts, because it uses computer graphics algorithms for the generation of soft and hard shadows, reflections and advanced lighting, as well as texturing techniques.It is worth-mentioning, that the realism of the visualization highly depends on the 3D modelling procedure.The 3D models used in this scenario are the highest resolution models the modelling team in the laboratory could achieve.
Moreover, interacting with the virtual exhibitions is one of the most important issues in kiosk presentations since it has to be performed in a natural way so that also inexperienced visitors can get used quickly to the kiosk environment.Different interface techniques may be implemented [31] to make exploration of the museum or exhibition environment, easier and much more compelling by combining standard (i.e.keyboard, mouse, joystick), advanced (i.e.SpaceMouse, InertiaCube, gloves), custom (i.e.custom sensors) and natural techniques (i.e. computer vision techniques, TouchScreen).The main advantage of a multimodal kiosk interaction interface is that it allows users to select the type of interaction (or the combination of techniques) that is more appropriate for their needs.Although, the interactions are not new the novelty comes in the way they are used by the participants.The natural manipulation of virtual information is one of the most useful operations of the AR interface since it can attract users that do not have any experience with computer systems.However, visitors can combine two or more types and experience a novel form of interaction with great flexibility.For example, the most significant combination of humancomputer interactions is the use of intuitive methods, like the physical manipulation of marker cards (based on computer vision techniques) with sophisticated, such as the SpaceMouse.Visitors can hold in one hand a marker card with a virtual object superimposed and on the other hand the SpaceMouse to perform graphics operations like translating the virtual artefact.

OUTDOOR SOLUTIONS
There are diverse candidate interfaces for the presentation of spatially referenced information on mobile devices and cultural heritage is an excellent application domain [32].In particular, there is the opportunity of presenting spatially referenced information relative to the device user's current location.Three alternative interfaces for the presenting information in situ include: 2D map; VR map and AR map.The digital map interface is the most obvious and widely used approach.In the pre-digital age, signage at cultural heritage sites often showed maps with 'you are here' arrows indicating the location of the sign, and by association, the person viewing it.In addition to the sign's location and contextual information, the location of specific features of interest was often shown on the map.In order to avoid map clutter, sign designers would often have to work with the site management to decide upon the most pertinent information to display, at the expense of features considered less important.Frequently, these maps displayed a wide range of information from basic utilities (e.g.toilets) to cultural information, such as the location of the most popular site attractions.
Mobile devices present opportunities and constraints when attempting to extend this traditional map-based interface in a digital environment [32].The major constraint is space: a sign may have a surface area of one square meter or more, however displays on mobile devices are typically less than six by ten cm.This suggests the need to reduce the volume of information still further than for the case of the physical sign.An opportunity in mobile computing is the ability to filter information according to the personal preferences of the device user, to just display the particular class of information that they are interested in.Visitors to a heritage site may choose to display just information about how industrial uses of the land have affected the area though history, and be presented with a map of all locations in the site associated with industrial development.This is a highly personalized map presenting one type of information at the expense of everything else, essentially placing the device user in the role of map designer, deciding the information that can be included or excluded.An additional opportunity for mobile computing is the opportunity to make the map interface location-aware.Whereas a physical sign presents a single 'you are here' arrow, on a mobile device, such a map can update to track the users position to any point on the site, allowing a personalized 'you are here' to be generated on demand.
An alternative to the map interface on the mobile device is to use a VR interface.Whereas digital maps adopt an allocentric (bird's eye) perspective on a two-dimensional abstraction, virtual reality adopts an egocentric perspective on a 3D abstraction [32].This perspective aims to mirror a visitor's perspective of a cultural heritage site.The scene itself can be a photorealistic reconstruction of the current sites, or may represent an alternative reality, such as the same scene from a previous time in history [33].This virtual space can be used to present further relevant information to visitors.Labels (such as building names) may be attached to objects in the scene, directions may be presented for a walking tour of a site, or anchor points may allow users to click through to reveal further background information about a feature, for example textual, graphical, audio or video.In the AR the real information can be provided from a live stream from the device camera.To overlay virtual information in the correct place over the video stream, the location and orientation of the mobile device must be known: this information can be provided from a positioning sensor and digital compass.This sensor-based approach to registration is not as precise as using visual approaches, such as fiducial markers and natural feature detection [34], but is sufficiently accurate for applying labels to objects in the real world scene.The information displayed can be similar to that described above for virtual reality interfaces: labels, directions and anchor points for further information.
Tracking of the user's position and orientation is one the greatest challenges of mobile tourist guides.This section provides detail on deign and implementation issues.Real-time and accurate tracking of position is one of the hottest topics of this decade since it provides a basis for many commercial applications.The most characteristic technology is GPS (Global Positioning System) which makes use of minimum of three satellites to detect position in real-time performance.However, the accuracy of standard GPS devices varies in the order of a few meters.This might not be of significant importance when navigating into rural guides areas but for heritage guides it is much more important.For example, when on-site navigation is performed then the GPS error might provide positional information inside the monuments or other structures.For the map guide this might be a small issue, but for the VR and AR guides the error might position the user inside the monuments.To tackle the problem a simple snapping algorithm that calculates the previous position and checks with the footprints provides an effective solution.Alternatively, computer vision techniques may be used to calculate the position using features from the real environment but although they can provide good results in controlled areas, they are not robust enough for applications that must work everywhere and anytime.Moreover, orientation may be obtained in a number of ways.The most popular approaches include accelerometers, gyroscopes, inertia sensors and electronic compasses but none of them comes as an out of the box solution for mobile guides.Similarly to the kiosk framework, the last type of interaction includes the use of a graphical user interface.However, the difference here is that this is aimed only for experienced with computer technology users.

KIOSK CASE STUDIES
To evaluate the personalization capabilities of the mixed reality kiosk multi-modal environment a case study are presented for Victoria and Albert Museum [35] and the Sussex Archaeological Society [36].All implemented GUIs and in particular the AR clients allow for a high customization in run-time consisting of four components: a menu, a toolbar, a status bar and a number of dialog boxes.Visitors can easily customize the appearance of the visualization such as shadows, fog, lighting, material, texture, color, transparency and shading.This allows participants to have the same access to the virtual information as they have had using standard interaction techniques.However, the greatest advantage of all GUIs framework is that it allows participants to perform complex operations very accurately [37].Specifically, sometimes it is of crucial importance to transform a virtual object in a specific location in the real environment.Using other methods, can take a great amount of time and effort (depending on the experience of the user) to achieve and it will definitely not be very accurate.However, the GUI interaction techniques offer the solution to the problem using double point precision manipulations which is impossible to achieve with the other interaction methods.
The most important representatives of the Sussex Past include two small museums: Anne of Cleves House [38] and Fishbourne Roman Palace [39].The Anne of Cleves house was built in the 16th century and formed part of Anne's divorce settlement from Henry VIII in 1541.It contains a wide range of collections such as Sussex pottery.Fishbourne Roman Palace was accidentally discovered in the 1960s and it represented a military base at the time of the Roman invasion in AD 43 to a sumptuous Palace by the end of the first century.The virtual museum visitor can select to navigate through the collections of each gallery.Figure 1 shows the customized form of web gallery visualization that presents thumbnail information about a virtual museum presentation.Each thumbnail represents a link to a different page, which contains extra information about the virtual museum artefact such as metadata, images and a 3D representation of the artefact.Then the user can get more information about a particular artefact contained in the gallery by simply clicking on the thumbnail.This will transfer the visualization from the classical web view to a Web3D presentation (which has embedded 3D information).The embedded Web3D browser (which is hidden on purpose from the user) allows for real-time interaction with the virtual objects using just the mouse.By changing content and presentation style using different XSL stylesheets the visualization can be altered in a number of different ways.Furthermore, one of the most impressive features of the system is that it allows the visualization of the whole collection of Anne of Cleves and Fishbourne Roman Palace museum galleries.Users can view the galleries over the web and then click on the 'AR gallery button' for inserting one or multiple museum artefacts into appropriate marker cards.In this way, they can customize the AR gallery presentation populating it with different artefacts.Figure 2 illustrates how a collection of artefacts from a gallery belonging to Anne of Cleves can be personalized in AR environments.

Figure 2. AR personalisation of virtual exhibitions [31]
When the user is happy with the visualization it can change back to the web-client and select a different museum gallery.Moreover, our configuration allows users to change a number of the rendering properties in real-time to examine the virtual objects in an efficient way.These include zooming, panning, shading, lighting and other computer graphics operations [3].Potential museum visitors can experience an AR exhibition of a museum's collection in real-time using the proposed cost effective visualization system.Similarly, to the MagicBook [41], users can visualise a number of objects using a self-made ring binder book containing page markers called 'Heritage Learning Book'.This interface consists of a ring binder book equipped with markers and operates as a physical database (Figure 3).

Figure 3. Interaction techniques in Heritage Book [31]
The ring binder book looks like any normal ring binder but it is carefully designed to be easily extensible.The current configuration contains fifteen marker pages containing the letters of the English alphabet including A, B, C, D, E, F, G, J, K, L, P, T, V, W and Y that were easier recognizable by the pattern recognition libraries used ( [41].The 3D objects can be assigned into the markers from the web presentation using a hyperlink (AR button) as an action button.Figure 4 illustrates in four different scenarios, how a museum visitor can make use of the 'Heritage Learning Book' in conjunction with the SpaceMouse to visualize and interact with the virtual artefacts.On the top left image, the user is only visualizing the virtual artefact (marker B) while on the top right image, the user translates the artefact using the device.On the bottom left image, the user interacts (rotates) with another artefact (marker A) and on the bottom right image the user visualizes another artefact (belonging on marker E).
Moreover, Victoria & Albert Museum is one of the biggest museums of London and faces one the classical problem big museum face, which is that it does not have the space and resources for all its exhibits.The customization that was performed was similar to the Sussex Past's museums but more emphasis was given to the realistic creation of Web3D gallery exhibitions.An example screenshot of a V&A gallery in Web3D is shown in Figure 4.The main advantage of this type of VR interactive presentation is that it allows users to freely navigate inside the gallery.Furthermore, some of the exhibitions of V&A are presented in the same way as in the previous section with the only difference that they use different stylesheets.Thus, the 'feel & look' of the exhibition changes according to the targeted audience.For example font size and image size can be easily customized for access from people with vision disabilities.Figure 5, (a), presents a list of some of V&A's virtual museum exhibitions, and Figure 5, (b) shows the list of artifacts existed in the so called 'vase exhibition'.Similarly to the customization explained before, the artifacts can be selected and superimposed in a table-top kiosk environment.To produce an accessible virtual museum it is important to involve potential users in the creation process via continuous evaluations.Information about their preferences can be gathered through discussions and feedback during the development stages.It is recommended to have ongoing consultation and evaluation to ensure the creation of a user-friendly, usable and entertaining system according the user requirements.Thus a window into how the virtual visitors use and interact with the virtual artefacts will be provided.In order to test the usability of virtual museum systems, they were evaluated under real circumstances, not only by a demonstration of their capacities, but also through the contribution of real end-users, so as it will be the possibility to make the necessary system improvements.The most applied evaluation methodologies are the expert-based and the empirical (user-based) approach.The tasks supported by the system interface are presented to an interdisciplinary group of experts in the case of the expert evaluation, or to a group of simple end-users, or even to a group of experts and simple end-users.Expert evaluation is a relatively lowcost and efficient formative evaluation method applied even on system prototypes or design specifications up to the almost ready-toship systems.To elicit users' and information requirements of the ARCO system with the aid of heuristic evaluation guidelines: learnability, efficiency, memorability, errors and satisfaction developed by [42], according to which a system must provide feedback and visibility of the system status employing simple language with clearly marked exits, consistency of user interface elements, minimised user's memory load, aesthetic and minimalist design, ability to deal with errors and availability of help and appropriate documentation, three main system aspects are assessed including domain suitability; technical usability and user acceptability.Domain suitability examines the appropriateness of the content of information and display presentations.Technical usability refers to the perceptual and physical aspects of the humancomputer interface and examines whether it is user-friendly or not.
Then, the following four steps were undertaken including: goal setting (users start with a plan of the tasks to be accomplished), exploration (users explore the interface and discover useful actions), selection (users select the most appropriate actions for accomplishing their task) and assessment (users interpret the system's responses and assess its progression).The results of the evaluation, in preliminary form already presented in [43], proved to be the channels through which two homogeneous target groups of users, the specialists and the end-users communicated their opinions to the system creators.A combinatory usability evaluation, in which participated museum curators, as domain specialists and end-users representing the museum visitors, has been undertaken to investigate the educational and entertaining experience offered by the ARCO system.The questionnaire was completed by the two groups and the collected data was of both quantitative and qualitative form.In the following stage a statistical elaboration of the quantitative data was performed and showed that the whole sample of the experts and the users can be considered as homogenous as regards their estimations to this survey, because there is no difference in the evaluators' opinions due to the fact that they belong to different groups.
Additionally, qualitatively analysis grouped the evaluators' comments into three main categories of positive comments, usability flaw characteristics and remarks/suggestions [44].For the usability evaluation of the system two approaches have been employed: a questionnaire-based survey and a Cognitive Walkthrough session by museum curators as domain experts and usability experts.Cognitive walk-through methods [42] involves the 'walk-through' of a number of tasks, exploring the systems' characteristics, locating and identifying potential problems and their causes.The result of the research showed a fair performance as regards the time constraints and the financial limitations and uncovered the usability problems of the interface and compared the assessments of the two different evaluators' groups concluding that in complex interfaces double experts (usability and domain experts) are inevitable for reliable and valid results [44].Evaluation methods, such as heuristic evaluation and cognitive walkthroughs were employed in the ARCO evaluation, in order to assess various components and interfaces of the system.The results of a research were related with the exploration of how a virtual museum system is coping with the demands of museum curators, the examination of the virtual museum visitor's needs, and the creation of a set of criteria and guidelines for defining effective evaluation of such systems [37].

MOBILE CASE STUDY
The mobile framework uses the capabilities of the LOCUS system [45], which allows users to switch rendering modes between the traditional digital map guide, a VR guide and an AR guide.This allows mobile visitors to select the most appropriate presentation type according to their needs.In addition, the system provides both position and orientation tracking in mobile devices operating anywhere in the word.As the user navigates inside the urban environment, the position and orientation is consciously computed from external hardware devices such as GPS and digital compass and the camera pose is updated respectively.Then depending on the user needs, a map and/or a virtual reality interface presents two different navigation options.The mobile guide uses client-server technologies and it specifically designed for mobile devices such as PDAs and smartphones.The main objective of the multimodal system is to provide location-based services to mobile users delivered through a web-browser interface (i.e.Pocket Internet Explorer).In terms of functionality, it offers software libraries for integrating positional and orientation information, via Bluetooth, to provide navigational information about the surrounding environment as well as 'mobile search' options.
Initially the user requirements for navigation and wayfinding were collected based on an expert user evaluation performed at City University.Six expert users from different backgrounds including GIS, geo-visualization, VR/AR/MR, information retrieval, humancomputer interaction and psychology were tested.Each test took approximately 30 minutes and users were asked to fill in two questionnaires: one about general issues about human navigation and another about testing four hypotheses for virtual navigation.To achieve this, two well known evaluation techniques were applied: think aloud and cognitive walkthrough.The resulted feedback gave us a very important insight in order to develop the mobile guide for City University.First of all, the speed of the navigator affects the navigation process and smooth and slow speed is preferred.All users thought that the field-of-view is very important to urban navigation.Having a view of the destination and the route to, is clearly advantage.Also, it is of crucial importance to see visual cues around you when navigating and the eye-level of view is preferred (in contrast to the bird's eye perspective).However, if the navigation guide allow the navigator to zoom in and out the environment bird's eye view could be useful in some scenarios.The position of the user is crucial for mobile navigation, as navigation involves knowing your position along a route or destination.In addition, we found that although the position of navigator is important (when making a decision) and changeable, orientation is essential for navigation and it should be considered as one of the fundamental aspects in modern navigation systems.
Furthermore, participants mentioned that they mostly use paper maps for navigation and an A-Z map is commonly used when there is a street address enquiry.However, reading paper maps is difficult and the use of digital maps has become very common.It is worth mentioning that all of the participants participated in the study make use digital maps and some of them use them in PDAs as well.As far as the hypothesis testing, expert users reported that the use of textures is an appropriate way for visualization in mobile environments.The use of textures in landmarks is vital but the use of 'wrong' textures can be also used in the rest of the 3D map.High realism is also preferred as well as the use of street geometry such as ground, benches, trees, etc.Based on these requirements, a multimodal mobile guide for City University has been designed, that consists of the three domains previously presented (digital map, VR and AR).The application uses position and orientation information at the same time to provide a map view, a VR view or an AR view.
The GPS is embedded on the mobile device while the digital compass is a separate component hidden inside a blue cylinder communicating through Bluetooth technologies.An example of how a pedestrian could use the prototype system in practice is shown in Figure 6.

Figure 6. Tourist guide using GPS and digital compass
Using the City University mobile guide, pedestrians can navigate intuitively within the real environment using both position and orientation information on a mobile virtual environment.Additional functionality such as dynamic switching of camera viewpoint from the pedestrian view to a birds-eye view can be accessed from the menu buttons.Another important aspect of the guide is that the digital compass can be also used as a virtual pointer to provide useful information about the surroundings such as 'what is the name of the building?' or 'how far it is located from me?' etc.Currently, we can present this information in textual format but in the future audio descriptions will be added to enhance the experience.This is of particular importance since it allows users to select the information that they want to be presented on the device.Furthermore, visitors can perform advanced searches (i.e.where is the exit?) that take into account the spatial location and personal interest as well as act as generators of content, adding placemarks (known also as spatial bookmarks) that can be shared with other visitors using any of the above visualization domains.

CONCLUSIONS
This work has presented how personalization of mixed reality multimodal interfaces can be used for targeting the needs of visitors for both kiosk exhibitions and mobile guides.We showed the importance of a unifying framework for heritage exhibitions especially if it can be highly customizable, user-friendly and intuitive to use to engage a broad spectrum of users.The framework proposed, is focused on open-standards for easy adaptation of content and presentation across different media and hardware including desktop and portable devices and takes into account of the diverse needs of visitors to exhibitions.The multi-modal functionality, allows for multimedia presentation and interaction with different modes depending on the user needs.For indoor exhibitions, like museum kiosks, a combination of Web3D, VR and AR can provide a very powerful interface that can be used successfully within any museum environment.Through the case studies, we showed that personalization of the presentation provides the visitors with a variety of options and offers accessibility to a wide range of people and permits them to choose according to their preferences and desires.
For outdoor exhibitions, such as mobile guides, the presentation of map, VR map and AR map can provide assistance to a wide range of users, even those who have problems reading paper maps.We showed that the use of position and orientation information is essential for mobile presentations.Visitors can perform advanced searches taking into account the spatial location and personal interest using any of the presentation domains.Based on this, three case studies were presented illustrating the power of customization: two for kiosk environments and one for mobile guides.For the kiosk environment we illustrated several visualization scenarios ranging from the well-known web-presentation but with 3D content, virtual galleries and several AR examples.The first case-study, illustrated a customization for Sussex Past, while the second for Victoria & Albert museum.For the mobile guides we illustrated three visualization domains ranging from the standard digital map, to a virtual 3D map and finally an augmented reality domain.We illustrated one case study, to provide navigational support but also offer real time landmark information based on the concept of the virtual pointer.Evaluation results for the above case studies have shown that both kiosk exhibitions and mobile guides have the power to provide a customization and personalization which has the potential to be very appealing to the public.

Figure 1 .
Figure 1.Virtual museum gallery exhibition in web client [40]