Georeferencing in the Field Using Constellations of Similar You-Are-Here Maps

In this paper we investigate a new interaction technique that enables users to capture You-Are-Here maps using their smartphone, and then manipulate the captured image in such a way that it can be used for navigation purposes. This technique utilises groups of similar You-Are-Here maps, which we call map constellations. Results from our field study, in which we tested a working prototype of our interaction technique, are presented. The results show an insight into users’ views towards the interaction, and the techniques they employed to identically frame two You-Are-Here maps using a smartphone camera.


INTRODUCTION
In recent years, mobile map applications have become commonplace on modern smartphones, with most modern mobile operating systems having mobile map applications pre-installed.Mobile map applications tend to obtain their map information from a centrally managed map database, thereby allowing the map data to be accurate, temporal, and detailed.
Our research investigates how GPS enabled smartphones can capture a You-Are-Here (YAH) map and georeference (Hill 2009) the digital map image so that it can be used for navigation purposes, thus enabling functionality such as location and orientation estimation.In this context georeferencing can be thought of as the mapping of any pixel on a digital YAH map image to a geographic coordinate (latitude, longitude).Capturing a YAH map and using it for navigating purposes allows the user to utilise a stylised YAH map (e.g. Figure 1) that may contain information (such as walking routes or paths) that may not exist on traditional mobile map applications.Further, the YAH map can be captured, georeferenced, and used for navigation, without an internet connection.
This study builds on work into georeferencing by Cheverst et al. (2008), Sch öning et al. (2009), and Molineux and Cheverst (2010a,b) by developing a new georeferencing technique called the map constellation interaction.This georeferencing interaction utilises collections of YAH maps that are situated in a single environment, all representing the same area.The main contributions of this paper are: 1) An initial investigation into the usability and accuracy of the map constellation interaction.2) An insight into techniques users employ when framing YAH maps in a camera viewfinder.

Traditional You-Are-Here Maps
The placement and orientation of YAH maps are key factors in their usability (Levine et al. 1984).In order to use the YAH map, the map reader must first relate the map to the terrain.Levine (1982)  structure matching and the orientation principle.Structure matching is the process of matching real world structures to their representations on the map (such as the lake in Figure 1), whereas the orientation principle uses the orientation of the map as a tool to establish the map's orientation.Using the orientation principle when the physical YAH map is not correctly aligned can lead to wayfinding errors, which is often referred to as the misaligned map problem (Levine 1982;Aubrey et al. 1994).

Mobile Maps
Below, we outline research in two areas of how mobile maps can be adapted, which are: adapting the content e.g.occluding parts of the map to increase readability, and altering the maps form e.g.rotating or deforming the map.
Adapting the content on mobile maps enables the mobile map to be individually tailored to a user.For example, Dunlop et al. (2007) investigated a mobile application that displayed personalised maps for skiers, that was based on a number of variables including weather conditions and the skier's ability.Schmid et al. (2010) investigated altering the map's scale or level of detail based on position and trajectory.For example, the system could occlude roads or generalise terrain to improve readability.Seager and Fraser (2007) explored users' preferences towards automatic rotation of mobile maps to enable the map to be correctly aligned with the terrain.They found users tended to preferred manually rotating the device.

MAP CONSTELLATION INTERACTION
To further illustrate the map constellation interaction discussed in the introduction, consider the following scenario: Michelle is visiting London for the day.On leaving the train station she spots a detailed YAH map of the city on a noticeboard at the station exit.She uses her GPS enabled smartphone and a georeferencing application to take a picture of the map (step 1).Firstly, she marks her current location on the map (using the digital map image's 'You-Are-Here' marker) (step 2); the application then captures the current GPS location.She then continues from the station towards the city centre.A short while later she finds a second YAH map, identical to the first except the YAH dot now displays her current location.She captures this second map image in the same way she captured the first map (step 3) and plots her new location, again using the YAH marker as a guide (step 4).Using this information the application georeferences the map image, so that as Michelle walks around, her location will be calculated and plotted on to the map image she initially captured.She has not needed to connect to the Internet to obtain this map and she is able to use the stylised YAH map to navigate.

Investigating Constellation Maps
To understand what styles of constellation map exist, we performed an analysis of the YAH map corpus developed by Molineux and Cheverst (2010a).At the time of investigation (03/2013) the corpus contained 413 maps situated in 14 countries.Using this corpus, three categories emerged of map constellation emerged, namely: Identical Map Constellation (35 groups).An identical map constellation is two or more maps that represent one area where each map is visually identical, with exception of the YAH dot's position, which reflects the map's location.

Rotated Map Constellation (6 groups)
A rotated map constellation is similar to the identical map definition, apart from the maps vary in their orientation.Rotated constellation maps tend to be rotated by 90 degrees to other maps in the constellation.
Shifted Map Constellation (6 groups) A shifted map is best described by imagining a large YAH map with each of the maps in the constellation representing a subset of that map.The design of the maps are consistent i.e. the map's colours and graphics are the same, and the YAH maps may contain some overlap.

USER STUDY
A user study was undertaken to better understand the map constellation interaction.There were four main aims for the field study, namely: 1) Investigate how users capture and frame two physical YAH maps from the same constellation.2) Measure the accuracy that can be achieved in the final georeference using this technique.3) Investigate how users interact with the physical YAH map and the smartphone e.g.how the user searches for locations and how they plan their route.4) Gain an insight into users' preferences and feedback on the map constellation interaction.

Experimental Platform
To evaluate and test the interaction, an experimental platform consisting of two systems was developed.The systems were: the client application, and the administrator viewer.
The client application was developed for a Google Nexus One Android smart phone running Android version 2.3.6.The Nexus one has a 480x800 pixel 3.7 inch display and a 5 megapixel camera (2,592⇥1,944 pixels).The user interface used a 'wizard' approach, where the user was guided through each individual step of the experiment.The application was set in a forced landscape perspective.An expert evaluation was undertaken by the authors to evaluate the client application GUI's prior to undertaking the study, to ensure the usability of the interface.
The administrator viewer was developed for an Android tablet, and designed to allow the experimenter to remotely view the client application and captured map images without the need for camera equipment or looking over the participant's shoulder.This functionality enabled the experimenter to further question participants in the field about any interesting behaviours as they happened.

Procedure
The field study was conducted on Lancaster University campus, which has an identical YAH map constellation of 22 YAH maps.Participants were met on campus where they were briefed on the study.Initially, the experimenter walked the participant to the first YAH map (Figure 2) where they were instructed to capture the YAH map as if they were going to use it for navigation purposes.Once captured, the application instructed them to plot the location of the YAH dot on the map image they had captured.Single finger pan, pinch zoom and long press to plot interactions were implemented to enable the user to manipulate the map image and plot the YAH dot's location.Once the YAH dot's location had been plotted, the experimenter walked them to the second YAH map ( Figure 2) where they were asked to repeat the steps they had undertaken at the first map.To assist this step, the participant was given the opportunity to view the picture they took of the first map prior to the camera viewfinder being loaded.Once the second map had been captured and the second YAH dot marked by the participant, the georeference was calculated.Once the map had been georeferenced, participants were instructed to navigate to, a particular location (the ISS building, Figure 2).Once complete, they were given a post-study questionnaire and a semistructured interview was conducted.During the post study interview participants were asked about their experiences and their opinions to the georeferencing technique.The studies took on average 30-40 minutes to complete.Participants were asked to use the think-aloud protocol, which was captured with a Dictaphone.Further, participants were filmed at each YAH map using a tripod mounted camera that was situated approximately five metres from the map.The video footage for all participants lasted 71 minutes in total.
The experiment was conducted in October 2012, and involved participants who were new to the campus and who had at least six months experience with mobile devices.Participants were rewarded with a small financial reimbursement for their time.The study was advertised by posters that were distributed across the university campus.Twelve participants were recruited, seven males and five females, with an age range of 18-29 and a mean of 20.Ten participants had experience using mobile map software.All but one participant stated that they were confident in their ability to use their smartphone.

RESULTS
The experimenter's field notes, voice recordings, captured maps, questionnaire data, and video footage were collated and analysed.Further, analysis of the map images was performed by means of visually comparing and coding the maps.

Capturing the YAH Map
A total of 41 pictures were taken (24 accepted), 18 at Map One (6 retakes) and 23 at Map two (11 retakes).At map one, six participants took one attempt to capture the map, and six took two attempts.For map two, five users took one attempt, four users took two attempts, two took three attempts, and one took four.
In their post study interview, participants were asked to rate their opinion of the following statement on a seven point Likert scale( 1 strongly agree -7 strongly disagree): " Overall, I am satisfied with the ease of completing the task".For map 1, eleven of the twelve participants stated that they either strongly agreed or agreed with this statement, and ten participants made the same statement for map two.Seven participants changed their response for map two to a higher value, which suggests that capturing map two was slightly more difficult than capturing map one.
One area of interest was how participants captured the YAH map, as the quality and composition of the YAH map image has an impact on the quality of the georeference, and subsequently the usability of the map image for navigation purposes.Molineux and Cheverst (2010a), found three initial categories that described how participants captured YAH maps: entire map, map overview and cropped map.In our analysis of the accepted 24 maps, we expanded the cropped map category to better describe the maps captured.Below is a breakdown of the new categorisation: Entire Map Figure 3a ,11/24 maps.A map image can be considered as an entire map if the whole map is captured within the viewfinder.Some parts of the YAH map sign e.g.index or borders can be cropped as long as the map remains complete.
Map Overview Figure 3b, 5/24 maps.Map overview is similar to the entire map category, but also includes additional parts of the map surrounds and background elements such as, the wall behind the YAH map.
Major Cropped Map Figure 3c, 2/24 maps.A major cropped image is a cropped version of the map where large parts of the map are occluded.
Minor Crop no Feature Loss Figure 3d, 3/24 maps.Minor crop with no feature loss is similar to entire map in that most of the map is captured, with the exception of parts that are non-essential for navigation (e.g. the key) have been occluded.
Minor Crop with Feature Loss Figure 3e, 3/24 maps.Minor crop with feature loss differs from no feature loss in that the cropping has occluded some part of the map that is deemed to be an essential feature.We classify essential map features using Lynch's five map elements (Lynch 1960), which are: paths, edges, districts, nodes, and landmarks.
The techniques used by participants to capture the YAH map images were also recorded by means of observation and direct questioning.A quarter of participants (3/12) were observed to align the horizontal edges of the map inside the map's viewfinder, whereas two (2/12) used the top and bottom borders of the YAH map.Two (2/12) participants aligned the top left of the map with the top left of the viewfinder.A further quarter (3/12) of participants said they had noted the position of features on the map on the viewfinder to help them frame the second map.
The map used in the study had an index situated beneath the map image (Figure 3 a-e), which listed building names and numbers.Three (3/12) participants captured the map's index in full, and a further third (4/12) captured part of the index, with varying levels of readability.Nearly half (5/12) of the participants didn't capture the index at all.

Interacting with the YAH Map
One motivation for the study was to investigate how participants interacted with both the physical and digital YAH map.Half (6/12) of the participants used only the phone to search for and plot the YAH dot, while the remaining half (6/12) used the physical map to help them find the YAH dot.One participant used the index on the digital map to assist them in finding the YAH dot, while the remaining eleven (11/12) visually searched the map for the YAH dot.
When searching for the ISS building (the location to navigate to), the majority (10/12) of participants used the physical map.Of the remaining two, one participant knew the destination, and the other used the index on the digital map image.Nearly half (5/12) of participants were observed touching the physical map.Three (3/5) touched the destination building, while two (2/5) traced their finger across the physical map to find the destination.

Accuracy
To measure how accurately participants framed the two YAH maps, the Euclidean distances between four points close to the four corners of each captured map pair were measured.The Euclidean distance measures the distances between a selected point on both maps as if they were in the same coordinate space i.e. as if a participant's first map had been laid directly on top of the second and the distance between points on each map were measured.The four points chosen were the most distinguishable features that were closest to each of the four corners of the campus map (Figure 5).Note, it was important to use distinguishable features rather than arbitrary points as this enabled the two map images to be compared.One participant was excluded from this test, as the level of cropping was too great.As such, only eleven map pairs were measured in this test.The four points were averaged for each participant to assess the overall average difference between the map pairs.Of the eleven participants, eight achieved an average Euclidean distance of less than 100 pixels.Figure 6 outlines the Euclidean distances for each individual point for the eleven participants.Point one has the highest overall level of accuracy.We suspect this is because some participants aligned the top corner when capturing the map pictures and point one was situated in the top left corner.As such, the offset would be greater the further away the point was from the top left corner.Conversely, point 4 (bottom right) has a high Euclidean distance, which suggests participants did not focus on the bottom corner.
Figure 4 presents examples of good, average, and poor georeferenced routes recorded during the field study.The dotted red line displays the actual route the participant took, and the solid black line represents the projected route that was displayed to the participant.The differences between the actual and projected route is the error in the georeference.Note the lower the average Euclidean distance the higher the accuracy of the georeference.

Participant Feedback
Below is a summary of participant feedback from the semi-structured interviews and user comments made during the think aloud transcriptions.
Participants were asked if they could think of anything that would assist them in framing and capturing the two maps pictures.Three (3/12) participants suggested that overlaying a grid on the smartphone's viewfinder may help them align the YAH maps.Two (2/12) participants commented that the application should aid them in correcting the orientation of the camera, and as such lower the skew in the final picture.Two (2/12) participants suggested that a translucent overlay of the previous YAH map image, placed over the camera's viewfinder would help them better align the two YAH maps.One (1/12) participant suggested adding a marker overlay onto the camera viewfinder that would be used to align to a feature on the physical map.

DISCUSSION AND CONCLUSIONS
The findings from the study and the participant feedback suggest that, even in it's most primitive form, the map constellation interaction is a strong candidate georeferencing interaction.
The constellation interaction complements current georeferencing interaction techniques, such as the two point interaction and smart alignment (Sch öning et al. 2009).We envisage all georeferencing interactions being combined into a single platform that enables the user to select an interaction depending on the resources available.For example, a user would firstly capture the YAH map image and plot their location with assistance of the YAH marker.They then begin walking away from the physical map and after a short distance would plot their new location manually (using the two-point interaction) or, if they encounter a second physical YAH map, use the map constellation interaction.
When designing this interaction technique, we avoided adapting the phone's viewfinder while the map image was being captured in order to view how users would capture the YAH map unaided.Further research is required to investigate which visual aids (such as grids, or borders) would best facilitate the user and complement the framing strategies witnessed during the study.
On analysing the maps a secondary finding emerged, only three participants framed the map in such a way that it contained the YAH map's entire index.It could be argued that the index is necessary to navigate in the University campus, as the map uses reference numbers to represent buildings.To this we suggest that applications designed to capture YAH maps for navigation ether instruct the user to consider the index or enable the user to capture a separate image for the index.
Although In-field georeferencing techniques may not be able to achieve the level of accuracy of current mobile map applications, they enable the user to use the stylised YAH map and its content (e.g.walking routes, historic information), which may not exist on a traditional mobile map application.Further, the accuracy and reassurance a user needs when wayfinding is relative to the wayfinding task they are undertaking.For example, if a user was following a trail that has a fork half way though the route.If all that they required was reassurance that they had taken the correct path, the application would only need to display the user's approximate location.This spectrum of reassurance when wayfinding reflects the varying levels of detail in YAH map's that we have observed in the map corpus.
In order to develop this work further, we intend to perform additional research relating to methods of aiding the user in framing the YAH map and investigate applying computer vision techniques to complement the map capture strategies found in the study.

Figure 1 :
Figure 1: Two maps of the same public park.(left) Google maps, (right) the park's YAH map outlined two main principles of YAH map orientation, namely c The Authors.Published by BISL.Proceedings of Proceedings of BCS HCI 2013 The Internet of Things XXVII, Uxbridge, UK.

Figure 2 :
Figure 2: The location of the two YAH maps, the route taken from map one and map two, and the ISS building

Figure 3 :
Figure 3: Examples of You-Are-Here maps captured for each map category: a) entire map, b) overview, c) major crop d) minor crop no loss, e) minor crop with loss

Figure 4 :
Figure 4: Examples of participants' actual routes and their projected routes.From left to right: Good (Euclidean distance 60px), average (Euclidean distance 89px) and poor georeference (Euclidean distance 159px)

Figure 5 :
Figure 5: The four points used to measure the differences between the two captured images

Figure 6 :
Figure 6: A box plot of the Euclidean distances for each of the four selected points used to measure accuracy