An Empirical Framework for Understanding Human-Technology Interaction Optimisation for Route Planning

A number of interactive systems have been developed in the past to simulate or improve optimised route planning as part of problem solving (e.g. Vehicle Routing Problems (VRPs)) focussing mainly in the utilisation of computational algorithms. Main reasons for developing such interactive systems is that they combine the strengths both computerised systems and humans have, to aid the generation of optimal solutions and promote green logistics. Under a joint-cognitive perspective, the system and the human operator (user) become parts of a single ecosystem, co-operating to complete a task and in which cognitive technologies aid them to reach a decision. This paper reports the performance-based design of such an interactive tool that supports optimisation in route planning. It aims to identify human performance, behaviour and opportunities for designing innovative user-centred interactive optimisation tools for route planning. Twenty-six users evaluated the interactive route planner. Results suggest that switching strategies while planning routes lead to increase in route optimality while providing different levels of control for the user. Results lead to the extension of a joint-cognitive approach framework for optimisation routing problems that takes into account both performance metrics and contextual factors such as changes within the task environment. Related implications to optimisation systems’ design and evaluation are also discussed with a particular focus on how new ubiquitous navigation technologies can be improved to promote co-operation and more optimal route planning. performance-based design, empirical model, joint-cognitive systems, route planning optimisation


INTRODUCTION
Problem solving constitutes a key component of human thinking processes that affect every day activities (e.g. from route planning to crises resolution) (Michalewicz & Fogel, 2000).Optimisation problem solving particularly incorporates a number of fundamental mathematical principles and heuristics to be employed to reach to the best solution in a given problem (Toth & Vigo, 2002).Whilst there is a plethora of computational methods for solving hard optimisation problems, humans are found to be competent in solving complex optimisation problems (such as Traveling Salesman Problem -e.g.MacGregor & Ormerod, 1996) creating a potential for exploring further human capabilities in optimisation planning.The main advantage computers offer in optimisation problem solving include 1) computational power to complete tasks and 2) speed for managing the demands for large numbers of customers (i.e.large problem space).However, a common challenge is that as customer numbers increase, the time needed to find optimal solutions increase exponentially (Maringer, 2006;Michalewitz & Fogel, 2004;Hu, 1982).Furthermore, the application of such computational algorithms is often tied to particular problem specifications making their applications not easy to be generalised in other domains.The efficiency and effectiveness of computational algorithms are still debatable as they are very much dependent on the problem specification and task environment (Crainic, Gendreau & Potvin, 2009).Furthermore, changes in the world market caused by globalisation and liberalisation of industries have led to the creation of branches of production units and depots that exist in ever more disparate and distant locations.This affects supply chains financial status and safety in the eco-system as more longdistance transportation needs result in increased fuel consumption.According to the International Transport Forum (Crainic et al., 2009), freight transportation is responsible for approximately one third of CO 2 emissions of world transportation.Transportation systems constitute an important part of the global economy (transportation systems determine around 10-20% of the final cost of the goods - Toth & Vigo, 2001).As such, a technique or systems that could provide optimal solutions in an efficient manner is demanded (Maffioli, 2003).Intelligent Mobility (IM) modern initiatives that recognise the need for optimising both the flow of people and goods (Bodhani, 2012) makes the development of intelligent optimisation algorithms even more critical yet complex as the stakeholders and assets involved increase within the network.
In the present paper, the design of the prototype of the Problem-Solver is outlined and the notion of an optimisation interactive platform with crowdsourcing capabilities is discussed.Although there are advanced computational techniques that tackle complex optimisation problems (e.g.see Toth & Vigo, 2001;Burke & Kendall, 2005), the effectiveness of these are often tied to problem specifications and sample sizes.The author argues that there is a great potential in crowdsourcing (in this context defined as user-driven) route planning optimisation to enhance user context awareness, learning and experience.

Capacitated Vehicle Routing Problems (CVRPs)
Capacitated Vehicle Routing Problems (CVRPs) are hard optimisation problems with high applicability in freight, supply chain and logistics.A CVRP requires a number of routes to be constructed under a weight restriction on the total load which can be transported for each route (e.g.no more than 100 weight units per route).A further characteristic of CVRPs is that the number of potential solutions is too great to allow problem solvers to exhaustively test each one before 'solving' the problem.CVRPs" objective is to minimize the total length of the distance travelled.In order to solve CVRPs, one has to discover a number of shortest routes taken by a capacity-limited vehicle from one or more depots to deliver to customers (nodes) distributed in Euclidean space (Figure 1).Each node must be visited once only and the vehicle must not exceed a weight limit (in Figure 1, it is 100) for each route.As such, CVRP solving -being a combinatorial problem-incorporates two core goals (e.g.shortest route and meeting the weight constraints per route construction) which are of equal importance and form the optimisation criteria (time constraints are not important for this VRP variation).If these goals are not met simultaneously, the generated solutions are deemed invalid (the valid solutions may be satisficing, close-to-optimal or optimal).In a more recent study, Kefalidou and Ormerod (2014) extended this research on CVRPs and found that human performance is still close to optimal even in more demanding optimisation problems such as CVRPs.They also found that certain cognitiveinspired heuristics such as calculating and clustering can be predictors of close-to-optimal performance manifesting a strong switching behaviour.As such, investigating human thinking processes and performance when attempting to optimise could prove to be extremely useful in the development and improvement of CVRP optimisation algorithms, leading to solutions that are closer to optimal.

Joint Cognitive Systems
Cognition in tasks performance has been explained before through the lenses of "control" (e.g.see COCOM model -Hollnagel & Erik, 1993), where competence (i.e.set of actions or responses that a system can apply within a given context), control (i.e.how competence is applied through orderly performance) and constructs (i.e.basis for selecting actions and interpreting information to create system assumptions about a situation) constitute the fundamentals for understanding orderly performance.Orderly performance have been investigated before through the notion of joint cognitive systems.Within a joint cognitive system, operators (as intelligent human beings) interact with machines on both internal and external levels but with a clear focus on external human-machine interactions where the system boundaries are clear depending on the context of use (Hollnagel & Woods, 2006).Optimisation research has utilised in the past such a framework in ways that facilitated either direct human interventions within an optimisation routing or indirect interventions (interventions in this case are associated to humanmachine interactions) in order to perform optimisation operational tasks.Such interactive applications for optimisation problem solving have been designed and implemented since the 1980s (Boyd & Vandenberghe, 2004) ranging from simple everyday problems to more specialised problems for academic and scientific research such as spacecraft design (e.g.Fukunaga & Stechert, 1997), optimisation models (e.g.Sreevalsan et al., 2007) and supply chain (e.g.Waters, 1984).
While prior research focused on testing "human-inthe-loop" frameworks in optimisation tasks where computational algorithms were either embedded within the system or operators manipulated solutions, in this paper, the author presents a first step in applying the "human-in-the-loop" framework in solving Capacitated Vehicle Routing Problems (CVRPs) without the use of any computational algorithm but by providing computational support for certain aspects of the CVRPs (i.e.calculations) and direct route construction by humans.In doing so, the author aims to provide a performance-based design for an interactive tool that offers support in direct route planning, providing insights as to how human cognition can enrich this task.Within this context, performance-based design is defined as the design process of joint-cognitive interactive technologies and systems that is driven by human performance, similar to Kalay"s (1999) notions.

DESIGNING FOR ROUTE OPTIMISATION
The present research followed a performance-based design acknowledging prior research in route optimisation that suggests that humans are able to generate close-to-optimal solutions to hard optimisation problems (e.g.TSPs (e.g.MacGregor and Ormerod, 1996) and CVRPs (e.g.Kefalidou and Ormerod, 2014).Kefalidou and Ormerod (2014) suggested that there are a number of human heuristics employed in an attempt to generate a close-to-optimal solution to a CVRP while at the same time the presence of errors in sub-optimal solutions was not absent.They also found evidence of switching behaviour when utilising different heuristics.
As such, the design of the interactive route optimisation planning tool focused on incorporating four main design features: 1) accommodating heuristic switching, 2) error tracking, 3) route planning and re-planning and 4) Logging.
Figure 2 shows the main actors involved in the system and the basic operations of the system towards solving a CVRP problem using the applet.The rectangle in the middle of the diagram indicates the interactive level of the system between the users (humans) and the computer.The users and the computer are the external actors of the system; both are fundamental for the successful integration and use of the system.The CVRP problem is displayed after the user determines which file needs to be opened.Then the software applet parses the text file with the coordinates and displays the visual CVRP problem on the applet"s window.Both actors (user and computer) are necessary for the initiation of the problem-solving phase.Furthermore, both users and computer are necessary for solving a CVRP.
While the user is solely responsible for solving the problems, the computer plays an assistive role by providing some basic and simple checks on the violations that a user might make, and outputting messages informing the user about these violations.Additionally, the computer applet calculates the length of the drawn routes on the fly, as well as calculating the current total weight of the nodes included in a route.The user has this information available throughout the completion of a problem and the use of the software.

Figure 2: CVRP tool basic operations and actors.
A Hierarchical Task Analysis (HTA) diagram is provided below (Figure 3) that shows the activities involved in solving a CVRP problem using the software applet as well as the flow of these activities.An HTA was employed because it allows better consideration of what tasks users can and are engaged to when attempting to plan their routes while at the same time offers potential alternatives for their actions in a hierarchical and structural manner (Annett, 1996).
At the first level of HTA diagram, the user needs first to open the software.The "open software" level has a number of sublevels that introduce the actions involved for solving a CVRP.For example, once the tool is initiated, users have four options that need to be executed (at least some of them) one after the other.More specifically, the users need to first feed to the system text files with the problem details to get the problem visually displayed on the applet"s window.They then can start solving the CVRP and plan routes, while modifying their solution, if they want.The latter two activities (starting solving the problem and modifying solutions) can be repeated as needed.The "Start solving a problem" activity is further decomposed to other activities such as "click the "Route" tab", "delete a route", "click the "Group" tab" and "reorganise cluster".It is not necessary to execute the "click the "Route" tab" and "click the "Group" tab" activities in a specific order.The "delete a route" and the "reorganise cluster" assume the previous use of the "click the "Route" tab" and "click the "Group" tab" activities.The latter activities can take place as many times as needed by the users.

Discovering human performance, interaction and heuristics
The experiment-based study was conducted to evaluate the produced software toolkit (CVRP Planner) and investigate human performance, interactions and behaviours when interacting with it.The study followed appropriate ethics requirements at Lancaster University where data collection took place.Observations and logs were obtained tracking human performance and behaviour.
Participants.Twenty-six participants (13 male and 13 female -M age = 20.1;SD age = 5.33) used the CVRP tool with strategy support.All participants were paid £6 upon completion.Participants were both English-native and foreign speakers.Materials.Participants solved 5 computer-based stimuli with CVRP nodes drawn on them.The CVRPs that they solved were the following instances (X-Y, where X is number of customers to serve and Y is number of routes to draw): 33-4, 33-5, 39-6 and 45-7.The CVRP problems used in this experiment were the same with the ones used in Kefalidou and Ormerod (2014).Problems were displayed as in Figure 4. Participants used the mouse to drag-anddrop the pointer to construct each route.Colouring for each route was automatic.Participants were instructed to use (if they wished to) a "Clustering" strategy through the use of the "Group" option tab of the tool and a "Calculating" strategy through the use of the "Routes" option (as per prior research literature e.g.Kefalidou & Ormerod, 2014).Camtasia, a desktop real-time recording software, was used to log video files of all participants" activities while they were solving computerised CVRPs.The software also recorded any informal verbalisations made by participants and had an embedded logging activity.Procedure.Participants were firstly, introduced to the aim and objectives of the study and were asked basic demographic questions.They were then had a brief training session with the computerised tool that lasted for approximately 10 minutes.The instructions on how to solve the CVRPs were both orally explained by the author and were also given a paper-sheet with them written on.The first problem that they were given to solve was a practice problem and as such it was not included in the analysis phase.The CVRPs were given to each participant in a random order to minimise practice effects.Participants could ask any questions -if they needed-throughout the duration of the experiment.There was no time restriction, however, participants were made aware that the experiment takes approximately 1 hour.

From human heuristics to a cognitiveinspired route optimisation model
All participants solved all CVRPs and used both strategies support provided (i.e.clustering and calculating).Table 1 shows the Percentage Above Optimal (PAO) as calculated based on published optimal solutions for these problems (as provided in Kefalidou & Ormerod, 2014).As it can be seen, participants who used the computerised tool to solve CVRPs performed better compared to prior paper-based results on the same set of problems.
Descriptive data in regards to solution times (Table 2), show that participants who used the computerised tool solved easier CVRPs (33-4 and 33-5) faster than prior research results on paperbased versions suggest.The opposite pattern was observed in the more difficult CVRPs (39-6 and 45-7).This may further suggest that as the CVRP instances grew in size and complexity, the time participants needed to complete the problems successfully increased as well.A potential speculation on that can be that the task environment within the computerised tool triggers a particular cognitive load (visual or not) that slows down the generation of solutions for CVRPs.Interestingly, this does not seem to apply for the paper-based versions as prior research suggests (Kefalidou & Ormerod, 2014).However, further research to unpack this aspect needs to take place.Furthermore, prior research in CVRP problem solving identified a number of different cognitive-inspired strategies employed by humans when solving CVRPs (e.g.see Kefalidou & Ormerod, 2014).Switching across these strategies manifested strongly suggesting that they perhaps contribute in generating close-to-optimal solutions.Switching particularly between Calculating and Clustering strategies were found to promote optimality in CVRP solutions (Kefalidou & Ormerod, 2014).The computerised tool designed and discussed in the present study facilitated the utilisation of either 1) Calculating or 2) Clustering or 3) both strategies or 4) none of these strategies through the "Route -Calculating" and "Croup -Clustering" tabs.Table 3 shows the average switching between the two strategies that had taken place using the computerised tool as observed through the computer logs.Switching occurred throughout route planning for the majority of the participants (i.e.69.2%) utilising the strategy support options.However, there was approximately 30.7% that did not use the strategy support despite they were told it was available.In these cases, participants planned the routes on their own as if on paper.
Considering PAO scores and strategy switching observations, participants appear to generate closer to optimal routes in 33-4 where they switched 115 times on average (for a general performance results -not on switching-please see Kefalidou, 2017).However, the results suggest that this is not a linear trend as participants" solutions to the next more "optimally-planned" CVRP (i.e.33-5) had the lowest strategy switching behaviour from them all (i.e.69 times).An explanation to that may be that the task environment (in terms of problem specification) triggered specific behavioural characteristics that lead to a non-linear performance.For example, the spread of the customers (i.e.nodes) on the Euclidean plane could potentially have been a factor that promoted different frequencies in strategies switching amongst the participants.This would in turn suggest that visual aspects of the routing problem specification is critical in finding close-tooptimal solutions fast and visualisation approaches for this kind of tasks need to consider carefully on how to promote optimisation in planning when interacting with them.However, further research needs to take place to unpack the potential impact visual aspects of customers/nodes spread may have on route planning task performance.

Towards an empirical framework for Human-Technology Interaction Optimisation
Participants made their own choice in using strategies support when interacting with the computerised CVRP tool.This suggests that they maintained a strong element of control on how they could use the interactive system.Whether this has been a conscious or unconscious decision it is difficult to speculate at this instance.However, it is a strong indicator that both the decision to use and not use the strategy support -within the proposed design-facilitated a high level of control in planner"s actions.Planners" performance on CVRPs suggests that this element of control may have been beneficial in promoting closer-to-optimal solutions.Furthermore, the variation that has been observed in terms of strategies switching frequencies, PAO scores and time completion suggest that there may be further contributing factors that interplay with levels (or thresholds) of control in optimisation planning that can affect performance.Examples of such are cognitive load as imposed by the task environment, personal behaviour (e.g.choice of strategies) and of course the problem space (e.g.problem specification and constraints).Taking into account Hollnagel and Erik"s (1993) COCOM model (where the "control" element highly features), the author proposes an adaptation of this model to fit the cognitive-inspired optimisation route planning for CVRPs (see Figure 5   problem space and specification) and context of route planning.In the route optimisation context, environment comprises of task complexity (including for example, customers spread in the Euclidean plane), number of routes and customers to serve, available solutions and search for solutions and finally mode of execution (in this case here it is the use of the computerised interactive CVRP tool).
Decision-making for choosing the appropriate strategies forms an iterative bi-directional loop to facilitate strategies switching behaviourthe same applies for strategies choice and effects on human performance.
Depending on the context, performance could affect the task environment in turn, forming another loop within the whole model (i.e.perforated arrow between performance and task environment in Figure 5).A next step would be to further test the validity of this model not only in other CVRPs but also within other optimisation contexts.

FURTHER DESIGN IMPLICATIONS AND CONCLUSIONS
Route optimisation being a complex planning and problem-solving process, is theoretically difficult to analyse.However, empirically has been found that human operators are good at solving and completing such complex tasks despite their difficulty.This provides new opportunities for building interactive tools (following a "human-in-theloop" approach) that employs all the strengths that a joint cognitive system entails (i.e.human operator and machines - Hollangel & Woods, 2006).
In this paper, the author presented some results from research done in identifying human performance in solving CVRPs using a computerised interactive tool that fully supported the construction of routes (from scratch) by humans and made available the utilisation of two different cognitive-inspired strategies (i.e.Calculating and Clustering).Strategy switching was found to be prominent in human route planning and supported the generation of good solutions, however, a strong element of control in planning was also apparent.This strong control element informed the adaptation of COCOM model (Hollnagel & Erik, 1993) to CVRPs optimisation.It was found that what technology can do in this case is to increase the likelihood of humans generating "close-to-optimal" solutions in route planning by supporting them in performing traditionally-challenging tasks such as calculations.However, at the same time, interactive technology was found to facilitate a strong element of control (from the user"s perspective) in regards to how they 1) interacted with the computerised tool and 2) planned the routes.It appears that the facilitation of a layer of control has promoted optimality in planning even in the case where people had to plan (from scratch) the routes.As such, clear design implications emerge for creating a joint cognitive route optimisation system that takes advantage of 1) human capabilities in employing cognitive-inspired strategies that lead to more optimal routes fast and 2) computational capabilities to ensure validity in planning solutions.The following set of design implications are discussed below: Implications for Human Operators.Individuals differ in the way they plan routes as they choose different heuristics and strategies to do so.Furthermore, as it has been shown within this paper, individuals make different decisions in regards to strategies switching.As such, a joint cognitive optimisation system needs to accommodate for this flexibility in decision-making and strategy choice.Human operators often carry their own experience and personality in route planning, which they often incorporate it when planning new routes.For example, within a real-life context, an individual that knows that a particular route suffers from traffic jam at a particular time within the day, they would avoid it, even if that meant that the route may be sub-optimal.
Similarly, within a theoretical context, a straight line in planning routes suggests that it is more optimal compared to a curved lineas such, when planning routes for solving an optimisation problem, they may choose to construct straight lines to serve the customers.In effect, context and goals can be critical in supporting human operators more effectively in planning routes.As context and goals can change over time, future joint cognitive optimisation system should be able to adjust and "learn" the process of this change.In such a case, the cognitive optimisation system can build a user/human operator model, which encapsulates operators" "stable" factors (e.g.priorities, current knowledge) and transient and dynamic factors (e.g.goals, solution history, current activities, and information feed).
Both these factors are manifested via attention focus, cognitive load and decision-making processes and they can be interlinked as part of the user cognitive model.Social Implications.A key design opportunity for a cognitive-inspired optimisation system is to support communication and knowledge transfer amongst the different human operators.This could be realised via the provision of collaborative optimisation tasks (e.g. for route planning or other optimisation).Collaborations could manifest based on e.g.location and goals.A machine learning cognitive approach would require operators to construct their routes, share them amongst others and through this process, learn and provide further suggestions.
Similarities and patterns amongst different operators (and even optimisation computational algorithms) would be tracked and matched to each other to speed up optimal solution generation.In such a joint socio-cognitive system, the system views people and it internal states as one for connecting and facilitating the knowledge transfer to promote optimisation.Despite the strong advantages of such approach, there are also clear privacy and security issues to take into account.As such, a careful consideration of these should take place when designing such interactive systems.For example, as has been demonstrated within this paper, control in optimisation can alleviate privacy and security concerns by facilitating high thresholds of control in sharing intelligence (e.g.route plans) with other individuals and/or computational systems.
Contextual implications.Context and task environment is important in determining performance.Context within a route optimisation setting can be related to real-life situations and theoretical situations.When designing joint cognitive optimisation systems, one would expect to have the availability of both real-life and theoretical context scenarios to be able to either 1) provide a solution to a CVRP problem or 2) practice route planning.Furthermore, location of where the planning takes place can alter the process.For example, a human operator is exposed to different stimuli and influences when they have to plan a route "on-thego" versus when they have to do within an operations room.Future systems design should allow for flexibility in this direction to facilitate different contexts and goals.
Usability implications There are clear usability implications, especially when considering route optimisation "on-the-go".For example, "on-the-go" optimisation via e.g.mobile devices engages human cognition differently from when it takes place via other media (e.g.large displays).There is long prior research that suggests that human visual perception alters depending on the task environment (e.g.Marr, 1980).As such, future joint cognitive optimisation system need to take into account the strengths and weaknesses of the human vision system so that their application does not negate the positive characteristics that human vision exhibits in e.g.route optimisation contexts.Furthermore, cognitive load and workload are known to have effects on human performance.It is also known that task environment can either facilitate or minimise such effects.As such, future cognitive optimisation systems need to enable users to "toggle" the visuals and information they receive at any given moment when completing an optimisation tasks such as CVRPs.This is also supported in the results of the present study.Technology in that way, has the potential to allow human operators to either be "open" or "closed" to task environment characteristics to minimise cognitive overload and distraction.
Route planning optimisation such as CVRPs is a hard mathematical problem task.However, this research suggests that humans are able to interact with a computerised system successfully to plan routes from scratch and to jointly reach to close-tooptimal solutions.While there are certain challenges and limitations in the present study, there has been a first step taken to provide a joint cognitive optimisation systems approach that allows humans to construct and plan routes from scratch without the use of any computational optimisation algorithm.Results are promising and a next step would be to encapsulate computational algorithms into the system to track interactions, behaviours and performance.The present research further suggests that control in optimisation appears to play an important role in promoting optimisation in route planning and as such an adaptation of the COCOM model has been suggested that fits the context of route optimisation.

Figure 3 :
Figure 3: HTA Diagram for solving CVRP using the computerised interactive tool

Figure 5 :
Figure 5: Empirical Framework for interactive optimisation in CVRPs (adapting the COCOM model)As it can be seen, similarly to the COCOM model, control forms the basis of interaction within a joint cognitive system perspective even within the route optimisation context.Factors such as attention, cognitive load and decision-making (in terms of strategies choice) appear to affect control and consequently human performance in solving CVRPs through a designed computerised tool.Events and context align to the task environment (including

Table 1 :
Mean % above optimal (PAO: SDs in brackets) for each problem

Table 3 :
Mean switching between Calculating andClustering strategies across problems using the tool