An Ontological Approach to Inform HMI Designs for Minimising Driver Distractions with ADAS

ADAS (Advanced Driver Assistance Systems) are in-vehicle systems designed to enhance driving safety and efficiency as well as comfort for drivers in the driving process. Recent studies have noticed that when Human Machine Interface (HMI) is not designed properly, an ADAS can cause distraction which would affect its usage and even lead to safety issues. Current understanding of these issues is limited to the context-dependent nature of such systems. This paper reports the development of a holistic conceptualisation of how drivers interact with ADAS and how such interaction could lead to potential distraction. This is done taking an ontological approach to contextualise the potential distraction, driving tasks and user interactions centred on the use of ADAS. Example scenarios are also given to demonstrate how the developed ontology can be used to deduce rules for identifying distraction from ADAS and informing future designs.


INTRODUCTION
Advanced Driver Assistance Systems (ADAS) are in-vehicle systems designed to reduce and even eliminate driver errors, enhance driving safety and efficiency as well as comfort in the driving process.There are various types of ADAS, ranging from automated systems that take control of the vehicle (e.g., Anti-lock Braking Systems) to those providing real-time advice, instructions, and warnings to alert drivers to potential problems (e.g., Blind Spot Information Systems) (Hojjati-Emami et al., 2012).There are also adaptive ADAS that can provide drivers with information initially and also assist them with automated actions eventually (e.g., Automatic Brake Control).More recently, the adoption of vehicle communication technologies as well as new sensors have enabled ADAS to generate more information about the surrounding environment.Such information can offer drivers a better understanding of the current situation and also help them make a precise prediction for unforeseen events.
Driving is a critical everyday task that requires a high visual and cognitive load (Underwood, 2002).Having more information available to drivers does not necessarily mean the information is always effective to assist them to make informed decisions in the driving process.In fact, there are concerns that ADAS may create a new source of distraction that affects driving safety if the system, and the functions they support, are poorly designed, presented, or used inappropriately (Lee et al., 2008a, Adell et al., 2011, Vaa, 2011).In other words, the Human Machine Interface (HMI) that is used to present ADAS information and warnings must be carefully designed to retain the driver's attention while eliminating potential distraction through the use of appropriate interaction techniques (Várhelyi et al., 2015).There are studies focusing on improving the HMI design for specific ADAS warnings in certain driving situations or for autonomous driving (Tonnis et al., 2007, Hülsen et al., 2011, Armand et al., 2014, Zhao et al., 2015).However, there still a lack of a holistic understanding of how drivers interact with ADAS An Ontological Approach to Inform HMI Designs for Minimising Driver Distractions with ADAS Botao Fan • Nan Jiang • Huseyin Dogan • Raian Ali • Jianbing Ma 2 and how such interactions can lead to distraction in the driving process, which is also fundamental to help develop new ADAS applications and modalities exploiting a range of emerging technologies.In this paper, we take an ontological approach to conceptualise and contextualise the potential driver distraction, driving tasks and user interactions centred on the use of ADAS.The developed ontology consists of three dimensions: ADAS, driving distraction and driving task.Two driving scenarios involving single and multiple ADAS sensors respectively are also provided to demonstrate the use of the ontology in real world.
This paper is organised as follows.The related work is presented in Section 2 followed by the ontology development in Section 3. Section 4 provides two driving scenarios to demonstrate how the ontology can be applied to detect ADAS driven distraction and the conclusions are drawn in Section 5 with discussions on the future work.

RELATED WORK
Advanced Driver Assistance Systems (ADAS) and In-Vehicle Information Systems (IVIS) can provide drivers useful in-traffic information and hazard warning to enhance their perception of surrounding environment (Golias et al., 2002, Rendon-Velez, 2010).However, there are concerns about a concomitant increase in driver distraction arising from their use (Harbluk et al., 2002) as they can deliver more intensive information to the driver in the driving process that requires their attentions (Brookhuis et al., 2001).Sometimes these systems can also trigger secondary tasks resulting a shift of their focus from driving, the primary task and make them unable to resume this task quickly enough.This will substantially increase the risk of accidents (Häuslschmid et al., 2017).Such issues will become more serious when the primary task demands high visual attentions and/or requires complicated actions from the driver.

ADAS classification
ADAS is designed to assist drivers to enhance or improve their driving through providing information to drivers, taking over the control from them or both.As the main goal of ADAS is to enhance driving safety, ADAS can be classified based on the possible safety benefits by considering road accident factors (Golias et al., 2002):  Avoidance of inappropriate speed;  Keeping appropriate longitudinal and lateral distances;  Support of driver awareness.Troppman (2002) provides a refined classification by considering the efforts and mechanisms to obtain such benefits as illustrated in Figure 1.That is, whether the stated safety benefits can be obtained by simply activating the relevant ADAS or through communications to drivers.For example, an active safety ADAS will take over the control of the vehicle once it has been activated while ADAS providing passive safety only make the driver aware of potential hazards in the surrounding environment.This research focuses on the latter as it involves information provision of ADAS.

Figure 1:
The ADAS evolution and its impact (Troppmann, 2006) It should be noted that there are other ADAS classifications based on the driving process or sensor types.For example, Rendon-Velez ( 2010) presents an ADAS classification model featuring three categories on the basis of the sub process of driving that ADAS are helping with: (1) perception, (2) analysis-decision and (3) action.There is also an ADAS classification based on sensor data cluster (Rezaei et al., 2010) such as environment sensors, driver sensors or vehicle sensors.

Interacting with ADAS
When an ADAS is mainly designed for enhancing drivers' perception of the road, its main functionality is to present timely information/warning about the driving environment and deliver warning messages to potential hazards to the drivers (e.g., route guide or speed limits (Lu et al., 2005)) through a Human Machine Interface (HMI).As the information is presented to enhance drivers' perception instead of distracting them from driving (Golias et al., 2002, Rendon-Velez, 2010), the design of ADAS user interface is crucial to the effectiveness of the delivery.Visual, audio and haptic feedbacks are the most common modalities considered by automakers.Visual modality is the fundamental strategy for information delivery.Presentation of information in the visual modality should enable the user to perform tasks (e.g.search for information on the display) effectively, efficiently and with satisfaction (ISO, 2017).Dashboard and central console is the main visual display device.A novel 3 device is called "head-up displays" which allows drivers to concentrate on the front window (Jakus et al., 2015, Kim et al., 2016).Meanwhile, visual warning design according to different colour, content or symbols represent different meaning and more important which could reflect the urgency of a hazard (Wogalter et al., 2002).
Compared to visual displays, auditory displays require little directional search and responses tend to be faster than to visual displays.According to (Campbell et al., 2004), audio modality can be classified into simple tones; auditory icons; earcons; and speech (verbal) message.Also, volume should be carefully designed that should not startle the drivers (Stevens et al., 2002).
Haptic warnings are suitable for alerting the driver about critical situations and helping with taking corrective action (Enriquez andMacLean, 2004, Lee et al., 2004).These technologies enable to deliver haptic (tactile or kinaesthetic) feedback at various areas of the car, such as the steering wheel, the seat, or the pedal.Counter-torques on steering wheel are most often implemented in systems that help drivers to keep in their lane or prevent them from leaving the road (Suzuki and Jansson, 2003).The haptic seat is a proven solution to provide spatial cues to the driver by vibration of multiple haptic zones (Fitch et al., 2007).A haptic pedal can be used to provide drivers an action recommendation through counter pressures, e.g.reducing speed or braking (De Rosario et al., 2010).A haptic seatbelt can use short tensions of the strap to warn of, e.g. a possible forward collision (Chun et al., 2012).
In addition, certain ADAS (e.g., information ADAS) may require drivers' interaction after delivering a message to them in order to determine whether they should keep sending messages or take over the control of the vehicle.In that case, drivers can interact with the ADAS through physical controls on the dashboard or central console (Kern and Schmidt, 2009) as well as voice commands (Damiani et al., 2009).More recently, there are also new interfaces that allow drivers for interaction by using air gestures (e.g., (May et al., 2014)).

Driving distraction
Driving distraction occurs when a triggering event induces an unintentional shift away from a safety driving.It is commonly understood that driver distraction is a subset of inattentions where an explicit activity other than driving (e.g., operating a cell phone) competes for the attention of the driver (Streff, 2000, Cohen and Graham, 2003, Lee et al., 2008a, Young et al., 2008).It is further considered as a mismatch between demanded and devoted attention (toward the road) under an overload of resources (Hurts et al., 2011).
Most research consider distraction as excessive workload (overload) and limited attentional resources (Hurts et al., 2011).Brookhuis and de Waard (2010) define workload as the mental resources or information processing capacity devoted to a task.As a consequence, it is generally associated with a recognition or processing delay (Pettitt et al., 2005) or a deterioration of driving performance (Regan et al., 2011).Another perspective is the engagement in a secondary task can be initiated by the driver but also by the task itself being a compelling stimulus (Regan et al., 2011).Some researchers argue that the latter is a controlled mechanism (and a conscious decision) that gave rise to distraction (Lee et al., 2008a).Drivers can time interaction better: they initiate interaction themselves when the current and predictable task difficulty is manageable.In contrast, when reacting to compelling stimuli (such as a ringing phone), drivers often neglect the upcoming driving demand (e.g., a unpredicted event) (Nowakowski et al., 2002).Compelling stimuli may put pressure on the driver.Regan et al. (2011) suggest that the psychological mechanisms involved in self-initiated and stimulus-initiated distraction might differ and could lead to different patterns of interference.
Four distinct categories of distraction have been recognised: visual distraction (e.g., looking away from the roadway), auditory distraction (e.g., responding to a ringing cell phone), biomechanical distraction (e.g., manually adjusting the radio volume), and cognitive distraction (e.g., being lost in thought) (Ranney et al., 2000, Young et al., 2007).Diver distraction has potential to cause or contribute to a crash as it may negatively affect driving performance, including reduced longitudinal (Hancock et al., 2003) and lateral control (Liang and Lee, 2010); reduced situation awareness (Ma and Kaber, 2005); and increase response times to potential hazard (Burns et al., 2002).These degradations can be translated into an increased risk of crash or near crash involvement, with estimates indicating that secondary task distraction is a contributing factor in up to 23% of crashes and near-crashes (Klauer et al., 2006).RoSPA (2018) indicated that distraction can be either driver initiated or non-driver initiated and it can be caused by objects, events or activities either inside the vehicle or outside the vehicle.It should be noted that this paper only focuses on distraction that is driver initiated and caused by in-vehicle activities.

Driving tasks
ADAS can generate secondary tasks for drivers in a way to support them.Secondary tasks are the tasks that a driver voluntarily or involuntarily engages in.These tasks do not directly pertain to the primary task of safe vehicle operation (Ersal et al., 2010).Traditionally, secondary tasks can be split into two or three groups of activities but with a bisection method (Häuslschmid et al., 2017), secondary tasks refer to as all other activities not to do with safe vehicle operations.For instance, tasks such as activating the headlight, enabling cruise control, or adjusting the windshield wipers can all be considered as secondary tasks.Primary driving task is subjected to a high visual load with variable cognitive load, leaving only limited resources remain for secondary task interaction (Bach et al., 2009).To understand the nature of interaction process impact on driving, defining primary task in a precise manner is required.Primary driving task analysis falls under the larger field of driver behaviour models (Michon, 1985).Driver behaviour models can be split into units, such as shifting gears, steering, and braking.But nearly all of the previous research concentrates on the action side of driving, and there is a lack of consideration of driving context components influencing driver behaviours (Bellet et al., 2010).Driving context means physical variables in environment in terms of noise, light levels and space availability that is unpredictable by nature (Wheatley, 2000).The weakness of the concept of action side of driving is that drivers' behaviour could be influence due to the changing of driving condition.This prompted the introduction of a taxonomy approach for the driving situation (Fastenmeier and Gstalter, 2007).The primary driving tasks can be divided into longitudinal situations, intersection situations and other driving situations.Here other driving situations are referred to as special situations or tasks which can be optionally modelled with the tasks in longitudinal traffic or at intersections, respectively (e.g.crossing traffic apart from intersections, special situations such as police car approaching and driving under special sight-and weather conditions).

ONOTLOGY-BASED APPROACH
Ontologies are widely used in the field of Artificial Intelligence (AI) to constitute specific knowledge bases.An ontology is a formal, explicit specification of a shared conceptualisation (Studer et al., 1998).Specifically, it can be considered as a formal description of concepts within a class/concept, descriptions of the features and attributes (slots) of the class/concept and restrictions on the slots (Noy and McGuinness, 2001).There is a process to follow when developing an ontology which usually includes: purpose identification, knowledge acquisition and formalisation, conceptualisation, modelling and evaluation.
There are four main methods to develop an ontology (Jones et al., 1998): TOVE, ENTERPRISE, METHONTOLOGY and IDEF5.In our research we followed the IDEF5 method.The main rationale is that the purpose and requirements were not clear enough to begin with due to the novelty of the idea and the lack of benchmarks.This means a stage-based approach like TOVE or ENTERPRISE would not be suitable (Jones et al., 1998).Moreover, as the development of ADAS is always evolving as a result of the rapid adoption of new sensor and communication technologies.This in turn results in an evolving domain of interest.Therefore, IDEF5 is more appropriate than METHONTOLOGY as it emphasises on the iteration of ontology refinement and validation when new knowledge is identified.
The general procedure of IDEF5 includes:  Organising and scoping: define the purpose and context of the ontology  Data collection: extract the raw data for developing the ontology  Data analysis: analyse the extracted data for establishing the ontology  Initial ontology development: establish a preliminary ontology  Ontology refinement and validation: the preliminary ontology will be iteratively refined and tested.

ORGANISING AND SCOPING
The main aim of this stage was to define the domain and scope of the ontology based on the understanding of the research purpose.The purpose of the research was to establish a holistic conceptualisation on how drivers perceive and interact with ADAS warnings and how such perceptions and interactions could lead to distraction in the driving process.In detail, a passive safety focused ADAS can detect the driving environment and generate data fusion as an input and demand relevant actions from the driver (Troppmann, 2006).As such actions are referred to as secondary tasks (Häuslschmid et al., 2017), when drivers are engaged in responding to ADAS to take the required actions, distraction may occur and compromise the primary driving task (Klauer et al., 2006) (Lee et al., 2008b).Therefore, the following objectives were set as 'completion criteria':  OBJ1.Identify the objects that can be detected by ADAS  OBJ2.Identify the possible interaction between a driver and ADAS  OBJ3.Identify the functions that can be provided by ADAS  OBJ4.Define the primary driving tasks  OBJ5.Define the types of distraction  OBJ6.Identify the impact of different distraction on driving performance  (Fu and Huang, 2010) (Pollard et al., 2013) Driver state detection Defined as the drivers' ability to drive (Daza et al., 2014) (Koesdwiady et al., 2016)

Ego-vehicle detection
Estimated the vehicles' current state (Pollard et al., 2013)

Free zone
Combined the state estimation of both unmoving obstacles and navigable space (Pollard et al., 2013) Moving obstacles Estimated the moving objects on road.(Pollard et al., 2013) Communication Vehicular communication (e.g.vehicle to vehicle) (Pollard et al., 2013, Chen et

Duration
Length of a distraction (Papantoniou et al., 2017) Impact Effect to driving performance (e.g.eye off road) (Papantoniou et al., 2017)

Data collection
Raw data was collected through conducting a literature review based on 16 source materials including 6 conference papers, 8 journal papers and 2 websites.The source materials were decided based on their relevance to the objectives as well as their quality (e.g., rigor and transparency in the method followed, expertise of the research team, impact factor and reputation of the publisher).As a result, a total of 134 distinctive instances were extracted in which 74 instances were collected for ADAS (Obj. 1 -3), 43 instances for driving tasks (Obj.4) and 17 instances for distraction (Obj.5 -6).

Data analysis
For each concept, an initial set of classes was created after analysing the instances collected.Note for ADAS and driving tasks, some classes and initial relationships were also decided by examining the existing classifications/ontologies identified through the literature review.In detail, an ADAS classification based on data fusion Pollard et al. (2013) was chosen as the fundamental model to derive classes as it presented a much higher degree of technical granularity (i.e., the capabilities of sensors used by ADAS instead of the safety features offered by ADAS).As this classification does not consider the full provision of ADAS warnings, two more classes, namely user interface and vehicle control were added to the classification.
Moreover, a driving task classification derived both from road traffic situations and drivers' information processing (Fastenmeier andGstalter, 2007, Pollard et al., 2013) was used as the initial classification to identify classes.This classification was further improved by considering the driving task criticality as stated in (McKnight and Adams, 1970).
For driving distraction, as the impact of driving distraction is determined by its duration (Baker and Spina, 2007) and type (Ranney et al., 2000, Young et al., 2007), three classes were created based on the distraction instances: duration, impact and type.
Table 1 shows the detail of all classes created and their references to the collected instances.

Initial ontology development
In order to develop the initial ontology, the structure will be established based on defined classes.The initial ontology consisted of three dimensions: ADAS, driving task, and distraction.
ADAS consisted of three main classes: sensor data fusion, user interaction and vehicle control.Sensor data fusion includes 'environment detection', 'moving obstacle', 'free zone', 'driver states detection' and 'ego-vehicle detection'.As 'vehicular communication' is to exchange information with other vehicles or traffic infrastructures (e.g.traffic light) for an early warning, such communication can be seen as an enhancement of sensor detection.
Thus, 'vehicular communication' will not be considered in here.In the class of 'user interaction', two new sub-classes were derived: 'user input' (e.g., button, touch screen) and 'ADAS output' (e.g., warning).For the latter one, it also included 'content', 'modality' and 'position'.
There were four classes in driving task: 'driving straight', 'turning', 'slow manoeuvre', and 'road character'.The first two were simplified from 'tasks in longitudinal driving' and 'tasks in intersections' based on previous work (Fastenmeier and Gstalter, 2007).

Ontology refinement and validation
A focus group study (Kitzinger, 1995) was chosen as the method to refine and validate the proposed ontology.Eleven participants were recruited in an academic environment.They were PhD students and academics with driving experience.Six participants had more than 10 years of driving experience, three had 5 -10 years of driving experience, and two did not have a full driving license at the time of the study but they were taking driving lessons.The initial ontology with the full list of classes, subclasses and instances were presented to the participants.They were asked to have a full examination on the ontology and discuss the naming conventions, definitions and relationships of the classes.They were encouraged to modify the ontology by adding or eliminating individual instances and proposing class changes based on their own driving experience.During the session, one author acted as the facilitator and two acted as the note takers.Audio recording of the group discussions was also made to assist the understanding of the modifications.The final results were collected after the session.The major changes are discussed as follows.
The initial ADAS ontology contained three main classes: data fusion, user interface and vehicle control.Eight participants considered 'data fusion' as an input source for ADAS which could overlap with 'user input'.Thus, ADAS classes were restructured into 'sensor data fusion', 'interaction' and 'effect'.Here 'effect' is referred to as the outcome of ADAS offering to drivers including 'warning' and 'vehicle control'.
The initial driving task ontology consisted of a class called 'road characteristics'.Participants suggested that the class should be combined into the 'road type' class in 'environment detection' as 'road characteristics' was not a type of task.
The initial distraction ontology was not changed as no participant questioned about the classification.
The revised ontology resulting from the focus group study is shown in Figure 2.

APPLICATION
The ontology was developed to help identify the potential driver distraction caused by ADAS in driving process.Here we present two example scenarios to demonstrate how to use this ontology to identify possible distraction.It should be noted that the two scenarios were based on traffic conditions and regulations in the UK.A state transition schematic diagram will be developed for a better understanding of the distraction process.Single ADAS warnings can be triggered when one specific object is detected.Figure 3 illustrates a driving scenario where the subject vehicle is following a slow-moving vehicle in the same lane (i.e., Vehicle 1) in a built-up area.In this scenario, the vehicle in front (Vehicle 1) is decelerating without any brake indicator (i.e. the driver stopped pressing gas pedal to maintain the speed and did not press the brake to reduce the speed).When the distance between the two vehicles is less than a predefined threshold, a forward collision warning will be triggered and the relevant sign will be presented on the dashboard to inform the driver.

Single warning scenario
Using the ontology developed, the following driving related parameters can be extracted with initial values set by the scenarios as shown in Table 2.Such parameters and values can be fed into a reasoning system to determine when an ADAS will be triggered and the relevant contextual information.Table 3 shows the parameters extracted from the distraction ontology and the assumed values captured in the scenario.As the visual warning will draw the driver's attention away from the road to the dashboard, when the driver is looking at the dashboard for too long, it can assume that the driver is now distracted by the visual warning.Such information can be used in conjunction with the contextual information from Table 2 to determine when a possible distraction can occur.Figure 4 shows the state transition schematic diagram of the single warning process.Multiple ADAS warnings can be triggered in certain driving scenarios when all relevant ADAS are equipped and activated.The Figure 5 illustrates a driving scenario where the subject vehicle is trying to overtake a slow moving vehicle in the same lane on the motor way (i.e., Vehicle 1) where there is another vehicle in the overtaking lane (i.e., Vehicle 2) which is running close to the subject vehicle.In this scenario, a blind spot warning will be triggered to indicate Vehicle 2 as an obstacle for taking over tasks.At the same time, as Vehicle 1 is decelerating, a forward collision warning may be also triggered to inform the driver the potential read-end collision if the subject vehicle is too close to the Vehicle 1.
The following parameters can be extracted from the ontology with values captured in the scenario.The warning parameters are adapted from the previous multi-conflict study by Fitch et al. (2014).

Audio Warning
Table 5 shows the parameters extracted from the distraction ontology and the assumed values captured in the scenario.As the blind spot warning will draw the driver's attention away from the road to the door mirror, when the driver is looking at the door mirror for too long, it can assume that the driver is now distracted by the audio warning.Another audio warning (i.e.forward collision warning) then alerts driver about decelerating vehicle.However, a sudden warning may lead to startle and interfere with driver's action of speed control.Figure 6 showed the transition schematic diagram of multi-warning process.

CONCLUSION AND FUTURE WORK
In this paper, we discussed the development of an ontology aiming to understand ADAS and driving distraction in the context of driving task following IDEF 5 approach.The ontology was refined and validated through a focus group study.Two example scenarios were also provided to illustrate how the ontology can be used to help identify distraction initiated from ADAS when single or multiple ADAS warnings are present.
As the proposed ontology was validated and improved through only one focus group study, further studies involving more participants with good knowledge of driving as well as ADAS need to be conducted to consolidate the ontology.
Certainly, more driving scenarios can also be developed to further validate the ontology.Moreover, although ADAS warnings can lead to driving distraction, some studies suggest that certain aspects of the distraction might be acceptable based on the drivers' ability (Baker and 9 Spina, 2007).Therefore, as a future work, identifying the acceptability level of distraction or assessing drivers' workload thresholds could be considered for consolidating the ontology.In addition, sometimes a multi-conflict driving situation (e.g., overtaking due to emerging vehicle with obstacles on the overtaking lane) may occur leading to the presence of multi-warnings if more than one relevant ADAS is activated.In such situations, the driver needs to prioritise their reaction promptly in relation to the driving task and situation based on the proper understanding of the warnings.This suggests that the future work on understanding the distraction and ADAS should also focus on multiple warnings.

ACKNOWLEDGEMENT
This research is supported by Bournemouth University match-funded studentship: My navigation system really works for me -Designing ubiquitous in-car interfaces.The authors would also like to thank all academic and student volunteers for taking part in the focus group to help them refine and validate the ontology.
The effects of a driver assistance system for safe speed and safe distance-a real-life field study.

Figure 2 :
Figure 2: Ontology of ADAS, driving task and driving distraction

Figure 4 :Figure 5 :
Figure 4: Example of driving distraction transition of single warning schematic diagram 4.2 Multi-warning situation

Figure 6 :
Figure 6: Example of` driving distraction transition of multi-warning schematic diagram

Table 1 :
Classes and definitions with references

Table 2 :
Parameters with assumed values

Table 3 :
Distraction parameters with assumed value

Table 4 :
Parameters with assumed value