An Eye-gaze Oriented Context Based Interaction Paradigm Design

The human eye's state of motion and content of interest can express people's cognitive status based on their situations. When observing the surroundings, the human eyes make different eye movements to interact with the observed objects which reflects people’s attention and interest intentions. Currently, most of the eye-gaze interactions lack the context information from the environment. To investigate the cognition awareness of people when they are performing eye-gaze interactions to the surroundings, we analyse the composition of the environment, and divide the essential factors of it into interactive subject, i nteractive object and context. The eye-object movement attention model and the eye-object feature preference model are constructed to understand people’s attention and preference diversities through eye-gaze interaction to different interactive objects in different contexts, and furthermore to predict their behavioural intentions. Then, an eye-gaze oriented context based interaction paradigm is designed to explain the relationships among eye movement, eye-gaze interaction and people’s behavioural intentions when they are involved and performing eye-gaze interactions in different environments. The paradigm shows the eye-gaze interaction patterns and people’s cognitive behavioural intentions in different context based environments, which can dynamically adapt the intention prediction results to interact with multiple interfaces properly, such as game, PC system, social robots, HCI & HRI applications and serve as one of the computable modals of cognitive computing.


INTRODUCTION
Eye-gaze related research is a popular topic in the field of HCI that has been applied in many domains.However, there is not any existing theory to systematically describe the compositions of eyegaze interaction.In addition, there is no further analysis of user's cognitive intention based on eyegaze interaction in different contexts.In this paper, we present a method to design an eye-gaze oriented context based interaction paradigm, which describes the eye-gaze patterns of users while they are performing interactions to the environments with different contexts.There are five contributions of this paper,1) a classifier is trained to categorize the eye movements; 2) The compositions of eye-gaze interaction and the essential factors of interactive environment are defined; 3) The personal eye-object movement attention model and eye-object feature preference model are construed by the machine learning algorithms to predict the attention and preference object of people; 4) These two models are applied in the process of human intention prediction in real-time context to analyse the user's cognitive status.5) A paradigm is designed to conclude different eye-gaze interaction patterns in different contexts.The paradigm can adapt to different eye-gaze based multimodal interactions according to different contexts.In addition, it can reveal valuable information for human cognitive computing during interactions, and offer guideline to develop the HCI, HRI and NUI applications.

BACKGROUND
Eye-gaze interaction can improve the efficiency of HCI, and HRI especially [1] [2].In general, the eyegaze interaction has been applied in many domains.Zhao proposed classification methods of eye physical movement and related application fields [3].Microsoft in collaboration with Tobii, embedded eye tracking technology [4] into the latest Windows 10 to have full-scale and hand-free interaction.Dagmar Kern [5] helps users quickly revert to previous visual concerns by recorded gaze-mark.WADE [6] analyses the physical eye movements of autistic patient in a VR driving scene and correct their actions to help them improve their driving skills.In addition to exploring the relationship between eye movement and physiological status, the hidden psychological information of eye-gaze behaviours is also of great value.Vicente [7] analyzed the state of the eye movement to determine the concentration of the driver while driving.Zheng [9] improve the UI design of car navigation system by analysing the drivers eye movement information.M Borys [9] analysed the relationship between the eye movement status and the level of fluency in presentation based on classification methodologies.In the field of psychology, researchers can combine eye attention with other methods to study how the brain works and to analyse brain-controlled behavioural patterns in different states [10], such as eye attention differences among people for crafts, advertisements, and films.When people are in an interactive environment, they show different behavioural intentions based on different information presented by the environment [11].

Paradigm design
The approach of HCI paradigm design treats embodiment as the interaction property, and meaning construction as the central focus associated with people in interactive contexts and situations [13].People may show different cognitive behavioural intentions by in different interactive environments.To understand people's cognitive behavioural intentions based on eye-gaze interaction in different interactive environments and help to develop the eye-gaze interaction based applications.It's necessary to design a paradigm to systemically explain the relationships among eye movements, eye-gaze interactions and people's behavioural intentions when they are involved and performing eye-gaze interactions in different environments.More specifically, to categorize the eye movement, model the eye-gaze interaction pattern, predict the cognitive behavioural intentions of user and finally dynamically adapt the intention prediction results to interact with different environments properly.In the following sections, the main processes of the paradigm design are presented.

Eye movement classification
According to the eye motion features, the categories of eye movement can be divided as divergence, fixation, saccade and smooth pursuit, shown in Table 1.In previous work [12], we propose a method to train the eye movement classifier with Naive Bayes algorithm, which can be applied in the real-time eye movement recognition.

Attention model and preference model
To understand people's eye attention and preference diversities to different interactive objects in different contexts, the personal eye-object movement attention model and eye-object feature preference model are constructed by three supervised learning classification algorithms, including label propagation, SVM and decision tree.
For the construction of eye-object movement attention model, according to the result of kinematics analysis [13], the moving categories of the interactive objects can be categorized as still, linear, curvilinear and random, as described in the Table 2. Firstly, we collect the eye-gaze data for still, linear, curvilinear and random moving objects on the 2D screen.Next, the eye-gaze point coordinates as well as the moving object central point coordinates are treated as the learning features.At the same time, whether the object is under focused is treated as the learning label to train the eye-object movement attention model for specific moving category of object.Finally, we can predict whether eyes are paying attention to specific object by referring the corresponding eye-object movement attention model in real-time.
For the construction of eye-object feature preference model, the object feature is firstly defined, including colour, shape, and movement category of objects, shown in Table 3.The object feature attributes are treated as learning features.Whether user is paying attention to the specific object features as the learning label to train the eye-object feature preference models for different individuals.The models can be used to predict the user's preference object among the ones with different features.
Table 3 The object feature list

Shape The shape of the object Colour
The colour of the object Movement category The current movement category of the object

Design process
Eye-gaze interaction is a behaviour when eyes are performing movements to observe the surroundings, which is supposed to change with different feedbacks from the interactive systems.Currently, most of the eye-gaze interactions merely consider the relationship between people and the object in the interactive system but ignore the context information from the external environment.To further investigate the cognition awareness of people when they are performing eye-gaze interactions.we divide the interactive environment into three essential factors, including interactive subject, interactive object and the context.
In the paradigm, a classifier is firstly trained to categorize the eye movements.The eye movement are supposed to transfer with the feedbacks from the interactive systems to become the system based eye-gaze interaction.
When the eyes are being involved in an interactive environment, the personal eye-object movement attention model and eye-object feature preference model are applied to predict the attention object of user.Firstly, the eye-object movement attention model can be used to predict the attention moving objects of user.However, the user may be attracted by more than one moving objects at the same time.
The eye-object feature preference model can be used to predict the most preference object of user based on the prediction result of eye-object movement attention model by comparing the characterization features of each object.Once the attention object is predicted, the context information from the external environment can be referred to predict the cognitive behavioural intention of user, and drive the eye movements to perform different eye-gaze interactions based on the intention results.
The paradigm can adapt the intentions prediction results to interact with multiple interfaces properly based on the corresponding contexts, such as game, PC system and other HCI applications.Figure 1 shows the framework of the paradigm.
Two context based scenarios are designed to test the ability of real-time interaction adaptions of the paradigm.

Real-time cognitive intention predictioncursor-aided shooting game
In this real-time context, by analysing the object of attention of the player in the current time, the system helps the user to perform quick cursorassisted aiming to the locked object, users only need to operate the mouse or the handle to perform an easy quasi-lock.Figure 2 shows the procedures of this game.

Real-time cognitive intention predictionhand-free interaction for social robot
The social robot can predict the user's behavioural intention by analysing the user's eye-gaze information with the support of eye-gaze based object attention prediction module and refer to its prior knowledge library to generate the proper interactive feedback.When the user is busy doing things with both hands, he can communicate with the robot through his eye-gaze behaviours.For example, when the user is making Kungfu tea according to the tutorial displayed on the screen and cannot free his hands to reach the screen.At this moment, the robot can perceive the area of interest from the user's eye-gaze behaviour to help the user switch the instruction steps on the screen, providing the user with a very convenient and efficient interaction experience.The scene photos are showed in Figure 3.The interaction operation time of the traditional screen touch and the eyegaze behaviour interaction can be collected to compare the efficiencies between these eye-gazed based or no eye-gazed based interaction paradigms.

CONCLUSION
In this paper, we design a paradigm to model the eye-gaze interaction patterns, predict the cognitive behavioural intentions of user and finally adapt the intention prediction results to different interactive situations.The paradigm provides an alternative way of thinking in context based HCI especially in eye-based interaction, and can be extend to the eye-gaze based interaction design in multiple interfaces, such as game, PC system and other HCI applications.

Figure 1 :Figure 2 :
Figure 1: The framework of eye-gaze oriented context based interaction paradigm

Figure explanation: step 1 ,
Figure explanation: step 1, mark out the eye-gaze region around the gaze point.Step 2 & step 3, predict the user's attention object.Step 4, output the result of the attention object and move the cursor automatically to it.The procedure firstly predicts the player's attention object with the eyeobject movement attention model, then moves the shooting cursor to the corresponding object to complete the shooting.

Table 1
The categories of eye movement

Table 2
The object movement category list