Infusing Mindfulness Capability into Pedagogical Agents

In this paper, a new vision for pedagogical agents is proposed. The pedagogical agents in the future should be able to identify the mindfulness state of the student and to remind the student to practice mindfulness for a short period before he/she continues with learning. For this purpose, this paper reviews different approaches to measuring mindfulness and identifies the weakness of the self-report measurement approach. In addition, the paper proposes to combine different physiological data types to identify the patterns of mindfulness state.


INTRODUCTION
Pedagogical agent is conceptualized as a virtual character that is employed in a technologyenhanced learning environment to serve specified learning goals (Veletsianos & Russell, 2014).The communication with pedagogical agents are ideally conducted through natural language and/or facial expressions.They may embody different roles in a learning setting, e.g., expert, motivator, mentor (Kim & Baylor, 2016), coaching, testing or demonstrating (Moreno, 2005) or even learning companion (Kim et al., 2007).A recent review in 2014 suggested that students may prefer the condition with pedagogical agents compared to the condition without pedagogical agents (Schroeder & Adesope, 2014).In addition, pedagogical agents may provide motivational benefits for some populations under specific conditions.However, with respect to affective impact, Schroeder and Adesope (2014) reported that the pedagogical agent may not be necessary for students to receive the affective benefits of the interaction.In this respect, other researchers and reviews (Veletsianos & Russell, 2014) also reported that results of reducing student's anxiety and frustration (e.g., by appearing welcoming and friendly) are often contradictory and mixed.
Mindfulness meditation is a kind of mental training.Thich Nhat Hanh, a Zen master, explains mindfulness as follows: "Mindfulness is the energy of being aware and awake to the present moment.It is the continuous practice of touching life deeply in every moment of daily life.To be mindful is to be truly alive, present and at one with those around you and with what you are doing.We bring our body and mind into harmony while we wash the dishes, drive the car or take our morning shower."(Thich- Nhat-Hanh, 2018).Similarly, in other literature, the technique of mindfulness is described: "Focus your entire attention on your incoming and outgoing breath.Try to sustain your attention there without distraction.If you get distracted, calmly return your attention to the breath and start again" (Smith & Novak, 2003).Mindfulness has been introduced in schools, e.g., in the USA1 , in UK2 in recent years.School students are trained to practice mindfulness.The effect of mindfulness has been demonstrated that it helps among other effects the concentration of school students and their calmness.It is therefore useful to remind a student when he/she is in a low mindfulness state, which may correlate with negative learning outcome.
How can we combine the benefits of pedagogical agents in cognitive tasks (that have been confirmed in many studies) and the benefits of mindfulness.One possible vision is deploying wearable devices that might control the mindfulness state of students and give them feedback as a reminder.

MEASURES OF MINDFULNESS
Among available measures, the self-report instrument is widely used in clinical and psychological studies of mindfulness.In a review of self-report instruments for mindfulness, Park et al. (2013) assessed and compared the properties of different self-reported instruments.The study reported that the 15-item Mindful Attention Awareness Scale (MAAS, Brown & Ryan, 2003) was evaluated by the most studies and had positive overall quality ratings for most of the psychometric properties reviewed.However, Park et al. (2013) also pointed to the lack of qualitative evaluations and accepted external referents to support construct validity of the self-report questionnaires.In addition, the self-report instrument may reveal two issues (Davison & Kaszniak, 2016).First, an affective state may increase in the early stages of the practice, because individuals become aware of this practice in their minds.Second, experience with only a short period in mindfulness practice may rely strongly on demand characteristics since the mental processes for internal attention could not be developed quickly.Baer et al. (2006) suggested including other mindfulness measure methods in addition to self-reports.Which other options are available to measure mindfulness?
Researchers agreed that the concept of mindfulness needs to be operationalized and psychometrically sound measures of mindfulness are necessary for understanding the nature of mindfulness and its mechanisms (Chiesa et al., 2011;Chambers et al., 2009).Electroencephalography (EEG) has been used as an alternative measure instrument to study mindfulness.Patterns of EEG activity for particular meditative states have been investigated.A commonly reported feature of meditation has been Theta and Alpha event-related synchronization (Fell et al., 2010), which are regarded as markers of internally directed attention processing (Shaw, 1996).Such synchronization has been observed across different meditation types, including mindfulness as well as transcendental meditation, which involves focused attention on an internallyvoiced mantra.Recently, Lomas et al. (2015) have reviewed EEG studies from 1996 to 2015 and focused on mindfulness meditation only.According to this review, mindfulness is associated with enhanced Alpha (in 67% of reviewed papers) and Theta power (in 58% of reviewed papers) as compared to an eyes closed resting state, and no consistent patterns were found with respect to Beta, Delta and Gamma bandwidths.This conclusion is in accordance with the reports of Fell et al. (2010) and Josipovic (2010) that suggested that EEG Alpha is regarded as one of the signatures of meditation (including mindfulness).However, in the review of Lomas et al. (2015), there are six of 18 studies regarding Alpha and eight of 19 studies regarding Theta, which reported decreased Alpha/Theta power during mindfulness or no significant differences between mindfulness in comparison to a resting state.It is worthy to note that, the reviewed studies did not consider the typical EEG activity of meditators and nonmeditators over longer time periods.
Beside EEG, another type of physiological signals, Electrocardiography (ECG), has also been deployed to measure mindfulness in research.Heart Rate Variability (HRV) is an ECG metric that shows the influence of the autonomous nervous system in response to the change of the internal/external environment.In healthy individuals, HRV appears to be large and complex, and it is reduced in a diseased or stressed state (Jeang, 2004).Measurements of HRV usually include variables for time domain and frequency domain analysis (Wang & Huang, 2012).The time domain analysis usually uses following variables: Standard Deviation Normal to Normal (SDNN) and square root of the mean sum of squared differences between adjacent R-R intervals (RMSSD), which report the activity of the cardiac system, Mean Heart Rate (MRT), and Physical Pressure Index (PSI) (Medicore, 2018).The frequency domain analysis usually uses following variables: Lower Frequency (LF) and higher Frequency (HF), which reflect sympathovagal balance of the autonomic nervous system (ANS), Very Low Frequency (VLF), Total Power (TP, which is a short-term estimate of the total power of power spectral density in the range of frequencies between 0.0-0.4Hz), Normalized LF (the ratio LF/(TP-VLF), and Normalized HF (the ratio HF/(TP-VLF).In addition to using different variables of HRV (e.g., Joo et al., 2010;Krygier et al., 2013) and combing EEG signals like ERP (Howells et al., 2013), other studies have been reported that meditation has impact on respiration rate.All conducted studies confirmed consistently that the respiration rate decreases during breathing meditation in experienced meditation practitioners (Lehrer et al., 1999) and novices after short-term instruction (Farb et al., 2013;Zeidan et al., 2015;Doll et al., 2016).This result has also been replicated in several smallscale studies (Grant & Rainville, 2009;Kodituwakku et al., 2012) and studies with long-term mindfulness training (Wielgosz et al., 2016;Lazar et al., 2005).
Recent studies investigating mindfulness employed functional near infrared spectroscopy (fNIRS) to evaluate hemodynamic changes.The first fNIRSbased study investigated (de)oxy-hemoglobin changes with a single wavelength over the left prefrontal cortex during Qigong meditation (Cheng et al., 2010).Practitioners showed decrease in deoxy-hemoglobin (HbR) and increase in oxyhemoglobin (HbO) concentration.This result suggests that meditation leads to left prefrontal activation during meditation.While Cheng et al. (2010) investigated on the left prefrontal cortex, Deepeshwar et al.
(2014) investigated hemodynamic changes on the right prefrontal cortex using fNIRS.The authors found that during meditation there was an increase in HbO and total hemoglobin (THC) concentration with reduced HbR concentration over the right prefrontal cortex, where in random thinking there was increased HbR with reduced THC on the same cortex.The study concluded that meditation increased cerebral oxygenation and enhanced performance.Since fNIRS-based research on mindfulness using brain hemodynamic and total hemoglobin changes is still in its infancy, there is very little empirical evidence.
Table 1 summarizes studies of different metrics, which showed correlations with mindfulness practice.
In the context of connection between meditation and activation is specific brain areas, there are evidences that attention regulation is associated with anterior cingulate cortex; body awareness with insula and temporo-parietal junction; reappraisal (emotion regulation) with dorsal prefrontal cortex (PFC); exposure, extinction and reconsolidation (emotion regulation) with ventromedial PFC, hippocampus, amygdala; change in perspective on the self with medial PFC, posterior cingulate cortex, insula, temporo-parietal junction (Hölzel et al., 2011).Engen and Singer (2016) found that compassion meditation is associated with medial prefrontal regions.These results can be partly confirmed by the recent study using fNIRS.Gundel et al. (2016) found differences in hemodynamic responses between meditation experts and a control group in a mindfulness condition.Compared to the control group, meditation experts showed an increase of activation in the higher auditory areas, while the control group showed a decrease of activation in the same areas.In addition, meditation expert had highly activated brain areas parietal cortex and frontal cortex.
The studies using physiological signals for measuring mindfulness raise several open issues.First, many different HRV variables, ranging from the list of variables used in the study of Joo et al. (2010) to the study of Krygier et al. (2013) and the study of Howells et al. (2014), could be relevant for mindfulness and which ones are representatives for mindfulness, is unclear.Second, the results of existing studies using HRV variables are not consistent and reveal the need for more extensive research with different sample sizes and demographic of the participants.Third, we can also note that most reviewed ECG, EEG, and fNIRS studies have been conducted isolatedly except the study reported by Howells et al. (2014).That is, associations between ECG, EEG, and fNIRS metrics have not been researched extensively.In studies related to affective computing, for example, Dobbins and Fairclough (2017) investigated the cardiovascular impact of anger during driving on real roads.They use HRV and Pulse transit time (PTT) calculated from the raw ECG and (Photoplethysmography) PPG signals to measure anger.Thus, researching mindfulness by combining different physiological signals, rather than evaluating a single signal in isolation, may give us more insight into the mechanisms behind mindfulness.This is a challenge for operationalizing and classifying mindfulness.

A VISION FOR PEDAGOGICAL AGENTS
The aim for infusing the capability of mindfulness into pedagogical agents is not only to help a student solve a cognitive task, but also to remind him/her about the student's mindfulness state.This can happen as follows.While the student is solving a cognitive task with the support of the pedagogical agent, the pedagogical agent controls his/her mindfulness state.If the student's mindfulness state is below a threshold, at this moment, the pedagogical agent gives a feedback about his/her mindfulness state.The student should now practice mindfulness for about five minutes by adopting one of appropriate techniques, e.g.mindful breathing or mindful listening to the sound of a bell.The pedagogical agent would stay back and may repeat its feedback after appropriate time.How long the period for repeating the mindfulness feedback should be, it is an open research question.
Figure 1 sketches a possible architecture for a pedagogical agent that aims at reminding mindfulness of students while solving a task.The first layer of the architecture consists of components that process physiological data and model mindfulness states.The mindfulness expert model represents typical physiological data of experienced mindfulness meditators.Based on that expert model, we may differentiate between different levels of mindfulness.For identifying typical physiological patterns of expert mindfulness meditators, we need to extract features from physiological data of the expert group.By combining ECG, EEG, fNIRS, and respiration data, a big number of metrics need to be considered.As a consequence, a plurality of data correlations will arise.This is a kind of big data issue.It is required to extract the features that could represent mindfulness.For this purpose, we could compare the Principal Component Analysis (PCA) and Factor Analysis techniques to reduce the number of irrelevant metrics, which are among the most useful techniques for dimensionality reduction (Sorzano et al., 2014).The correlation model (PCA or factor analysis) with higher validity can be considered represent typical physiological metrics for the group of expert mindfulness meditators.Based on the mindfulness expert model, we operationalize the continuum between expert mindfulness level and non-mindfulness level and specify thresholds for different levels.These thresholds are deployed by a classifier to determine the mindfulness level of each individual student.
Using the thresholds for mindfulness levels, the Mindfulness Diagnoser can be developed to infer whether the mindfulness state of the student is below a threshold of low mindfulness level (or medium mindfulness level), which might be too "far away" from the expert model.The second layer represents the user model proposed by Mladenovic and colleagues (2017).According to the authors, a user model should capture stable characteristics, profile information and context dependent information of the user.The third layer of the architecture of the pedagogical agent consists of two typical components of an intelligent tutoring system: domain model and pedagogical model (Sottilare et al., 2014).In addition to these components, the communication model, which is also usually one of the four typical components of an intelligent tutoring system, is embedded in the fourth layer of the architecture.The domain model represents the tasks and the problem solving strategies for the tasks, which should be mastered by the student.The pedagogical model represents the tutoring strategies that can be used to adapt to individual students.This component might be connected with the dialog manager of the fourth layer of the architecture.
The fourth layer of the architecture is responsible for enabling the conversation between the student and the pedagogical agent in natural language.This layer consists of a pre-processing component, a natural language understanding component, a dialog manager, and a natural language generation component, which are typical components of a dialog system (Masche & Le, 2017).The preprocessing component usually deploys several natural language techniques (e.g., tokenizer, parsing) to transform the conversational input (a text) of the student into an interpretable form.After that, the natural language understanding component tries to interpret the pre-processed student's input.The dialog manager traces the state of the conversation between the pedagogical agent and the student and decides the next dialog move, which may be initiated by the student or the pedagogical agent.When the next dialog state has been decided, the natural language generation component generates an answer or a question for the student.

CONCLUSION AND RESEARCH DIRECTION
Picard (1997) defined affective computing "that relates to, arises from or deliberately influences emotions".Although mindfulness is not a type of emotion, rather an affective state, modeling mindfulness might also be considered a sub-topic of affective computing.While several pedagogical agents for identifying student's emotions and for adapting tutoring strategies based on student's emotions have been devised (e.g., AutoTutor for classifying the student's emotion (Graesser et al., 2004), MOODS for diagnosing the level of students' motivation (De Vicente, 2003)), pedagogical agents controlling the mindfulness state of students using physiological data have not been considered in research agenda until now.Thus, in the context of deploying capability of identifying the student's mindfulness state, one of new research directions for pedagogical agents may be using physiological data for determining the mindfulness state of the student.In relation to enhancing the capability of pedagogical agents with mindfulness feedback, operational mechanisms of mindfulness mediation have been found mostly from the psychological perspective using self-report questionnaires.
Looking for alternatives (e.g., patterns of physiological data) to self-report is worth exploring.

Figure 1 .
Figure 1.The architectural concept for a mindfulnessreminding pedagogical agent