Designing self-monitoring technologies for emotional self-awareness and wellbeing

In order to sustain and improve our wellbeing we must be aware of our emotions and what triggers them. In my dissertation I propose to design and develop a self-monitoring mobile application that allows people to express and better understand their affective states, in a fun and engaging way. Through this tool I intend to explore how creative expression techniques, such as photography and painting, can be used to improve emotional self-awareness and to foster engagement.


INTRODUCTION
We live in a fast-paced, technology-dependent world where information and communication technologies (ICTs) are deeply ingrained in our lives, constantly competing for our attention and keeping us busy 24 hours a day, 7 days a week. This leads many of us to experience negative emotions in actual or anticipated interactions with technology -a disorder known as Technostress [24]. In the midst of this chaotic lifestyle, many of us neglect to take care of our wellbeing, leading to emotional exhaustion and other health problems. This adds up to an already heavy burden: an estimated 450 million people have a psychological health disorder, making mental illnesses one of the leading causes of sickness and disability worldwide [30]. Still, the benefits of technology are difficult to outweigh, since it makes our lives significantly more convenient, safe, fun and productive. It is sensible to anticipate that technology will continue to permeate our lives in the near future, so it is vital that we focus our efforts on exploring new ways to minimize its damages and invest on its potential for positive outcomes.
Fittingly, scientific research has recently emerged on how technology can be used to promote wellbeing and human potential. Positive Computing is a budding research field that envisions a future where all technology contributes to the optimal functioning and thriving of individuals and communities [4,6]. At the same time, more and more people are now using technology to collect data about themselves (e.g., physical activity, sleep patterns) in order to examine and improve their behaviours. This type of technology is known as self-monitoring, and is defined by Choe [8] as "technology that facilitates capturing of the occurrences of target behaviour and provides feedback to help people increase awareness and self-reflection.". Research has established that selfmonitoring increases awareness of the targeted behaviour and that it could be used as a simple and low-cost method to increase Emotional Self-Awareness (ESA) in order to prevent depression and other mental health problems [28]. ESA is also a key component of Emotional Intelligence (EI) and has numerous psychological and general health benefits. However, is important to ensure that individuals become more aware of their emotions in a productive way and limit negative self-focus such as rumination and worry [28]. In my project, I propose to build a self-monitoring mobile application that allows people to express and better understand their affective states, in order to improve their ESA and wellbeing.
Despite the growing number of applications and interventions aimed to improve mental and emotional health, the great majority lacks a solid foundation on psychology theories and scientific validation [5]. In order to operationalize my research and prevent this from happening, I studied the framework for research and practice proposed in the Positive Computing book [4]. I then selected two intrapersonal factors (regarding the self) from this framework: self-awareness and motivation & engagement. I believe that these two factors are vital for self-monitoring technologies. Improving (emotional) self-awareness is the main outcome, but in order to achieve it is important that individuals are motivated to do it in the first place. And, even though self-monitoring is usually a result of a personal motivation [15], many self-monitoring technologies present problems of attrition (low adherence and/or high drop-out rates) due to barriers like lack of time and motivation, or difficulty in data interpretation [9,27]. For each factor, the above-mentioned framework suggests literature & theory, strategies and methods & measures, which I will describe next.
Regarding self-awareness, the strategy will be to build a self-monitoring mobile application, based on ESA and EI literature. As measures I will use Emotional Intelligence measures and/or wellbeing measures. A need for further research regarding whether interventions to develop EI also increase wellbeing has been identified [5,19], so it could be interesting to assess and compare both measures.
Concerning motivation & engagement, I will explore the Self-determination Theory (Ryan & Deci) and the Flow Theory (Csikszentmihalyi). The suggested measures for this factor are The Motivation and Engagement Scale and the Self Regulation Questionnaires. Lastly, the strategy to address this factor will fall in the use of creative expression techniques, like painting and photography. For instance, painting can be characterized as an autotelic activityan activity that is intrinsically motivated and rewarding in itself, and can facilitate a flow experience. In the 1960s, Csikszentmihalyi found out that when the painting process was thriving, the painter would endure "single-mindedly, disregarding hunger, fatigue, and discomfort", a state known as flow (a subjective experience that puts an individual in a state of absorbing engagement [17]. Furthermore, I hypothesise that creative expression techniques can, not only improve the engagement with the technology, but also contribute to improve ESA. For instance, Art Therapy (where activities like drawing and painting are used to facilitate the expression of feelings) has been used for years in clinical practice [23]. Photographs, on the other hand, can be used as a way to access and explore memories and its accompanying emotions. Looking at any kind of photographic image produces perceptions and reactions that can reveal a lot about a person's self [29]. Emotions are complex and difficult to express, so these non-verbal techniques may also help with the expression of emotional states.

RESEARCH SITUATION
I am a third year student of the Digital Media PhD (part of the UT Austin | Portugal doctoral program) at the Universidade NOVA de Lisboa. In the first year I completed the required courses and worked towards defining my thesis topic. In the second year I got a full scholarship and could begin to focus entirely on the PhD. That year I designed and developed a working prototype and used it on an exploratory study [18]. I also attended a Doctoral Consortium at the NordiChi 2016 conference. During the third year I did a pilot study and began to design and implement the mobile version of the tool. I was also part of the organizing team of an international new media and digital arts festival 1 . In the next two years I intend to carry out the plan succinctly described in the present document. I look forward to participate in this Doctoral Consortium in order to share ideas and experiences with my colleagues, and to get insights into my work from an experienced panel.

RELATED WORK
In this section I provide a small overview of related work in self-monitoring technology research. In the future I plan to review more thoroughly these and other research projects for the self-monitoring of emotions, focusing on: capture mechanism, feedback mechanism, literature and theory, and validation studies.
All the projects described here use mobile phone applications where users can record everyday moments and its associated emotions. The users are then encouraged to reflect on the recordings, in order to improve their emotional self-awareness. Some systems collect only self-reported data, while others use data automatically collected through the mobile phone as well.
The Affective Diary [25,26] is a digital diary that uses sensors to automatically add reminiscences of bodily experiences. The tool provides the users with a semi-automated entry creation that incorporates abstract representations of mood and activity, data streams from the user's phone, and the user's handwritten journal entries, in order to foster personal reflection and learning. Echo [12] is an application that allows people to record everyday experiences and later revisit those recordings, in a practice coined technologymediated reflection (TMR). The records entail a label and an emotional reaction to that event (a simple scale from 1 -negative to 9 -positive). Optionally, the user can add a description of the event, photos, videos, and audio recordings. Mood Meter 2 is a project from The Yale Center for Emotional Intelligence whose goal is to improve emotional intelligence, according to the RULER approach [2]. When users record one emotion they 3 are encouraged to reflect on why they are feeling that way, and to write about it. The app also offers regulation strategies (e.g., it tells users to think about their favourite joke in order to cheer up) and reports where users can see the details of their emotions over time. So far, this is the only project I found specifically tailored to improve the emotional intelligence construct. In the Mobile Therapy [16] project a mobile application is used to register the user's emotional states and to offer coaching based on the well-evidenced practice of Cognitive Behavioural Therapy. Lastly, in EmotionSense [21] the main goal is to study the contextual factors (e.g., physical activity, sociability, sleep) that affect emotions, offering users basic feedback about the relations between the self-reported data on emotions and the data automatically collected form the cell phone.
Regarding the capture mechanism, Mobile Therapy, EmotionSense and Mood Meter base their emotion assessment interface on Russell's circumplex model of affect [22], which offers a visual representation where affective states can be organized in terms of two dimensions: valence and arousal, the most frequently used dimensions in affect research. The Affective Slider [1] and CAAT [7] are research projects that explore the visual self-report of emotions based on this model as well.
Projects such as Know Yourself [13] and E-Tree [10] blend art and emotions in order to bring emotional awareness to people. In the future I intend to look for more artistic projects, and explore them in order to obtain inspiration for the creative expression of the emotional states.

METHODOLOGY
In this section I describe what I have done (subsections 4.1 and 4.2) and what I intend to do next (remaining sub-sections), in order to complete my proposed thesis. Figure 2 show images of a web prototype I did during the second year of my PhD. The prototype was inspired by Plutchik's wheel of emotions [20] and its analogy with the colour wheel as I aimed to explore the concept of "painting with emotions". In the emotion assessment interface users can choose a brush and a colour (each colour representing an emotion) from the palette on the right, and then paint the photography on the left. They can also complement their entry by writing a short text (available on a different tab).

Figure 1 and
In a subsequent exploratory user study [11] I explored, along with my supervisors, how different colours or paints could represent emotions, and how the act of painting on a device could be used to express affective state. From this study resulted several insights that will be taken into account on the mobile version of the tool.

Pilot Study
Earlier this year we conducted a pilot study using the photo-sharing social media platform Instagram. This 4-week long study, with 25 participants, was designed to explore how emotion tracking can be incorporated into everyday digital photographic practices. We instructed the participants to assess their emotional states, according to a preestablished set of rules, while taking photographs.
In the end, we obtained 281 submissions, each composed by a photograph and textual data regarding the emotion assessment. In the next months we will interview some of the study's participants and analyse the data more rigorously.

Application Design and Development
I will design and develop a mobile application that will be used in the remaining research activities. I will take into account the lessons learned from the previous studies and from the relevant literature. The application will have a capture mechanism and a feedback mechanism (referred to as the Collection and Reflection stages in the Stage-Based Model of Personal Informatics Systems [14]), and will take into account five themes regarding ESA, as defined by Kauer

Field trial -Deployment study
We will perform a field trial to determine how users actually use the application, in the context of their daily lives. We will give participants a tutorial before the trial, and in the end we will conduct semistructured interviews to determine perceived usage and benefits.

Usability Study
After designing and developing the mobile application, a formal usability study will be conducted in order to assess its usability, and guarantee a proper user experience.

Validation of the emotion assessment interface
A study to validate the emotion assessment component will be performed. This study will be conducted in a controlled setting, where subjects will be asked to look at emotion-eliciting images and then report on their emotional reactions to the stimuli. The reactions will be reported using the component developed for emotion assessment and through other emotional assessment tool(s) that have already been proved (e.g. SAM [3]). Then, the study results will be compared in order to validate the self-reporting user interface.

Efficacy study
The last study will be a longitudinal efficacy study of the system (randomized controlled trial) where the effects of using the application (versus a simple mood tracker) will be tested, based on standardized wellbeing scales and engagement metrics. In this study we will use metrics to measure wellbeing, emotional intelligence and engagement. We will also try to uncover which mechanisms (e.g., painting to express emotions, reflecting on past submissions) contribute, and how, to the improvement of the measures.

CONCLUSION
I have presented the current state of my PhD work, whose goal is to make a sound contribution to the field of positive computing, by designing, developing, and running studies on a tool to improve emotional wellbeing, enhanced by creative expression techniques.