Food & Mood: Explorations in Technological Intervention

Over-eating is often the result of consuming food for reasons other than needing fuel and nutrients: we eat for pleasure, from habit, for emotional support. Such non-homeostatic eating is an established contributor to overeating towards overweight or obese weight gain that in turn is correlated with numerous non-communicable diseases such as Type II Diabetes Miletus. Traditionally, cognitive behavioural therapy (CBT) and interpersonal Therapy (IPT) approaches help people develop strategies to change these eating responses, but they rely on a person deliberately reaching for those strategies: there has not been a way to provide just-in-time support: emotional eating experience triggers may not happen on a schedule. Our approach is to explore whether and how persistent sensor data may help us determine moments appropriate for intervention.


INTRODUCTION
We eat not just because we are hungry and craving nutrients (homeostatic/physiological eating), but for a host of emotional and habitual reasons as well.Such eating beyond our requirements for energy and nutrients is often a contributor to levels of weight gain that can move us towards noncommunicable disease conditions such as Type 2 Diabetes Miletus, metabolic syndrome and cardiovascular disease 1 .There is no single term that encompasses the combination of lifestyle, hedonic, emotional or habitual over-eating that leads to obesity.For this paper we will use the term "emotional eating" as a placeholder for this range of non-homeostatic eating behaviours.Reducing the amount of emotional eating has also been shown to have a positive impact on mitigating these disease conditions (Hilbert et al., 2012).Models of behaviour change, such as transtheortic psychology's Stages of Change model suggest that people must be interested not only in changing a behaviour, but ready to make a change before any kind of intervention can be helpful (Norcross, Krebs, & Prochaska, 2010).Even with readiness to change, it is common to experience setbacks: 1 "What are Common Health Consequences of Obesity and Overweight" Obesity and Overweight, World Health Organization Fact Sheet No. 311,May 2012 http://who.int/mediacentre/factsheets/fs311/en/index.htmlcombined stress and habit -two frequent attributes of emotional eating -are well known to make behaviour change more difficult, particularly depending on the degree to which one is interested in changing (Norcross, Krebs, & Prochaska, 2010).Evidence based treatment approaches 2 to behaviour change such as Cognitive behavioural therapy (CBT) and Interpersonal Therapy (IPT) both suggest that providing support for behavioural change, in preparation for these moments, can be a powerful assist in this change process (Hilbert et al., 2012).A hypothesis in this space is to suggest that providing support as close to a moment of crisis -or just before it -may have particular value to help a person support change.A challenge for human-based intervention is that it is very difficult and potentially very costly to be sufficiently present, aware, and skilled to offer the right kind of support to help the person get through that crisis moment.In this project we are how technology may get interventions delivered closer to these moments.

Persuasive technology is an area of Human
Computer Interaction focusing on the design of technologies to support behaviour change, particularly around health practices such as diet and fitness from making better eating choices to encouraging a more active lifestyle.These systems have for the most part relied on fixed contexts to offer support: alarms at particular times of day; showing performance updates when one glances at their mobile phone (Consolvo et al, 2008) providing information when a person actively requests support (Chi et al., 2008;Mankoff et al., 2002), etc.In emotional eating situations, we are already reaching for food as our support.An effective persuasive technology system may be most effective if it seems to intervene proactively more closely to the crisis moment -or just before it -to help disrupt this action chain.Detecting that pivotal moment for intervention, however, is a significant challenge.Our approach is to see if, by extending our previous work in emotion detection (McDuff et al., 2012) we can use physiological sensor data to detect a pattern or signature that would be able to cue a system for when to offer a just-in-time intervention for someone about to engage in emotional eating.In other words we have two key goals: first, to explore if such emotional eating trigger patterns are detectable by the means we have, and second, does timing an intervention to connect with these moments provide potentially more effective, enduring change.
We are at the early stages of this work.In this paper, therefore, we introduce our explorations in state detection for behavioural change around emotional eating.We provide an overview of the background theory and related work informing our approach to emotional eating.We also touch on related systems within this space.Finally, we describe our design methodology, apparatus for exploration and baseline results to date.

Eating for Non-Homeostatic Reasons
There are many theories around why we eat.One is largely homeostatic: we eat because we need fuel to survive; we crave certain foods because we require certain nutrients to function.Other reasons we eat, however, are more nuanced, though these too, are entwined in physiological responses.For instance, many of us eat when we are stressed.Though we are not hungry, we reach often for calorically dense foods, like a donut.Given a choice, when stressed, we tend to pick the donut over a salad.Susan Roberts's research team characterises these responses as "instinctive" in that at some point these behaviours may have served a survival function (Gilhooly et al., 2005).For instance, stress releases hormones that trigger a fight or flight response.Grabbing high energy available foods (like the sugary donut) is smart from a must have the fast fuel for energy to run away from the tiger perspective.Likewise when confronted with the variety of a buffet, it seems a scarcity mentality kicks in, and we load up.The challenge is that in a food abundant culture, these instinctive responses are no longer helpful.In fact, we see them lead to harm, such as towards noncommunicable diseases (NCDs).
An important insight around over-eating behaviour from both hormonal studies, cognitive behavioural therapy (CBT) and Interpersonal psychotherapy (IPT) is that non-homeostatic eating patterns can be re-educated (Hilbert et al., 2012).In other words, both our physiological responses (the release of ghrelin, say, to cue stomach grumbling at particular times; Davis et al., 2011) and psychosocial responses (eat in the presence of food or in response to stress) are malleable.A key to changing or re-educating these patterns is: identifying the triggers for a particular emotional eating response, and having strategies in place to address those triggers, including providing support for acting on those strategies when needed.An example of such a strategy, drawn from the Beck Diet (Beck, 2007), a popular CBT approach to diet change, including addressing emotional eating, is to have an "Advantages List" available to remind the person of reasons they are engaged in a particular change; likewise to have alternatives to the usual Emotional Eating food available -such as going for a walk or having fruit rather than cake.
Our approach in particular will be to explore detecting the (pre) crisis/trigger moment to see if offering that agreed strategy at that time is more effective than current state of the art, which largely leaves the person to their own best devices to operationalize their change strategies.This just-intime approach is a key differentiator in our work

Persuasive Health Technologies
Persuasive technologies for health and weight management have become very prominent over the past few years, with off-the-shelf fitness technology like FitBit (2012) and BodyBugg (Apex Fitness, 2012) making logging of activity and physiological state more accessible.These fitness gadgets are designed in particular to help people measure physical activity on one end and sleep quality on the other to motivate better movement and recovery.Ecosystems of such devices are growing so that the Fitbit pedometer, for example, works with its WiFi scale, so that weight fluctuations, rest quality, and step counts can be associated.With the availability of many different sensing technologies, Amft and Troster (2009) discussed on-body sensing solutions for monitoring food consumption.The sensing technologies in their proposed system include monitoring chewing, swallowing, food intake gestures, cardiac response after food intake, and so on.Such sensing could be added to systems like FitBit to begin to associate frequency and duration of eating with weight/sleep fluctuations to help motivate, for instance, longer meals and less frequent (emotional) chomping.
© The Authors.Published by the British Information Society Ltd. Proceedings of HCI 2012.Purpura et al. (2011) also proposed a similar, yet fictional, persuasive system.The technology in this system included, to name a few parts, an earpiece for monitoring chewing and swallowing; augmented reality glasses for capturing foods consumed; and heart rate recording for sensing exercise.This proposed system also connected through a mobile phone application in order to process the data and more importantly, in order for the person to receive feedback.The feedback --whether from a social network, a close friend, or pre-recorded messages --serves as a health intervention to encourage the person to be more active or consume less food.
Not at all persuasive systems need to be as elaborate as those proposed by Amfst and Troster (2009) and Purpura et al. (2011).In fact, a persuasive system with peer support alone (Maitland & Chalmers, 2011) can help people with weight loss.For example, Mutsaddi and Connelly ( 2012) utilized text messages as a form of social support for encouraging more physical activity.They found that even after the novelty of text messages wore off, the text messages were still beneficial because they served as reminders to participants to be more active.

PROJECT OVERVIEW
While persuasive technologies exist for monitoring physical activity and nutrition, it is difficult to provide support for emotional eating that could be timed for an optimal moment to disrupt the activity.Our hypothesis is that by offering "just in time" interventions at the moment before the crisis will be more effective than current best practice: to manage as best one can to act on strategies one has developed for coping with emotional eating triggers.Two questions emerge from the hypothesis we wish to test: when is the best moment before an emotional eating event to intervene, and of course, how detect that moment?Our goal in this phase of the project is first to detect emotions that are associated with emotional eating and then to explore timing an effective intervention prior to that person turning to food.
The technology for emotion detection involves custom hardware and software, which is similar to our AffectAura system (McDuff et al., 2012).This system utilized multiple sensors and sources, including a webcam, Microsoft Kinect sensor, file logging activity, a calendar scraper, the Affective device for collecting electrodermal activity (EDA) and GPS.We then used machine learning classifiers to detect various emotional states using signals about posture, facial features, speech prosody, user activity, location, and EDA.Emotion detection was primarily based on the arousal/valance model of emotions (Boucsein, 1992;Lang, 1995;Larsen & Diener, 1992;Picard, 1997).Specifically, this model consists of two dimensions of emotion: arousal (i.e., intensity or excitement level) and valance (i.e., whether it is a positive or negative state).In the AffectAura system, the purpose of emotion detection was to create an emotional prosthetic that allowed users to reflect on their emotional states over long periods of time.However, in our current work, our goal is to not only detect and make users more aware of their emotions, but to also provide just-in-time support for the problem of emotional eating, in order to see if a technological intervention can facilitate behavioural changes.To this end, our goal for the new system will be to help us detect particular states that pre-sage emotional eating so that the system can offer various interventions when there is concern.
For our current work, the technology is similar to AffectAura, but it is designed to be more mobile in order to measure emotions throughout the day and without the user being tethered to their desk.Our system consists of sensors that are embedded into clothing for detection of heart rate, skin conductance, acceleration, and rotation.Specifically, users are wearing a custom EKG/EDA sensor board and conductive fabric that is embedded into clothing.These sensors communicate through a Windows 7 mobile phone application in order to process the sensor data in real-time through a machine learning classifier that predicts emotions that may be associated with a particular user's emotional eating patterns.Drawing on related work in persuasive technology support for healthy eating, when these states are detected, an intervention technique will be triggered through the mobile phone app.
The mobile phone app, EmoTree, includes a user interface that shows users their emotional state, asks them to report simple food intake information and to report their mood, and also provides just-intime support for emotional eating by triggering an intervention.In our current study, the intervention consists of a reminder for users to remember what we are calling a micro-goal, informed by Martha Beck's (2008) work on dietary change.Here, in keeping with CBT strategies to prepare for change, at the start of the study, we ask participants to develop a strategy they believe is highly doable ("ridiculously easy") even in the context of an emotional eating trigger.So, for example, a microgoal may be if one usually eats a full bag of popcorn when triggered, rather than commit to eating only half, eat all but ten popcorns in the bag.
For comparison, we have two study conditions in our study: the app-only conditions (without sensors, using only EmoTree) and in the full sensor+app condition.We will measure their affect pre-and post-study to see if EmoTree has introduced any changes to mood or to the self-report of emotional © The Authors.Published by the British Information Society Ltd. Proceedings of HCI 2012.
eating.We will also try to find "trigger" emotional moments in the sensor condition, towards just-intime interventions.This study is currently running, and we will report initial findings at the workshop.

CONCLUSION & FUTURE WORK
Emotional eating is a well-known contributor to NCDs like obesity, metabolic syndrome and related diseases.The overall benefits of addressing emotional eating for those at risk for weight-related NCDs is considerable.In this paper we have briefly outlined the particular attributes of emotional eating that make it both a desirable place to provide persuasive technology styled interventions, but also a challenging one.Our hypothesis is that by using sensors and software to see if we can detect and an emotional signature that indicates the onset of an emotional eating moment, we will be able to deliver support for change that may be more effective than current, more manual ad hoc strategies.To this end, we have also presented an overview of the system design and methodology that we are using to explore this space.