Personal state and emotion monitoring by wearable computing and machine learning

One of the major scientific undertakings over the past few years has been exploring the interaction between humans and machines in mobile environments. Research projects have brought forth various possibilities of interaction between humans and computers. Our daily lives are becoming more and more pervasive: we now have smart devices with high computational power. Until now, there is no smart device which has the capability of determining whether the user's heart beat is normal or not while also taking the emotion states into account. To find out whether the user's heart beat is normal or not; the user's physical activities and emotion states have to be identified. Recognizing human physical activities with a body worn sensor is not a new field in research, a lot of the research has been done in this area. We can identify the user's physical movements using different techniques like body movement suit [12], we do have other research project where researchers identify the users' physical activities using some sensors like [1,2,3,4,5,6]. We can identify the user's emotion states as described in [7,11]. I want to develop a physical activity and emotion state recognition system that should be able to identify the physical activities (Sitting, Standing, Walking, Running, Laying, Climbing stairs, Cycling, Strength training) and the emotion states(Grief, Excited, Tension, Frustration, Stress). 2. PROBLEM STATEMENT

One of the major scientific undertakings over the past few years has been exploring the interaction between humans and machines in mobile environments.Research projects have brought forth various possibilities of interaction between humans and computers.Our daily lives are becoming more and more pervasive: we now have smart devices with high computational power.Until now, there is no smart device which has the capability of determining whether the user's heart beat is normal or not while also taking the emotion states into account.To find out whether the user's heart beat is normal or not; the user's physical activities and emotion states have to be identified.Recognizing human physical activities with a body worn sensor is not a new field in research, a lot of the research has been done in this area.We can identify the user's physical movements using different techniques like body movement suit [12], we do have other research project where researchers identify the users' physical activities using some sensors like [1,2,3,4,5,6].We can identify the user's emotion states as described in [7,11].I want to develop a physical activity and emotion state recognition system that should be able to identify the physical activities (Sitting, Standing, Walking, Running, Laying, Climbing stairs, Cycling, Strength training) and the emotion states(Grief, Excited, Tension, Frustration, Stress).

PROBLEM STATEMENT
There is a project named Vital Signs[9]; the user has to attach a heart beat sensor on his body.The sensor connects with a user's PDA via Bluetooth.The PDA sends the heart beat level to the monitoring server.Communication between PDA and the intelligent system is done using GPRS.A monitoring systems keeps the record of the user's heart beat level[9].However, we cannot rely on this system.How would the system figure out whether the user is in normal condition or not?For example, the user would have a different heart beat level, if he is running; and have a different heart beat level, if he is sitting or standing.The user also would have a different heart beat level, if he is frustrated or stressed and have a different heart beat level, if he is happy or sad.It also depends on the user's health condition, age, gender and may be other things [10].The monitoring system(vital signs) does not take these factors into account.

Figure 1: Overall architecture
As a result, we can identify the following questions: What is the meaning of heartbeat?What heart beat is normal?When is it normal?For whom is it normal?I will attempt to develop a system which should be able to identify the user's physical activities and emotion states, I will also take the user's physiological data into account, based on this information, I will try to determine whether heart beat is normal or not.

STATE OF THE ART
I am dealing with the user's physical and emotional activities in my research.I need to know, which activity matters and which does not?In the following, I will explain the terms "physical activities" and "emotion states"

Physical activities
Everyone has his own body structure and daily routines.We need to know, what the user is doing, what kind of physical activity is he being involved in?I am interested to figure out the user's physical activities to match it with the emotion states.A lot of research has been done for the recognition of physical activities using wearable sensor.Researchers have figured out several physical activities using wearable sensors like sitting [1,4,5,6], standing [1,4,5,6], laying [6], walking [1,2,3,4,5,6], climbing stairs [1,2,4,5,6], running [3,5,6], cycling [3,6], strength-training [6] and so on.

Emotion states
I am also interested to figure out the user's emotion states in my research in order to know whether the user's heart beat is normal or not?Researchers have identified some emotion states using sensors [7,11].In paper [7], a Bluetooth enabled skin conductance device is made as shown in Figure : 2 This skin conductance sensor was employed in gathering data in addition to the "blood volume pulse", "electromyogram" and "respiration".The researchers used the information obtained from the aforementioned techniques to mark the physical activities neutral, anger, hate, grief, love, romantic, joy and reverence emotion states [8].

PLANNED RESEARCH
To know the user's physical activities, emotional states, physiological data and user's personal data, determines whether the user's heart beat is normal or not.
Influence of physical activity on the heart beat level?Influence of emotion state on the heart beat level?Influence of physiological data on the heart beat level?Influence of personal data like age, gender, height etc.
Firstly, I will attempt to develop a system for the needed physical activities using some sensors.It will be an empirical research; I will conduct some experiments and do the analysis.I could use the equivital system[10], but it identifies only three physical states, which are "stationary", "Low" and "High".Secondly, I will attempt to develop a system for the needed emotion states using physiological sensors like EMG (electromyogram), blood volume pulse, respiration and skin conductance.I will be working on the equivital system, since it contains a blood volume pulse sensor, body temperature sensor and the respiration sensor as shown in Figure : 3. I will integrate the EMG sensor separately.

Figure 3: Equivital device [14]
I need to implement a computer game/application which will rely on the psychological experts for the evaluation.I will integrate some "critical incidents" into the game.Later, I will conduct the experiments and do the analysis in order to identify the user's heart beat level.