Emotional Resonance : An Interactive Installation Controlled by Biometric Data

Emotional Resonance is an interactive installation that transforms humans’ pulse into motion and light reflections. As our pulse is influenced by emotion, and as a machinery reflection on our body signals creates a fascinating feedback loop, this work aims to explore emotions mediated by an interactive art installation. The project aims to foster the discourse on how biometric technology can cause emotions and consequently, how it connects us with a responsive machine or interactive art installation. The core questions, this work is provoking are (1) Can a machine, which is synchronized with our pulse create an emotional response? (2) How can an interactive machine or an art installation generate emotional resonance?


INTRODUCTION
Emotion is any mental experience with high intensity and high hedonic content, precisely, pleasure or displeasure (Cabanac 2002).Emotions are strong memory cues, and thus, art that is creating emotions will be better memorized (Grau 2000).Moreover, interaction is strengthening the immersion of a user experience, such as a cognitive involvement with media (Burt 1992).Consequently, creating emotion through art is a common strategy to get the attention and intense reaction of and on the art work, which is essential for the art work recognition and success.Artworks using bio-signals directly involve the viewer by going a step beyond interactivity through creating co-activity (Koch et al. 1990), which is a promising strategy to gain even more attention, emotional response, and memorability.
The pulse is widely accepted to be a linear and independent response to our emotions.A review of 134 studies made by Kreibig (2010) shows that the pulse changes during both, negative and positive emotions.Slovák et al. showed in a controlled laboratory studies that showing the heart rate can make people feel closer together (Slovák et al. 2012).The findings of this study let suggest that people tend to interpret (their own) pulse as emotion.In conclusion, Slovák et al. demonstrated that sharing heart rate or pulse frequency can be seen as a form of emotional self-disclosure.This works intends to provoke emotion resonance using biometric sensors so that the artwork is able to sense the pulse of the person touching it and then reacting accordingly and in synchrony with the pulse of the person.The measured pulse frequency is synchronized with water dropping at the same frequency to mimicry the signal of one being alive, the pulse.This transfer of life signals from a person to the installation results in an infinite resonance loop: the installation is mirroring the pulse of the person through water dropping in the pulse's frequency, which then is perceived by the person who may create empathy for the art piece.Through this loop, the art piece becomes alive when the person is touching it and stays alive as long the touch lasts.
Emotional Resonance allows the audience for observing their real-time biometric data creating responsive content.A feedback loop is created, consisting of pulse which is displayed as water and light movements, which is then observed and might influence the pulse through an emotional reaction on the art experience.
Emotional Resonance has been exhibited twice, and in this paper, we share impressions collected during these exhibitions.We will describe observations of the interaction created between the installation and people.After discussing our observations, we will also conclude how Emotional Resonance extends the current discourse on bio-art causing emotion resonance between art and people and vice versa.

RELATED WORK
Nowadays, the development of different fields, such as engineering and medicine, allow us to better understand the meaning and implications of human bioelectrical signals.Moreover, we better understand the correlation between bio-signals and emotions.Finally, bio-sensors became affordable and microcontrollers that allow for easy use, which results in media artists working with them more often than ever before, e.g., incorporate physical sensors in objects to capture real-time data.Using bio-data allows for deeper explorations of the human being and enriches the spectrum of art.In art, biometric sensors are often used to collect bio-data to communicate with the viewer in real time.Such synchronization and mimicry of the biological insight allow the biofeedback to become a medium that triggers emotions through showing bio-data that is associated with emotions as well as through showing information that is causing emotions.The second case can allow for creating a feedback loop of sensing emotions, showing emotions, and influencing emotions, which then are again sensed.
A practical example, which well demonstrates a realtime synchronization of emotions, is an artwork based on the real time caption of the pulse.In Enactive Cinema (Tikka P. 2008), the public's heart rate is monitored and creates changes in the narrative, soundscape, and rhythm of the storyline of the movie.It is all based on how the spectator experiences the emotional dynamics between the movie characters.
Lozano-Hemmer's Pulse Room ( 2006) is an installation that is based on a heartbeat sensor, which provides evidence in how integrating biosensors in installations.As such, it allows for affecting the audience.The heartbeat is translated to light being shown in a room with a blinking ceiling, which creates a sentient atmosphere.The theme of sentiment and the use of biometrics, sensorium, and immersive space also contemplates the work Kiss of Lancel and Maat (2015).In this project, the action of a kiss becomes a matter of art.While kissing, the kisser's brain waves are measured and visualized in real time as streaming E.E.G. data.A typical private experience is transformed into an artwork embracing the emotions of viewers and participants.
Emotional Resonance contributes to the existing body of artwork through addressing emotion.It creates a shared experience, measures, decodes, and transforms heartbeat waves into water drops, and light reflections.This characteristic allows the machine to represent the personal and real-time rhythm of a person's pulse aiming for encouraging self-recognition and exploration is the emotional resonance between the interactive installation and the audience.

METHOD
Kim (2014) stated that "human express emotion based on five senses, but they also use overall functions of the body to communicate with one another.The feeling toward an artwork are communicated in a multi-dimensional way, and the process of sending and receiving biometric data with everything is repeated.[...] Accordingly, human body generates physical, intellectual and emotional rhythms."This is the major motivations using biometrics in this work.The ability to synchronize with people's heartbeat data creates a personal communication with their body.According to Wolf (2010), when we track our steps, breath, heartbeat, acceleration, and even emotion, when "we quantify ourselves, there isn't the imperative to see through our daily existence into a truth buried at a deeper level.Instead, the self of our most trivial thoughts and actions, the self that, without technical help, we might barely notice or recall, is understood as the self we ought to get to know".As we do not have a pedometer in our feet or a breathalyzer in our lungs, Wolf says that we need machines to help us take stock of ourselves, to know who we are.
The work presented here is using sensors enabling to quantify ourselves or, in our case, the exhibition's audience.Through that, we will be able to collect impressions and make observations of the co-action created between the installation and the people perceiving the artwork.
Emotional Resonance was developed with the intention of creating an emotional dialogue between the participants and the work.It was designed to synchronize with the heartbeat.
When entering the space where Emotional Resonance is exhibited, two enlightened and almost empty bowls of water are presented (see Fig. 2).The water in the bowls creates abstract light shapes on the flour.This setup, which is consisting out of water, light, and reflections, starts to become alive when a person touches one of the two spheres that are placed in front if each bowl (see Fig. 3).Pulse sensors, that are embedded into the spheres, sense the pulse frequency of a person and send this frequency to an Arduino that controls the water drop frequency.The dropping water creates a subtle choreography out of drop sounds, water waves, and light animated by the moving water.Every bowl has the same mechanism and works on its own, but the meaning of having two of them creates interaction between them, not physically, but through their reflections on the ground.That animation draws a light projection on the ground that moves synchronously to the visitor's pulse.
The synchronization of the heartbeat with visuals and light enables people to see a direct correspondence of their bio-signals in the artwork aiming to trigger another layer of experience, the emotional resonance between the artwork and the audience.
To explore if a machine, which is synchronized with our pulse can create an emotional response and how an interactive machine or an art installation can generate emotional resonance, we exhibited Emotional Resonance and conducted interviews through asking open questions to the audience.

RESULTS & DISCUSSION
During two exhibitions, the first in Berlin and the second lasting for two and a half months at Biennale Internazionale Donna in Italy, about 600 people saw Emotional Resonance.When observing them, we recognize behaviour patterns, which we describe here, and we also recorded their experience in open questioned interviews.
During the exhibitions of Emotional Resonance, we were surprised by the complex behaviour of the installation in practice.We noticed repetitive coaction patterns and recognized longer than expected periods of user interaction.
When moving forward to the installation, the familiar shape of the sphere in front of the participants made clear how to interact with the remote control.However, the first participants mostly read the instructions before using it.Visitors that approached the work later just observed the interacting visitors to then copy their action.
We made some observation is about the age and interest of the public expressed by the comments given and by our observation.The young public's curiosity was noticeable.Families with children were completely amused by the bio-sensing interaction and convinced every family member, even kids, to experience the artwork.(see Fig. 3) Young adults responded really well.They often came back after some time to try the experience again or with other friends explaining in detail the concept and the instructions of the interaction.
Most visitors stayed longer than just the time of their own interaction watching what other's experience was looking like.
We collected some impressions and opinions from the audience, which we share in the following section.
"Other than the originality, what really moved me is how interaction can make you one with the installation, entering deeper in yourself.Hearing your heartbeat creates a strong emotion and complicity with the partner next to you.Complicity makes the installation other than engaging, amusing."Andrea C. (26 years old) The public aged over 40 years was more aware of the experience, reading twice the instructions and was often just looking at others interacting without trying by themselves.More questions about the medical data were asked, and comments about the stress release were made.
"It is like a 'Natural' heart rate monitor.It can be useful in yoga schools to help you to concentrate on your breath or in waiting areas of hospitals for helping you not to stress" Rada T. (62 years old) Another observation was made regarding the extended periods of visitor's interaction.Our impression was that the work created entrainment for the majority of visitors.Many people spent more time than expected with the installation, which ranged from 90 up to 120 seconds on average.The interplay and discussion between the public deserve a highlight in this matter.The public often started to discourse about the and to share the experience they just had.The desire to comment on the experience made, was often explored in both exhibitions.
Our observations show that real-time feedback of biometric data can, beside connecting emotionally with the artwork, enrich self-recognition and supports creating emotions connected to art and the interaction with it.This might be testimony to the intrinsic richness and complexity of working with biosensor data in art installations.

CONCLUSION
The recent development in engineering and medicine allows us to use human bioelectrical signals in the digital arts.Incorporating a physical sensor into interactive objects has become easy and an increasing number of biometric art pieces has been developed in the past few years.These works open a door for deeper explorations of the human being and the potential of art and machines in the matter of emotions.
Emotional Resonance is aiming at creating emotions in a person that engages with an interactive installation through synching with their bio-signals.Synchrony is created through transforming the rhythm of the person's pulse into water dropping and light reflections.Through such synchrony, Emotional Resonance creates an invisible bond between people and the art piece in front of them in the moment people start interacting with the work.
The synchronization of the heartbeat with visuals and light enabled people to see a direct correspondence of their life in the artwork and to unlock the second layer of experience, the emotional resonance within the artwork and the user as well as between individuals who interact with each other.Emotional Resonance has been used as medium to share sensations and to see their resonance in the space.
Emotional Resonance is extending the body of work on emotional art through combining art and qualitative research practices to better understand how art, in this case emotional art, is perceived by its audience.
We showed that a responsive installation engages people, and that engaged people again motivate other observers to get the same experience.The reaction on the interactive art varied from rational reflection about the kind of signals and for what application they could be used for to an intuitive and emotional interaction between the artwork and the audience.If people came in groups, the installation also triggered some group dynamics and fostered a social experience.In summary, the perception of the art, and especially the reflection about vary across people and depend on individual aspects, such as age or social context, but also personality plays an important role.
The findings indicate that using biometric data increases entertainment and focus of the audience, and that combined forms of interaction including biosensor foster an enthusiastic participation with the public.Real-time feedback, which is communicated through an intuitive connection of body and machine, causes instant interest.
We believe that such physiological-social technologies form a promising possibility for future interactive artworks, and we hope that this work contributes as a step towards unpacking the sentient art space.In particular, this research aims at providing a basis for a discussion of implications and future directions for engaging with interactive art.

Figure 1 :Figure 2 :Figure 3 :
Figure 1: Emotional Resonance is a kinetic installation that responds to the pulse of people

Figure 3 :
Figure 3: A family interacting with the installation