Inherent Feature Connection (I-Con) Map for Liking Emotion Detection: an EEG Study

Achieving a good recognition and simulation of human emotion is becoming a hot topic in recent years. In this study, we exploited this problem in the liking emotion detection through EEG analysis. To improve the discriminant power of the extracted EEG features, a novel concept of inherent feature connection (I-Con) map was proposed, in which feature-based active channels were estimated and the hidden connections of channels in terms of features were uncovered. This method provided us an efficient and effective way to characterize EEG signals in a better way to reflect the emotion changes in the liking dimension. The performance of the proposed method was demonstrated on a well-known public emotion database.


INTRODUCTION
Human emotion is recognised as a wide research topic, covering many perspectives: psychology, neuroscience, health science, and engineering.In the past decades, an interdisciplinary research, combining biometrics, computer science and human emotion, has emerged and attracted great research attention.Various computation-based emotion recognition/detection systems have been proposed based on, for example, facial information [Cruz et al., 2014], acoustic parameters [Eyben et al., 2016], or fusion features of using both visual and audio modalities [Chen et al., 2017;Kim and Provost, 2017].
However, Affective characterization in brain signals could be considered as a more direct way, leading to a higher resolution to index the changes in human emotion.When monitoring cortical activities in real-time, EEG features extracted at specific brain regions, have potential to be connected with emotion dynamics.Koelstra et al. constructed a public emotion database (DEAP) with simultaneous recording EEG signals and applied it to an analysis of human affective states [Koelstra et al., 2012].This database provided researchers a platform to perform EEG-based studies for spontaneous affective state estimation and also presented a benchmark in the emotion state recognition using EEG modality.In this database collection, the participants reported their emotions in a fourdimensional space: valence, arousal, dominance and liking.Comparing to the other three emotions, liking emotion dimension, indicating how much the participants like or dislike the presented stimuli, is a subjective and complex psycho-physiological process, which is associated with various aspects such as experience, interpretation, education and other personal factors.With an attempt to unveil the complicated cognitive interactions underlying emotion responses, a number of studies have been conducted and evaluated on the DEAP database [Bahari and Janghorbani 2012;Naser and Saha, 2013;Wang and Shang, 2013;Li et al., 2015;Shahnaz et al., 2016].
In a typical EEG feature extraction method for liking emotion detection, EEG features are extracted from every channel and averaged over all the channels, or the EEG features are extracted from a predefined brain channel(s) based on the prior knowledge and heuristics.Then, supervised feature selection or dimension reduction approaches are commonly applied to select the optimal features that achieve the best performance in regression or classification.On the other hand, a more powerful method to improve the EEG feature discriminant could be: only extract the features from the featurebased active channels and then characterize the patterns that are identified as correlated with the changes in the liking emotion dimension.In this study, we proposed a novel EEG-based decoding pipeline (Fig. 1) to solve liking emotion detection problem, where the feature-based active channels were explored and an inherent feature connection (I-Con) map was generated correspondingly.Modelling in liking emotion detection was conducted on the formed I-Con features, and the performance were verified in terms of 10-fold cross validation procedure.

DATABASE
DEAP database [Koelstra et al., 2012] is a multimodal dataset for studying human affective states, involving total 32 participants and 40 trials for each participant.During one trial collection, 5s fixation was first conducted and then a music video with a length of 60s was played.After that, the participants gave subjective feedback on their emotions during video watching.By using the selfassessment manikins (SAM) system [Morris, 1995], the feedback on liking emotion dimension was given in the ranging of 1 to 9, where 9 indicated extremely like emotion and 1 referred to extremely dislike emotion.Simultaneously, EEG signals were recorded at a sampling rate (fs) of 512Hz from 32 active AgCl electrode sites, according to the international 10-20 system placement [Jasper, 1958].Further, the collected EEG signals were down sampled to 128Hz, pre-processed by applying blind source separation technique for EOG removal, and filtered by a band-pass filter with the range of [4.0 Hz, 45.0Hz].The preprocessed data provided by [Koelstra et al., 2012] was applied in this study.

Feature Extraction
We focused on EEG characteristics in the frequency-domain, which has been proven to be the most widely used features in the EEG decoding for emotion recognition [Jenke et al., 2014].

IAF calculation
Typically, EEG frequency-domain features are extracted under the fixed edges of frequency bands [Jenke et al., 2014].However, as stated in Klimesch's study [1999], it pointed out that the range of alpha frequency band would vary cross different ages.In other words, EEG rhythms would differ in the frequencies depending on the differences of participants and other brain dynamics.Considering the participants' ages involved in the DEAP database (27.19±4.45;from 19 to 37 years old) and to have a better study on the variability of EEG rhythms cross participants, the individual alpha frequency (IAF) was firstly calculated for each participant at each trial, and then applied to dynamically define the edges of frequency bands for individuals.The computation of IAF values was based on a center of gravity frequency-based approach, by calculating the weighted sum of spectral estimates divided by the sum of alpha power, as defined in Eq. ( 1) where was the average power spectrum at the frequency range of .Here, we adopted the average power spectrum of all the collected electrodes in fixation period for IAF calculation.The frequency interval was set to [7.5,12.5]and the frequency resolution was 0.25Hz.Thus, was 21 and { } was {7.50Hz, 7.75Hz, …, 12.5Hz}.Through calculating IAF values, the large interindividual difference in frequency bands can be well adjusted.The corresponding frequency bands were dynamically defined as shown in Table 1.

Frequency-domain feature extraction
The power spectral density estimation algorithm [Welch, 1967] was applied to compute the spectral power distribution using a Hamming window with 50% overlap.Subsequently, the average EEG power spectra in each individual-based frequency sub-bands (defined in Table 1) were extracted from each EEG channel.Furthermore, the peak frequency has been studied as another critical EEG characteristic in the frequency domain, which was proved to be related to memory capacity [Moran et al., 2010] and cognitive process [Haegens et al., 2014].We extracted four peak frequencies by searching the discrete frequency with the largest magnitude within theta , alpha -1, -2, -3 , beta -1, -2, -3 , and gamma -1, -2, -3 bands, which were termed as theta peak (ITF), alpha peak (PAF), beta peak (IBF), and gamma peak (IGF).In total, for each trial, the extracted EEG features were 14 features 32 channels.

Feature normalization
To reduce the inter-trial and inter-individual variance, the extracted features in the video period were also normalized by the features obtained in the fixation period, as (2) where referred to the EEG channels ( ), and indicated the feature types ( .was a very small constant (equal to ) to avoid the potential division by zero.

I-Con Map Computing
In this section, a novel methodology was proposed to efficiently search the feature-based active channel(s) under each feature .The active channel(s) could be interpreted as: feature at has a better reflection of the changes in the liking emotion, comparing to at the other channels .To find for each feature , we planned to compare brain dynamics at each channel under extreme like and dislike emotion states.If the feature at certain channels significantly reflected the changes from extreme dislike emotion to extreme like emotion states, then these channels could be considered as the -based active channels.The whole procedure of I-Con map computing was illustrated in Fig. 2, consisting of three steps: (1) template selection: searching brain dynamics under extreme like or dislike emotion states; (2) contrast computing: measuring the changes in brain dynamics between extreme like and dislike states; (3) I-Con map generation: detecting and at feature .

Template Selection
To find good templates for representing the brain dynamics under extreme emotion states, the trials with extreme high or low feedback scores were firstly selected.Suppose the subjective feedbacks on liking emotion was , where indicated the participants ( ) and referred to the trials ( ).For each participant , all the liking scores were denoted as , .We defined the trials whose scores were at the top 5% and bottom 5% ranks in as the extreme like and dislike states, respectively.The selected trial IDs were denoted as ̂ and ̂ , and the corresponding scores were ̂ and ̂ (the top 5% and bottom 5% values at the distribution of ).We treated the brain dynamics under the selected trials ̂ and ̂ as the templates and defined the features as: ̃ and ̃ , where and .

Contrast Computing
To eliminate the individual effect and find the general feature-based active channels, the selected templates from all the participants were firstly averaged as where and were the size of the selected templates from all the participants corresponding to the states of extreme like and dislikes, respectively.The obtained ̃ and ̃ could be considered as the general templates.The changes between ̃ and ̃ were measured in Eq. ( 4) to highlight the channels which had the most significant reflections along with the changes of liking emotion in terms of features.

I-Con Map Generation
To identify the relative active channels, the obtained was further binarized into 0 or 1 values using K-means clustering.The values cross all channels under feature (termed as ) were clustered into 2 classes.For the channels belong to the class with a larger value in , it was considered to have relative better reflection of the emotion changes in terms of feature .These channels were defined as active channels ( ) and assigned to 1.The remained channels in another class were defined as in-active channels ( ) and assigned to 0. Noted that the conditions here included , , and .The binarization result was called I-Con map (denoted as ), where ( ) and ( ) .had the same size as .The obtained I-Con map helped us to identify the feature-based active channels and improve the feature discriminant through forming the I-Con features below.

I-Con Feature Formation
The original extracted features were 14 features × 32 channels.Through the generated I-Con maps, we only retained the feature from the corresponding active channels , instead of from all the channels .The total extracted features  could be represented as ( is feature dimensionality, ). was defined as ‖ ( )‖ . (5) A classic feature extraction and data representation technique, principal component analysis (PCA) (Jolliffe, 1986), was applied here to project the features to a low-dimensional sub-space in which 95% variance of the data distribution was retained.
The procedure of projecting to ( is feature dimensionality, ) was illustrated in Algorithm 1.

RESULTS
The proposed I-Con map based liking emotion detection algorithm was evaluated on the DEAP database and compared with the existing literature.Similar to the other studies, all the liking scores were firstly grouped into two classes by using a threshold of 5: low ( ) and high ( ) classes.The performance of liking emotion detection was evaluated by two criteria as presented in [Koelstra et al., 2012]: accuracy and F1-Score .was defined as , where and were the number of correlatively predicted samples in two classes (lowand high-classes), respectively.and were the corresponding misclassified sample numbers to low-and high-classes.
measured the fraction of the corrected detected cases over the total detected cases, while was an overall performance of both precision and sensitivity, which was given by , where referred to the precision and indicated the sensitivity.was a real positive value that reflected the importance of precision and sensitivity in the computation.The larger the value , the more important the sensitivity was.Here, was set to 2 as the other studies.In the experiment, total 194 trials out of 1280 trials (32 participants x 40 trials) were selected as the templates for I-Con map computing.The original feature dimensionality 448 (14 features 32 channels) was reduced to 148 ( ).Based on the remained 1086 trials, the support vector machine (SVM) with RBF kernel ( ) was modelled to classify like emotion into two classes.Table 2 reported the 10-fold cross validation classification results based on DEAP database.The results showed that our proposed pipeline with I-Con map outperformed the existing methods, where and were equal to and , respectively.To evaluate the effect of I-Con map on the performance of liking emotion detection, we removed I-Con map computing from the proposed pipeline as a comparison, which was termed as "baseline method" in Table 2.In this method, after feature extraction in Section 3.1, the features were directly transformed by PCA with 95% variance retained and then modelled in SVM using same kernel parameter settings.The performance in liking emotion classification was and , respectively.The results evidenced the proposed I-Con map was beneficial for discriminant feature extraction in the application of liking emotion detection.

Table 2:
The performance on liking emotion detection.

CONCLUSION
In this paper, liking emotion detection was manifested via the proposed EEG-based decoding pipeline.This study proposed a concept of I-Con map and offered a new way to improve the discriminant power of EEG characteristics in various decoding applications.

ACKNOWLEDGE
This study was supported by the Project for Next-Generation Core Robotic and AI Technology Development (New Energy and Industrial Technology Development Organization [NEDO]).

Figure 2 :
Figure 2: The procedure of I-Con map computing.

Table 1 :
Definition of frequency bands with referencing to IAF values.