3,632
views
0
recommends
+1 Recommend
1 collections
    4
    shares

      Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65

      scite_
       
      • Record: found
      • Abstract: found
      • Conference Proceedings: found
      Is Open Access

      Human Centric Facial Expression Recognition

      proceedings-article
      1 , 2 , 1
      Proceedings of the 32nd International BCS Human Computer Interaction Conference (HCI)
      Human Computer Interaction Conference
      4 - 6 July 2018
      Facial Expression Recognition, Emotion Recognition, Deep Learning, Convolutional Neural Network
      Bookmark

            Abstract

            Facial expression recognition (FER) is an area of active research, both in computer science and in behavioural science. Across these domains there is evidence to suggest that humans and machines find it easier to recognise certain emotions, for example happiness, in comparison to others. Recent behavioural studies have explored human perceptions of emotion further, by evaluating the relative contribution of features in the face when evaluating human sensitivity to emotion. It has been identified that certain facial regions have more salient features for certain expressions of emotion, especially when emotions are subtle in nature. For example, it is easier to detect fearful expressions when the eyes are expressive. Using this observation as a starting point for analysis, we similarly examine the effectiveness with which knowledge of facial feature saliency may be integrated into current approaches to automated FER. Specifically, we compare and evaluate the accuracy of ‘full-face’ versus upper and lower facial area convolutional neural network (CNN) modelling for emotion recognition in static images, and propose a human centric CNN hierarchy which uses regional image inputs to leverage current understanding of how humans recognise emotions across the face. Evaluations using the CK+ dataset demonstrate that our hierarchy can enhance classification accuracy in comparison to individual CNN architectures, achieving overall true positive classification in 93.3% of cases.

            Content

            Author and article information

            Contributors
            Conference
            July 2018
            July 2018
            : 1-12
            Affiliations
            [1 ] Faculty of Computer Science, University of Sunderland, Sunderland, SR1 3SD, UK
            [2 ] Faculty of Health, Sciences and Wellbeing, University of Sunderland, SR1 3QR, UK
            Article
            10.14236/ewic/HCI2018.44
            712f4246-3bfe-4070-b622-65b6ffb4b975
            © Clawson et al. Published by BCS Learning and Development Ltd. Proceedings of British HCI 2018. Belfast, UK.

            This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

            Proceedings of the 32nd International BCS Human Computer Interaction Conference
            HCI
            32
            Belfast, UK
            4 - 6 July 2018
            Electronic Workshops in Computing (eWiC)
            Human Computer Interaction Conference
            History
            Product

            1477-9358 BCS Learning & Development

            Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/HCI2018.44
            Self URI (journal page): https://ewic.bcs.org/
            Categories
            Electronic Workshops in Computing

            Applied computer science,Computer science,Security & Cryptology,Graphics & Multimedia design,General computer science,Human-computer-interaction
            Facial Expression Recognition,Emotion Recognition,Deep Learning,Convolutional Neural Network

            REFERENCES

            1. 1979 Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face Journal of personality and social psychology 37 11 2049 2058

            2. 2013 Emotion recognition: The role of featural and configural face information Quarterly Journal of Experimental Psychology 66 12 2426 2442 doi:10.1080/17470218.2013.789065

            3. 2000 Configural information in facial expression perception Journal of Experimental Psychology: Human Perception and Performance 26 2 527 551

            4. 2008 Facial expressions of emotion (KDEF): Identification under different display-duration conditions Behavior Research Methods 40 1 109 115 doi:10.3758/BRM.40.1.109

            5. 2009 Eye-movement assessment of the time course in facial expression recognition: Neurophysiological implications Cognitive, Affective and Behavioral Neuroscience 9 4 398 411 doi:10.3758/CABN.9.4.398

            6. 2016 Perceptual and affective mechanisms in facial expression recognition: An integrative review. [Article] Cognition and Emotion 30 6 1081 1106 doi:10.1080/02699931.2015.1049124

            7. 2012 „Multi-column Deep Neural Networks for Image Classification‟ February doi: 10.1109/CVPR.2012.6248110

            8. 2017 Automated Representation of Non-Emotional Expressivity to Facilitate Understanding of Facial Mobility: Preliminary Findings Intelligent Systems Conference 7th – 8th September London 2017 doi: 10.1109/IntelliSys.2017.8324218

            9. 2017 „Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units‟ Available at: http://arxiv.org/ftp/arxiv/papers/1706/1706.01509.pdf%0Ahttp://arxiv.org/abs/1706.01509

            10. 2014 Increased sensitivity to happy compared with fearful faces in a temporal two-interval forced-choice paradigm European Conference on Visual Perception, Belgrade, Serbia. Perception 43 1 75 doi: 10.1177/03010066140430S10

            11. 2015 Happiness is in the mouth of the beholder and fear in the eyes Vision Sciences Society, St. Petes Beach, Florida, US. Journal of Vision 15 1378 doi:10.1167/15.12.1378

            12. 2016 Evidence for a Face Inversion Effect in People with Parkinson‟s Perception 45 S2 295 doi: 10.1177/0301006616671273

            13. 1997 Featural evaluation, integration, and judgment of facial affect. Journal of Experimental Psychology: Human Perception and Performance 23 1 213

            14. 2015 „Development of deep learning-based facial expression recognition system‟ 2015 Frontiers of Computer Vision, FCV 2015 2 5 doi: 10.1109/FCV.2015.7103729

            15. 2008 „Preferred spatial frequencies for human face processing are associated with optimal class discrimination in the machine‟ PLoS ONE 3 7 1 5 doi: 10.1371/journal.pone.0002590

            16. 2016 „Hierarchical committee of deep convolutional neural networks for robust facial expression recognition‟ Journal on Multimodal User Interfaces. Springer Berlin Heidelberg 10 2 173 189 doi: 10.1007/s12193-015-0209-0

            17. Gradient-based Learn. Appl. Doc. Recognit. 86 11 1998 2278 2324 http://dx.doi.org/10.1109/5.726791

            18. 2016 „Facial Expression Recognition with CNN Ensemble‟ Proceedings - 2016 International Conference on Cyberworlds, CW 2016 163 166 doi: 10.1109/CW.2016.34

            19. 2017 „Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order‟ Pattern Recognition. Elsevier 61 610 628 doi: 10.1016/j.patcog.2016.07.026

            20. 2015 „A Facial Expression Recognition System Using Convolutional Networks‟ Brazilian Symposium of Computer Graphic and Image Processing, 2015–Octob 273 280 doi: 10.1109/SIBGRAPI.2015.14

            21. 2010 „The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression‟ 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010 May 94 101 doi: 10.1109/CVPRW.2010.5543262

            22. 1998 „Coding Facial Expressions with GaborWavelets‟ third IEEE Conference on Automatic Face and Gesture Recognition 200 205 doi: 10.1109/AFGR.1998.670949

            23. 2012 Processing orientation and emotion recognition. Emotion 12 1 39

            24. 2014 Judgments of subtle facial expressions of emotion Emotion 14 349 357 http://dx.doi.org/10.1037/a0035237

            25. 2015 „Going Deeper in Facial Expression Recognition using Deep Neural Networks‟ doi: 10.1109/WACV.2016.7477450

            26. 2016 „Understanding how deep neural networks learn face expressions‟ Proceedings of the International Joint Conference on Neural Networks, 2016–Octob 227 234 doi: 10.1109/IJCNN.2016.7727203

            27. 2004 Photographs of facial expression: Accuracy, response times, and ratings of intensity Behavior Research Methods, Instruments & Computers 36 634 638 http://dx.doi.org/10.3758/BF03206544

            28. 2016 „Facial Expression Recognition using Convolutional Neural Networks: State of the Art‟ Available at: http://arxiv.org/abs/1612.02903

            29. 2016 „Facial Expression Recognition with Convolutional Neural Networks‟ 1 8

            30. 2014 Recognizing dynamic facial expressions of emotion: Specificity and intensity effects in event-related brain potentials. Biological psychology 96 111 125

            31. 2017 „Automatic Facial Expression Recognition Based on a Deep Convolutional-Neural-Network Structure‟ IEEE Computer Society 123 128

            32. 2014 „Deep Learning Face Representation from Predicting 10 , 000 Classes‟

            33. 2017 „Facial expression recognition with FRR-CNN‟ Electronics Letters 53 4 235 237 doi: 10.1049/el.2016.4328

            34. 2015 „Image based Static Facial Expression Recognition with Multiple Deep Network Learning‟ ACM on International Conference on Multimodal Interaction - ICMI 435 442 doi: 10.1145/2823327.2823341

            Comments

            Comment on this article