4
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Temporal Multimodal Fusion for Video Emotion Classification in the Wild

      Preprint
      , ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          This paper addresses the question of emotion classification. The task consists in predicting emotion labels (taken among a set of possible labels) best describing the emotions contained in short video clips. Building on a standard framework -- lying in describing videos by audio and visual features used by a supervised classifier to infer the labels -- this paper investigates several novel directions. First of all, improved face descriptors based on 2D and 3D Convo-lutional Neural Networks are proposed. Second, the paper explores several fusion methods, temporal and multimodal, including a novel hierarchical method combining features and scores. In addition, we carefully reviewed the different stages of the pipeline and designed a CNN architecture adapted to the task; this is important as the size of the training set is small compared to the difficulty of the problem, making generalization difficult. The so-obtained model ranked 4th at the 2017 Emotion in the Wild challenge with the accuracy of 58.8 %.

          Related collections

          Most cited references17

          • Record: found
          • Abstract: not found
          • Article: not found

          The Structure of Current Affect: Controversies and Emerging Consensus

            Bookmark
            • Record: found
            • Abstract: not found
            • Conference Proceedings: not found

            Hierarchical recurrent neural network for skeleton based action recognition

              Bookmark
              • Record: found
              • Abstract: not found
              • Book Chapter: not found

              Sequential Deep Learning for Human Action Recognition

                Bookmark

                Author and article information

                Journal
                21 September 2017
                Article
                1709.07200
                6b6bbf61-7a52-484c-a72a-4f85f67ac8a1

                http://arxiv.org/licenses/nonexclusive-distrib/1.0/

                History
                Custom metadata
                ACM - ICMI 2017, Nov 2017, Glasgow, United Kingdom
                cs.CV cs.LG cs.MM
                ccsd

                Comments

                Comment on this article