Blog
About

0
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Listen to Your Face: Inferring Facial Action Units from Audio Channel

      Preprint

      , ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Extensive efforts have been devoted to recognizing facial action units (AUs). However, it is still challenging to recognize AUs from spontaneous facial displays especially when they are accompanied with speech. Different from all prior work that utilized visual observations for facial AU recognition, this paper presents a novel approach that recognizes speech-related AUs exclusively from audio signals based on the fact that facial activities are highly correlated with voice during speech. Specifically, dynamic and physiological relationships between AUs and phonemes are modeled through a continuous time Bayesian network (CTBN); then AU recognition is performed by probabilistic inference via the CTBN model. A pilot audiovisual AU-coded database has been constructed to evaluate the proposed audio-based AU recognition framework. The database consists of a "clean" subset with frontal and neutral faces and a challenging subset collected with large head movements and occlusions. Experimental results on this database show that the proposed CTBN model achieves promising recognition performance for 7 speech-related AUs and outperforms the state-of-the-art visual-based methods especially for those AUs that are activated at low intensities or "hardly visible" in the visual channel. Furthermore, the CTBN model yields more impressive recognition performance on the challenging subset, where the visual-based approaches suffer significantly.

          Related collections

          Most cited references 5

          • Record: found
          • Abstract: not found
          • Article: not found

          Toward detecting emotions in spoken dialogs

            Bookmark
            • Record: found
            • Abstract: not found
            • Book Chapter: not found

            Disentangling Factors of Variation for Facial Expression Recognition

              Bookmark
              • Record: found
              • Abstract: not found
              • Conference Proceedings: not found

              Deep learning the dynamic appearance and shape of facial action units

                Bookmark

                Author and article information

                Journal
                2017-06-22
                Article
                1706.07536

                http://arxiv.org/licenses/nonexclusive-distrib/1.0/

                Custom metadata
                cs.CV

                Computer vision & Pattern recognition

                Comments

                Comment on this article