Voice emotion recognition by Mandarin‐speaking pediatric cochlear implant users in Taiwan

Abstract

Objectives

To explore the effects of obligatory lexical tone learning on speech emotion recognition and the cross‐culture differences between United States and Taiwan for speech emotion understanding in children with cochlear implant.

Methods

This cohort study enrolled 60 cochlear‐implanted (cCI) Mandarin‐speaking, school‐aged children who underwent cochlear implantation before 5 years of age and 53 normal‐hearing children (cNH) in Taiwan. The emotion recognition and the sensitivity of fundamental frequency ( F0) changes for those school‐aged cNH and cCI (6–17 years old) were examined in a tertiary referred center.

Results

The mean emotion recognition score of the cNH group was significantly better than the cCI. Female speakers' vocal emotions are more easily to be recognized than male speakers' emotion. There was a significant effect of age at test on voice recognition performance. The average score of cCI with full‐spectrum speech was close to the average score of cNH with eight‐channel narrowband vocoder speech. The average performance of voice emotion recognition across speakers for cCI could be predicted by their sensitivity to changes in F0.

Conclusions

Better pitch discrimination ability comes with better voice emotion recognition for Mandarin‐speaking cCI. Besides the F0 cues, cCI are likely to adapt their voice emotion recognition by relying more on secondary cues such as intensity and duration. Although cross‐culture differences exist for the acoustic features of voice emotion, Mandarin‐speaking cCI and their English‐speaking cCI peer expressed a positive effect for age at test on emotion recognition, suggesting the learning effect and brain plasticity. Therefore, further device/processor development to improve presentation of pitch information and more rehabilitative efforts are needed to improve the transmission and perception of voice emotion in Mandarin.

Level of evidence

3.

Abstract

Better pitch discrimination ability came with better voice emotion recognition. Besides the F0 cues, cCI adapted their voice emotion recognition to rely more on secondary cues such as intensity and duration. Although cross‐culture differences existed for the acoustic features of voice emotion, Mandarin‐speaking cCI and their English‐speaking cCI peer exhibited a positive effect between age at test on emotion recognition, suggesting the learning effects or possibly maturation effects.