Automatic motion tracking of lips using digital video and OpenFace 2.0

Abstract

Because the lips are external organs, they can be easily observed by means that are non-invasive, including simple video recording. The current paper introduces a free-of-charge, highly portable, automatic solution for extracting oral posture from digital video, based on an existing face-tracking utility. We describe how this solution might benefit various lines of laboratory phonology research, including analysis of articulatory coordination and audiovisual speech, as well as phonetic fieldwork. We then provide a tutorial, framed as a simple experiment in metronomic speech. The tutorial describes how to configure the software to work with one’s digital camera of choice, how to extract relevant articulatory parameters from the face tracker, and how to approach statistical analysis. We have supplied pre-written Python scripts to aid the reader in following along with the experiment. The portability, ease, and affordability of the described solution have important implications for the accessibility of articulatory measurement, both in the lab and out in the field.