Predicting users’ task difficulty using Social Signals: a Preliminary Model

Humans communicate social intentions through patterns of nonverbal language, using posture, gestures and body motion. This social signalling is present in human to human interaction as well as in human-computer interaction. Our daily dependence on computers emphasizes the need and importance for good interaction quality. While humans have an innate ability to recognize and respond to social signalling, machines don’t. Our work aims to develop a Social Signal Processing model based on features extracted using simple video processing techniques, applied in a real context and running in real-time, to predict interaction’s difficulties and problems. In this study we report a preliminary model where features extracted from user motion within 60 seconds of video recordings can predict 46,6% of variance in task difficulty.