A Vision-based System for Display Interaction

This paper presents a system for interaction with a display via hand pointing, where a single CCD camera on top of the screen is directed towards the viewers. An attention mechanism based on face and hand detection allows one user to take control of the interface. Face recognition is used for identification and customisation. The system allows the user to control the screen pointer by tracking their fist. Onscreen items can be selected using one of four activation mechanisms. Current sample applications include browsing image and video collections as well as viewing a gallery of 3D objects. In experiments we demonstrate the performance of the vision components in challenging conditions.