A Gestural Interaction Design Model for Multi-touch Displays

Media platforms and devices that allow an input from a user’s finger/hand touch are becoming more ubiquitous, such as Microsoft Surface and DiamondTouch, as well as numerous experimental systems in research labs. Currently the definition of touch styles is application-specific and each device/application has its own set of available touch types to be recognized as input. In this paper we attempt a comprehensive understanding of all possible touch types for touch-sensitive devices by constructing a design model for touch interaction and clarifying their characteristics. The model is composed of three structural levels (action level, motivation level and computing level) and the relationships between them (mapping). In action level, we construct a unified definition and description of all possible touch gestures, first by analyzing how a finger/hand touch on a surface can cause a particular event that can be recognized as a legitimate action, and then using this analysis we define all possible touch gestures, resulting in touch gesture taxonomy. In motivation level, we analyze and describe all the direct interactive motivation according to applications. Then we define the general principles for mapping between the action and motivation levels. In computing level, we realize the motivation and response to gestural inputs using computer languages. The model is then used to illustrate how it can be interpreted in the context of a photo management application based on DiamondTouch and iPod Touch. It allows to reuse touch types in different platforms and applications in a more systematic and generic manner than how touch has been designed so far.