DESIGNING A HAPTICALLY EXTENDED INTERFACE FOR DIGITALLY ANIMATING 3D ARTICULATED CHARACTERS

Conventional non-digital physical stop-frame animation is based on the inherently intuitive direct two handed manipulation of a passive figure. Digital media may provide powerful software tools for animation but they do so through less intuitive interfaces. We argue that we can unite these two areas in a new system which utilises the full potential of the stop-frame animators’ tacit skills. We seek to create a haptic animation tool, the exact form of which is determined through the conduct of participatory design. This paper first introduces a theoretical framework on the design of the proposed animation system which is built in the digital/physical frontier. We then discuss the selection of participatory design as our methodology and describe the initial steps of this process such as interviewing animators about their practice and exposing them to the haptic device. We finally give a brief description of the initial technical and conceptual conception of the future system.


INTRODUCTION
Digitality has infiltrated most of our work methods yet it is still far from providing a seamless environment that will mediate and evolve our physical skills.The primary source of this problem is not technological artefacts themselves but the way we design their purpose.
The tools have changed.In computer mediated practice, software tools are used for carrying out all tasks while mouse and keyboard are the main tactile communication point with the digital artefacts.In fact, mouse and keyboard have been the main input devices for over 20 years, an interval disproportional to the rate at which computer science changes.At the same time the design of digital tools has been dependent upon the available technology rather than on the human abilities with all their potential and limitations.This is more distinctly observed in disciplines that have traditionally used physical practice for creative purposes.For example, architects have shifted from maquette making to using computer-aided design (CAD) systems for building 3D models and the majority of animators have advanced to using computer graphics (CG).Work environments have also moved from a physical location to a virtual working place.The problem with this shift is that, while crafts practitioners are sophisticated in EVA 2009 London Conference ~ 6-8 July Mariza Dima et al. _____________________________________________________________________ their sensory interaction with and manipulation of physical media, they do not interact naturally to the same degree with digital media.The widely used mouse and keyboard devices coupled with the graphical user interfaces are not adequate tools for those practices.
Humans naturally interface with their body.Traditional craftsmanship requires a lot of 'skilled' hand practice.Manipulation of a physical artefact is an act that demands the involvement of multi-sensory channels and, naturally, provides richer interaction than when the same motion is repeated within the digital world.The latter fails in terms of incorrect mapping and lack of directness during interaction.The tasks that the hands carry out are three dimensional while the generic mouse is a two dimensional device.Hence, hands' motion is limited to the x-y plane and, no matter how intuitive the graphical interface is, the medium maps a 2D input motion to a 3D output motion.
Additionally, the medium cannot render the same degree of direct interaction that the hands provide.In creative practices in particular, we may consider the vital component of skill.Skill composes to a large degree what Polanyi [10] termed "tacit knowledge", knowledge that exists but cannot be articulated and is therefore difficult to share.In certain practices, when skill is expressed through somatic movement, either full-body or through parts of it, it acquires a stronger physical dimension; it becomes "embodied knowledge".In contrast, using computer software to carry out a task requires managing visual representations of symbolic structures which appear in front of our screen as desktop metaphors of objects we use daily.This management happens mentally.The shift from handling physical objects to working with immaterial symbolic notational media requires adaptation of the human cognitive model from "the skilful hand to the reasoning mind" [9].It also presents a new challenge for human-computer interaction designers.Technological advances and design theories which are constantly enriched with principles from a variety of disciplines, can, in combination, assist the design of 'better', more 'natural' interactions for the skilled practitioner.

MOTIVATION
Our interest in the discipline of stop-frame animation is based partially on the premise that, although much of commercial animation is now digital, the art of traditional stop-frame animation has not been rendered obsolete.Identifying the reasons for this provides a substantial starting point for our investigation.Secondly, stop-frame animators use their inherent physical skills and hands-on tactile direct manipulation to animate a physical puppet, which makes this craft an appropriate study case for the transformation/mediation of the craftsman's tacit knowledge when digitality is added to the working environment.Finally, the stop-frame animators' tacit knowledge embodies a sense of time and acting, two elements that we are eager to analyse through this research.It should be noted here that we focus on articulated figures merely because their animation is more complex than that of a very simple item.
We have selected a haptic device as the primary tool of our system.Haptic technology, in contrast to generic two dimensional devices, utilises additional sensory channel bandwidth of the user by employing tactile and kinaesthetic values that convey the sense of touch in the form of force-feedback.Our choice is driven by the EVA 2009 London Conference ~ 6-8 July Mariza Dima et al. _____________________________________________________________________ fundamental role that the sense of touch plays in stop-frame animation and in skilled practice in general.Haptic tools allow the tactile exploration of three dimensional virtual artefacts.We intend to use a 6-degrees of freedom (DOF) Sensable TM Phantom Omni haptic device which consists of a base and a robotic arm with a stylus at its end.Interaction takes place by the user moving the stylus in space.When the stylus comes in contact with a three dimensional virtual surface, the arm provides force-feedback through internal mechanical actuators.The haptic device can be programmed to identify different values of texture and material properties like hardness, roughness etc.
We begin with the hypothesis that an appropriate method for creating a digital animation system more in accordance with the tacit knowledge of stop-frame animators is through the creation of a haptic animation tool, the exact form of which is determined through the process of participatory design.Following an iterative design scheme, we search for evidence that support this assumption and, furthermore, we investigate how haptically-mediated work influences skilled animation practice and vice-versa.In the following paragraphs we will attempt to outline the theory that drives our research and then present the initial work that has been completed so far.

THEORIES OF PHYSICALITY, DIGITALITY AND DESIGN
Our main research questions concern the properties of interaction.With an ultimate aim to build a system which will provide natural interactions between the animator and the virtual artefact, we initially seek to enquire about those affordances and visceral qualities of the haptic device that allow its use to become natural.We then seek to design the haptic animation system by drawing on a theoretical framework that connects physicality and the digital.

The creation of symbolic context
An entirely natural human-computer interaction depends on many things, most of which cannot be taken into account ahead of the design process.However, to start with, we may draw attention to the way the properties of the tool we intend to use endorse naturalness.Our system consists of the animator, the device which operates through the medium (computation) and a set of actions that will determine the overall structure of the interaction.Effectively, we bring a user who has mastered several ways of tactilely manipulating a physical character to work with a medium that can be handled by manipulating symbols.Handling of symbolic structures by our brain requires the construction of mental models.Mental models are explanatory models of how one perceives a function in the world.As representational elements of a specific activity, they are a key aspect to bringing the user closer to media that use layers of abstraction such as computing [9].The most direct is our engagement with these elements, the more natural the interaction is.In our hybrid system, we need to create symbolic context so that we establish a link between physical action and digital activity.For this, we have decided to remove any graphical parts from the interface and use instead a mimetic approach to translate gestures for moving and animating the virtual character.The gestures should meet with the corresponding motion of the character.Designing symbolic context is an essential

Physicality
Adding the sense of touch in a digital system for creative practice offers a great potential for engagement.Especially in crafts, where hand gestures and tactileness are fundamental aspects of practice, the continuity of artistic experience with normal cognitive processes is situated in the impulse to handle materials and to think and feel through their handling [1].Additionally, manual practice is not abstract.Hands are the medium through which the embodied knowledge of the craftsman surfaces.Gestures convey meaning.They are the externalisation of the thought process and they reflect the practitioner's interpretative methods [5].It is essential to take into account these phenomena as a design guide for mapping animators' gestures to output motion.
We have highlighted the importance that direct engagement with the visual representations plays in making the interaction natural.In our system we support this argument in two ways.Firstly through the device we use and secondly by designing symbolic context that allows direct manipulation of the virtual character.The idea of direct manipulation has been formed from the development of graphical user interfaces and has been explored by many [13] in terms of a one-to-one mapping between mouse and graphical buttons within the point-and-click method of interaction.Its basic principle is that digital interfaces represent clearly the objects to be handled and that their handling is very direct.We investigate how direct manipulation is enhanced with the introduction of the haptic device, since haptics augments a generic device with two properties: it requires additional sensory-motor skills, the same which are used during physical manipulation and, in addition, performs a one-to-one mapping between three dimensional motions.

Context of the interaction
The final step to consider for establishing natural interaction is creating the appropriate context of the interaction, the relation between the tool, the medium and the practitioner's actions which will create and maintain seamless experience.When experience flows, users' actions are situated.Suchman points out that interaction context is being formed by the actions of the users -'actors' in an "exhibited momentby-moment improvised character" [14].In this scope, Dourish emphasizes the link between action and meaning and suggests that these together are responsible for the formation of context [3].
A well-designed mapping between the animators' gestures and the output motion of the virtual character may create an environment in which animators are able to produce the desired result by making actions similar to the ones embedded in their tacit skills.For this to happen, the device which has substituted the hands will have gradually EVA 2009 London Conference ~ 6-8 July Mariza Dima et al. _____________________________________________________________________ disappeared into the background and let information flow seamlessly between human and machine.The device will have now become a 'ready-to-hand' [7] tool.
The process of a tool becoming 'ready-to-hand' passes through what Polanyi explained as distal and proximal phenomena.Polanyi suggests that embodied skills lie on our focus on phenomena that happen in the space around us while our sensory experience takes place close to our body [10].In our example, the haptic device operates through the control metaphor of a virtual probe which acts as the visual extension of the device in the virtual world, while the reactive forces are exerted locally at the base of the stylus the user hold.In our hybrid system physicality does not really exist but is simulated through a series of symbolic representations, so that sensory experience of force-feedback 'happens locally' at the hands of the animators while the result is visualised on the screen where the character exists.
Continuous 'play' with the system [9] has the potential to unveil important information about the way "our tools shape us as we shape them", to use Marshall McLuhan's central theory on the relationship between man and media.On this matter, we may use as a tool for qualitative investigation Donald Schön's work on reflective practice and his insight into the processes that practitioners utilise to: Surface and criticize the tacit understandings that have grown up around the repetitive experiences of a specialised practice, and make new sense of the situations of uncertainty or uniqueness which he may allow himself to experience.[12] There is another strand attached to this process and that is practice.By practising with a tool one acquires new levels of awareness of one's craft and becomes gradually more competent in one's work.This presents a great opportunity for observing how physical and symbolic tools in cooperation mediate the practitioner's embodied knowledge and how personal mental models shape the user's experience.McCullough suggests that an experienced person often redirects contextual awareness back onto the process itself [9].Dourish also considers practice to be an essential method of extracting those fundamental elements that "develop the meaning of the use of technology as it is incorporated into practice" [2].
These points all contribute to our better understanding of the processes that embodied knowledge undergoes when situated in a digital setting.We expect that the initial tests of a first prototype system will provide us with valuable material for further investigation.So far, we have outlined the theory upon which our design will be based and have shown how each part of it connects to our initial hypothesis.In the following paragraphs, we will describe our methodology.

DESIGN APPROACH Participatory design
Drawing on our previous discussion over interaction context, designers are responsible for creating a coherent set of functions but, ultimately, users are those who will add to the system the context that will make it engaging.We argue that a collaborative process, enriched with several qualitative methods for gathering EVA 2009 London Conference ~ 6-8 July Mariza Dima et al. _____________________________________________________________________ information in the beginning and during each design stage, provides a productive, wellestablished framework for our purposes.For that reason, we choose to engage in participatory design, working with a team of four stop-frame animators from Edinburgh College of Art's Animation Department.In participatory design, final users of the system are not only consulted but are actively involved in the development of the system as its end-users or 'inter-actors'.We plan to follow an iterative design scheme where during participative session we collect the relevant data and interpret them through open discussions with the animators.After processing the outcomes, we then structure the conceptual, theoretical and practical work.
Participatory design entails several qualitative methods which have been used to collect data based on our initial hypothesis.Two methods that that have been extensively used so far in this research are participant observation and semi-structured interviews.In the following paragraph we describe the pilot study that we conducted before work on a prototype system begins.

Pilot study
The first part of the collaboration consisted of a series of semi-structured interviews and some informal discussions through which we have sought to understand the way the animators use their skills, the elements of hand practice that they regard as important and their thoughts on the stop-frame animation practice as a whole.
A point they strongly highlighted was that they always want to have complete control over the motion of the articulated character.In this way, they have the freedom to exaggerate movements or slightly twist the model in a way that will render its motion believable but not entirely correct in terms of human motion.We should note here that stop-frame animators are interested in creating character animation which resembles but does not reproduce human motion in the highest detail.This statement will be our guide in designing the mapping between the animators' gestures and the corresponding output motion.
Algorithmic processes that calculate the position of an articulated character's parts were not of interest to them, due to the aforementioned will for control but they regarded as an advantage of digital animation the fact that the animator can easily navigate in the timeline and 'undo' a frame.
They have also identified acting and timing as two important elements that all stopframe animators are trained on.In particular, they have described themselves as actors who, instead of acting themselves, mediate the desired output through their gestures onto their models.They are also trained to manipulate motion in relation to time, knowing exactly how to represent the position and posture of each part of the character in each time frame.
The Sensable TM Omni haptic device was presented to the team and some initial tests were run so that they had a first idea of how it works and, most importantly, how it 'feels'.Their response was very positive.They were particularly impressed by the feel of touch that the device would re-create.One of them stated that he "would not see any reason for not using such a device for stop-frame animation".

EVA 2009 London Conference ~ 6-8 July
Mariza Dima et al. _____________________________________________________________________ One important observation was the significance of using both hands for manipulation.We have therefore decided to add a device for the non-dominant hand which will perform complementary functions such as moving the animator's viewpoint or the character, zooming in and out of the virtual world and panning it.Our choice was 3DConexxion's Space Navigator, a 3D mouse with six DOFs which, due to its design, we believe to be an appropriate tool to simulate these functions.
The interviews were followed by three sessions in which the animators were observed while working in their studio space.

SUMMARY
In this paper we have touched on notions of designing human-computer interaction that brings physical skilled practice in the digital realm.We have described our proposed system and methodology and the initial qualitative research that we have conducted.
We are now at the stage of implementing our design in the first prototype system.Upon completion of this first step, we will evaluate the system and the outcomes of the evaluation will be analysed and discussed.This phase will lead us to the design, build and evaluation of the second prototype and then the process will be repeated until we have a well-designed haptic animation system.To achieve a well-designed system, we seek to gather material from each completed design and implementation step related to our research hypothesis and to reflect upon it in combination with our theoretical framework and with the participation of the stop-frame animators.
EVA 2009 London Conference ~ 6-8 July Mariza Dima et al. _____________________________________________________________________ part of this research and it is the part that is more bound to change at each design step until we reach a stage where our design becomes effective.As McCullough suggests:Skillful operations of physical devices are given leverage through effective symbolic structures[...].Through the abstraction of symbolic representation, practices, playful talent finds new outlets and develops new kinds of appreciation.[9]