LiteFoot-Auditory Display of Footwork

This paper describes the development of LiteFoot, an interactive floor space that tracks dancers steps, and converts the steps into auditory and visual display. The system can also record steps, for further analysis for use in dance research programmes, choreographic experimentation and training.


Introduction
To explore possibilities of human expression in dance and music, a prototype floor space was developed that could record the location and impact force of dancers' feet and to let the captured data control other kinds of media.A number of similar systems exist, such as Johnstone [1], Pinkston [2], and Paradiso [3] that detect contact, force or weight of a dancer that is then converted into controls for sonic events.The hardware and software described in this paper -the LiteFoot project -was based on direct user participation and requirements of the dance practitioners and researchers in the Irish World Music Centre in the University of Limerick.They wanted to be able to record and analyse traditional Irish dance.This could be done through existing technology that, in principal, tracks a dancers movements in 3-D.Unfortunately the existing solutions are either expensive, invasive 1 or not fulfilling the requirements.The existing noninvasive solutions were not capable to fulfil the users' requirements for speed, resolution or range.

Previous designs
The PodoBoard [1] was developed to facilitate the extension of the use of clackage, a form of seated dance that normally takes place on a small wooden floor.The PodoBoard provides an accurate set of co-ordinates for the position of feet contact with the floor, which is a matrix of 1 inch square aluminium tiles.The reported reaction time of the floor to footsteps is extremely fast 2 .However, although very impressive in speed this system has some drawbacks.It works through using the shoes to complete an electrical circuit -these have metal contacts at toes and heel.The system is critically dependent upon good electrical contact between shoe and floor.A different solution to using footsteps was developed by Pinkston [2].This floor used force sensors arranged in strips.The granularity of the system is approximately 6 inches.This is again a very useful floor, but the sensor technology is relatively expensive.The floor also has two characteristics that make it less attractive as a potential medium for recording dance steps.Firstly, the system is not designed to deliver the precise co-ordinates of the footsteps.It might be possible, by using orthogonal, overlapping strips to calculate positions, but as the floor size increases it is likely to prove to be impractical.MIT Media Lab's Magic Carpet [3] also uses the force of a persons footsteps.The technology used is based on cables insulated with piezoelectric material that responds to compression and bending by producing a change in capacitance.The output is multiplexed and the cables are scanned 60 times per second.The floor is very sensitive to foot pressure.The system was designed to allow people in public spaces to create and manipulate a sound environment.There are a number of aspects that again make it less useful as a medium for recording dance.Firstly, the resolution of the floor is 100mm which is a relatively coarse grained matrix when considering the translation of steps into a choreographic representation.The rate of detection is also unable to cope -for example -with Irish traditional dance where steps can be as rapid as 30 per second.The 'Magic carpet' cannot guarantee to locate the steps of more than one foot at once.This is due to the use of a multiplexed grid of wires, i.e. that one foot can shadow another, because the system scans the periphery of the total area.All these designs have in common that their output can be recorded and/or control a MIDI synthesizer.

Designing LiteFoot
The following requirements for were identified as key elements for this design:

•
The floor should be able to respond at a rate corresponding to at least 30 steps per second.

•
The floor should have a reasonable spatial resolution (40 millimetres).

•
The floor should be able to track multiple feet and dancers.This list is by no means exhaustive, but covers the key aspects that were identified in the design process through direct participation by dancers.The LiteFoot prototype is a 1.76 meter square and 10 centimetres high floor element, filled with a matrix of 1,936 optical proximity sensors.When a person stands on the floor the spatial locations of the feet are detected.A single accelerometer detects the total impact force.The floor has an embedded micro-controller that scans all sensors every 10 ms.If any change has occurred since the last scan, a message is sent via a serial highspeed connection to a PC 3 running two special application programs, 'TipTapToe' and 'FootWare'.The former provides the serial communication functions and the latter interprets the received data and converts it into auditory and visual representations.The fundamental principles underlying the LiteFoot software are that the software should enable simultaneous modes of independent use, i.e. performance, recording and analysis to take place concurrently.Arbitrary mappings between the data coming from the floor and representations can be defined by the user.The visual representation has, so far, been a direct mapping of location to displayed groups of pixels, with the colour controlled by the impact force.For the auditory representation the incoming data-stream controls the MIDI synthesizer of a standard sound card, with musical scales in one dimension (X) and sets of timbres in the other dimension (Y).The impact force has been mapped to loudness (Z).(See Figure 1)

LiteFoot Live
The LiteFoot system was first displayed in public in September 1997 in a performance in the National Concert Hall at the University of Limerick.Both traditional Irish dance an improvised modern dance was performed on the system and several members of the audience reported the performance to be aesthetically and perceptually engaging.Secondly, in January 1998 the system appeared on Irish national television as an example of new art and technology.Since then it has been demonstrated in several different settings, including an international workshop on dance at the University of Limerick as well as, most recently, in an interactive art exhibition in Limerick City.

Loudness Colour
So far, only informal evaluation has been done.Dancers have worked together with designers in what could be described as live rehearsal situations, where the dancers' comments directly has resulted in source code modifications.From the comments from users, the following applications have been suggested:

•
Training -a dance teacher can not only tell a student to 'do as you see me doing' but also, at the same time, 'do as you hear me doing'.Training modes could be both direct or differential (to make differences between teacher and student audible).
Although not yet formally tested, users have indicated that this is potentially of high interest.

•
Performance -Enabling persons to have the motion of their feet converted into musical structures opens up a number of new possibilities in performing arts.We suddenly have a system where a dancer can produce his or her own accompaniment, but also a system that can make very small and subtle movements audible and visible.

•
Play -In the recent art exhibition mentioned above, the LiteFoot system was exhibited as an interactive installation.It was much appreciated by children who could just by walking, jumping and crawling on it, produce sounds and visuals that corresponded to their actions.With a number of people on the floor (at this occasion, up to 12 children), they could also see and hear how their collaborative actions combined.

Conclusion
The evaluation of LiteFoot is just starting.With its existing software utilities it can be used for both performance and play.There are, of course, numerous other possibilities to be explored, for example to use other kinds of mappings.Instead of direct mappings, orchestrations and sound or music sequences with parametric control can be mapped to areas of the floor.When several persons are active on the floor simultaneously, the level of 'collaborative harmony' could be mapped to tonal harmony and temporal structures.From the input data, higher order products can be integrated or derived, to allow for example the acceleration of a movement to be mapped to sonic properties.Musicians need not to fear, as there are possibilities in having musicians create dynamic mappings, in collaboration with dancers.As a play space, LiteFoot seems to be highly engaging (every child loves to make noise), but there are also a number of possibilities to be evaluated with for example disabled children who might be able to extend their action range through training with auditory feedback.