MAKING MUSIC WITH IMAGES : INTERACTIVE AUDIOVISUAL PERFORMANCE SYSTEMS FOR THE DEAF

This paper describes the technical and aesthetic approach utiliz ed for the development of an interactive audiovisual performance system designed specifically for use by children with multiple learning difficulties, including deafness and autism. Sound is transformed in real-time through the implementation of a Fast Fourier Transform (FFT) and translated into a moving image. This image is adapted so that relevant information can be understood and manipulated visually. Finally, the image is turned back into sound. The translation process is based on research in computer music, neuroscience, perception and abstract film studies, supported by the Arts and Humanities Research Council (AHRC).