Multimodal Music Composition

Multimodality is inherent in the majority of our media experiences, most commonly exemplified in the cinematic arts. A combination of two (or three) dimensional video, dialogue, audio, soundtrack, and occasionally, vibrating chairs or emotive smells, create an engaging environment that submerges audience members into the action. Despite our continuous engulfment in stimulating environments, it can be difficult to isolate how sensory information effects our perception of an experience. By understanding an individuaPs perceptual and emotional response to multisensory experiences we can harness communication to enhance user experience. This paper discusses multimodal and multimedia technology communications that can enhance music composition by incorporating additional sensory stimulation. First the work’s context is given, followed by previous work that has informed this project. The paper presents design concepts, developments and considerations, concluding with the current status and future directions.