Exploring Musical Creativity through Interactive Cymatics

What does music look like? Representation of sound has taken many forms over time, from sinusoidal wave patterns, through to random algorithm-based visualisations based on the amplitude of an audio signal. One aspect of music visualisation that has not been widely explored is that of Cymatics. Cymatics are physical impressions of music created in mediums such as water. This paper presents CymaSense, an interactive sound visualisation application based on Cymatics.


INTRODUCTION
Sound visualisation takes many forms: musical notation is used for pedagogical purposes; VU meters display audio signal information; while music visualizers use arbitrary mapping of audio attributes to create artistic visual feedback.Cymatics are physical impressions of sound created in liquid or particulate mediums (Figure 1Error!Reference source not found.).Inspired by the patterns created when sound is visualised in water, a prototype has been developed as a tool to inspire musical creativity.CymaSense is an interactive multimodal interface that generates realtime 3D graphics, based on Cymatics (McGowan, Leplatre, & McGregor, 2017).
The development of CymaSense has been driven by the need for accessible therapeutic tools for people with sensory impairments, specifically to improve communication for people on the autism spectrum.Autism Spectrum Disorder (ASD) is a lifelong neurodevelopmental disorder where people share difficulties in social communication, interaction, and problems in the use of language and verbal communication (Paxton & Estay, 2007).Music therapy is considered an effective approach to address these issues for people with ASD (Lim & Draper, 2011).Music therapy could be made more accessible by creating an audio-visual interactive tool that responds to clients' own abilities.A series of interviews with music therapy practitioners identified therapist and client needs which provided functional requirements for the tool's design.The CymaSense prototype is currently being evaluated in an on-going study with autistic participants.Initial results indicate increases in communication for verbal and non-verbal participants.

CYMATICS
Cymatics are physical impressions of sound, created as a result of diffraction and refraction of sound waves created within the visualising medium (Jenny, 1968).Cymatics until now have been viewed as quasi-3D patterns in water or on the surface of a Chladni plate.2D and 3D Cymatic software simulations exist, but are based on Chladni's 18th century principles.The visualisations represent a cross section of a sonic bubble created by the resonance of a particular size of plate, not the shape of sound.Sound propagates in a spherical manner from its source contrary to typical representations of sound waves.

CYMASENSE
CymaSense is an interactive application that generates real-time 3D  Users will experience sound as interactive Cymatic translucent bubbles, controlled by the volume, pitch and timbre of the input sound (Figure 2).The size of the Cymatic shape displayed is affected by volume.Pitch determines which one of twelve 3D Cymatic shapes are visualised, also implementing an increase in colour lightness as pitch rises.Tonal differences create subtle changes in the surface qualities of the shapes, where brighter sounds are more pointed, with darker sounds being more rounded.The shapes themselves are comprised of an outer shell and two sets of animated particles for effect.The user interface provides colour choices for the basic shell and particles, as well as background colour, allowing a variety of colour combinations.Rotation of the 3D shapes can also be facilitated, as well as recording and playback of audio input.CymaSense exists as a single and two-user version (Figure 3).
CymaSense has also been employed within a tangible user interface, where a custom built table processes vibration from a contact microphone and projects the resultant visualisation below the table surface, thus creating a novel interactive percussive audio-visual instrument (Figure 4).

FUTURE WORK
Potential alternative environments for CymaSense include audio-visual art installations, virtual or augmented reality applications.Other sensory impaired groups for example, the hearing impaired, could benefit from its use.

REFERENCES
Jenny, H. (1968).CYMATICS.The Sciences, 8(7), 12-18.http://doi.org/10.1002/j.2326-1951.1968The CymaSense application is run on a laptop.An audio interface provides the appropriate connections for microphone and MIDI keyboard.A set of small speakers is used for audio output and a Pico projector is used for visual output.For synthesised sound, a small Roland sound module is connected to the MIDI keyboard.

Equipment required:
• A projection screen, or suitable white/blank wall space • A small table will be required for this equipment All remaining equipment will be provided by the researcher.The application can either be projected onto a traditional screen or a white wall with sufficient clearance between the projector and wall.
The light within the room would ideally be low for greater immersion and clearer views of the shapes an internal space would be preferred.If desired, a larger projected image would require use of a large projector as well, but this is not a requirement.Alternatively, a large monitor could be used for visual output (which would need to be provided).
Duration of the use of the application is dependent on each user and their engagement.Previous open days have seen extensive use of the application, running continuously for the whole day.

Figure 1 :
Figure 1: Cymatic sound vibrated through water a 3 rd year PhD student and parttime lecturer in the School of Computing at Edinburgh Napier University.His interests lie in audio, music production and 3D animation.Grégory Leplâtre is a lecturer in the School of Computing at Edinburgh Napier University.His research interests pertain to Auralisation, Visualisation and Music Technology.Dr Iain McGregor researches sound design and listening at Edinburgh Napier University, where he is also the Programme leader for both the undergraduate and postgraduate degrees in sound design.