The Augmented Tonoscope

The Augmented Tonoscope is an artistic study into the aesthetics of sound and vibration through its analogue in visual form the modal wave patterns of Cymatics. Key to the research is the design, fabrication and crafting of a sonically and visually responsive hybrid analogue/digital instrument that will produce dynamic Visual Music based on the physics, effects and manifestations of sound and vibration. This paper describes the first stage of the study a series of artistic investigations into analogue tonoscope and digital tone generator design integrating light and camera control driven by an artistic experimental method devised specifically for the project. The paper concludes with future directions for the research: Can the inherent geometries within sound provide a meaningful basis for Visual Music? Will augmenting these physical effects with virtual simulations realise a real-time correlation between the visual and the musical?


INTRODUCTION
How far can artistic investigation into Cymaticsthe study of wave phenomenon and vibrationcontribute towards a deeper understanding of the interplay between sound and image in Visual Music?
The term Cymatics (from Greek: κῦµα "wave") was coined by the researcher Dr Hans Jenny (1967,1972) who studied this subset of modal wave phenomena using a device of his own design -the ʻtonoscopeʼ.As John Telfer (2010) explains "Sound can induce visible pattern.When physical matter is vibrated with sound it adopts geometric formations."I aim to develop this line of research by exploring the aesthetics of cymatic patterns and forms.Is it possible to develop a cymatic visual equivalence to the auditory intricacies of melody, harmony and rhythm?
The focus of my research is to design, fabricate and craft a sonically and visually responsive hybrid analogue/digital instrument -a contemporary version of Jenny's sound visualisation tool -the Augmented Tonoscope.I then intend to play, record and interact with it to produce a series of artistic works for live performance, screening and installation.

CONTEXT & FIELD MAP
From Isaac Newton's colour circle correlating hues with musical notes, which inspired a succession of colour organs and Lumia, to the synthetic sound production film experiments of Oskar Fischinger and Norman McLaren, there is a long and rich history of scientific and artistic study into the theoretical and aesthetic relationships between the aural and visual.This evolving practice is broadly defined by the term Visual Music and currently includes: • methods or devices which translate sounds or music into a related visual presentationpossibly including the translation of music to painting; • the use of musical structures in visual imagery; • systems that convert music or sound directly into visual forms (and vice versa) by means of a mechanical instrument, an artist's interpretation, or a computer.
This last definition is especially true of my research since I plan to utilise all three suggested means in the Augmented Tonoscope.
It also describes the work of filmmakers such as Oskar Fischinger and Norman McLaren who worked in this latter tradition of creating a visual music comparable to auditory music in their animated abstract films.Oskar Fischinger believed his synthetic sound production experiments held extraordinary potential for the future of musical composition and sound analysis: "Now control of every fine gradation and nuance is granted to the music-painting artist, who bases everything exclusively on the primary fundamental of music, namely the wave -vibration or oscillation in and of itself." The use of the computer in film making stemmed from the early approaches and experimental 16mm work by key figures John and James Whitney in the late 1940s.For John Whitney (1980)

Artistic Experimental Method
I'm intrigued by the potential of a creative counterpoint to the established scientific experimental method of gathering observable, empirical and measurable evidence subject to specific principles of reasoning.Can I devise my own robust, investigative yet creative research technique and critical reflection tool -an 'artistic experimental method'?My approach is to source germane and concise definitions for a set of artistic paradigms -beauty, aesthetics, authorship, process, serendipity and technology misuse -and then apply these as a framework of systematic measures to gauge, reflect on and draw effective conclusions from the outputs of a rolling series of artistic experiments.I hope to gather and then use this 'evidence' to drive the research forward.
While I'm still some way off a working set of definitions, I have made useful progress: • Serendipity -the art of making an unsought finding; the "art of loose blinkers" i.e. the talent and ability to combine systematic approach with intuition, surveillance and hunch; a type of explorative scientific method, a category that may be called abductive.
I expect these definitions will crystallise over the course of the research as my focus shifts -based on my current experience in building analogue tonoscope prototypes where I seem to be focusing on serendipity -the art of making an unsought finding -and technology misuse -the application of technologies in ways they were never In response I've tried to identify contemporary art practice that has similar foci and might critically inform my own research and think I have found a good example in the work of circuit benders and liberators such as Brian Duffy and his Modified Toy Orchestra who explore the hidden potential and surplus value latent inside redundant technology by converting abandoned children's toys into sophisticated new electronic instruments.

Open Source Knowledge
I believe the Internet has revolutionised the way we share and interchange -in large part due to the fact that it was designed by academics who reflected their own culture and economy of peer review and free information exchange within it's infrastructure.So I concur with Barbrook (1998) who argues, "The design of the Net therefore assumes that intellectual property is technically and socially obsolete."I contend this is amply illustrated through the rapidly expanding community of people worldwide who are taking advantage of new, cheaper technology and a growing pool of shared knowledge to make things for themselves.Open source development platforms and sharedknowledge projects such as Arduino and "how-to" websites such as Instructables are propagating technological know-how and tools previously the sole domain of large-scale, expensive, corporate engineering departments.As artists and the general population leverage these tools to redesign and re-engineer their environments, I suspect we are on the threshold of another 'paradigm shift' in cultural production not seen since the Industrial Revolution and the birth of mechanised industry.
So I plan to share my knowledge and insights with the research and wider communities through a decidedly ʻopen sourceʼ modus operandimaking my own evolving tool set, methodology, code and software, electronic and design schematics, documentation and outputs freely available under a Creative Commons Attribution-Non-commercial-ShareAlike licence.This lets others remix, tweak, and build upon my work noncommercially, as long as they credit me and license their new creations under the identical terms.Others can download and redistribute my work and can also translate, make remixes, and produce new work based on my work.All new work based on mine will carry the same license, so any derivatives will also be non-commercial in nature.

BUILDING PROTOTYPES
Key to the first stage of my research is the development of a refined and responsive prototype analogue tonoscope.While I could use other practitioners' recommended hardware and software configurations and follow tried and tested methodologies this runs counter to my research approach.My prototype analogue tonoscopes are driven by a sine wave generator (SWG) built from scratch -measured and calibrated using a customised version of a frequency measurement library distributed by Lab III -the Laboratory for Experimental Computer Science at the Academy of Media Arts Cologne -with visible outputs captured using a hacked PlayStation 3 Eye camera and lit by a custom-made visible/ultraviolet/infrared LED lighting rig synchronised to the output from the SWG.While it's taken me longer to build I've learnt much about the fabrication and control of these essential tools and, moreover, feel I've developed an intimacy with them which can only help my research be more serendipitous in the future.

Sine Wave Generator
I decided to develop prototypes that explored the sine wave generating potential of the Arduinopartly out of a curiosity in how far the platform can be pushed and partly out of expediency since it can simultaneously control the LEDs and camera.

Figure 2: Sparkfun Breakout Board for AD9835 Signal Generator
While there are several ways to generate sine waves with an Arduino I opted for the method of controlling a dedicated signal generating IC, since it'll produce the highest quality output with the minimum CPU overhead.Specifically I chose the Sparkfun breakout board for AD9835 signal generator driven by an Arduino Mega (1280).Despite comments that it wasn't trivial to programme I persevered with the example code and learnt much about SPI or Serial Peripheral Interface Bus and synchronous serial data; addressing and transferring data bytes to IC registers and using hex, binary and bitmath in the process.
By looking at a range of relevant software on OSX and iOS I was able to list the functionality and user control I might implement in a custom SWG and was soon able to: • change frequency via a DFRobot rotary sensor V2; • enter a frequency to two decimal places via a keypad; • switch between the AD9835's two independent frequency registers using a Force Sensitive Resistor (FSR); • save up to 9 different frequencies to volatile 'memories' in an array -and recall them via the HOLD event on the keypad; • save and recall up to 9 different frequencies via non-volatile EEPROM.

Lights, Camera, Sound
In reading Jenny's ( 2001  Inspired by the design of the Peau Productions Custom PS3 Camera Housing -particularly liking the use of magnets to hold the lid onto the base -I designed a circular mount built from several layers of 3mm acrylic incorporating some of its features.I then integrated visible, ultraviolet and IR LEDS into a series of interchangeable lids -each ring of 16 LEDS driven by a PCA9685 LED driver controlled by the SWG over I2C -effectively a USB cable.
Looking for high quality yet compact and affordable amplification I came across the family of Tripath/Tamps much moded by DIY audiophiles to produce high quality amplifiers with a low price tagspecifically the Sure 2X100 Watt TK2050.Although I' ve not yet modified the on-board components I have built a compact acrylic case.

FUTURE DIRECTIONS
As I use and iteratively refine these tools over the coming months I intend to define the emerging design of a sonically and visually responsive analogue tonoscope.Of particular interest is John Telfer's (2010) research, which provides much food for thought.Through the Augmented Tonoscope I hope to approach something of the artful mastery of Oscar Fischinger's synthetic sound production experiments -as Moritz (1976) observes: "Ah, but those visuals contain formulas and gestures that communicate with us subconsciously, directly, without being appreciated or evaluated."In looking for a cymatic visual equivalence to the auditory intricacies of melody, harmony and rhythm I suspect that I'm looking for something elusive and fleeting.I hope that a systematic search for the unfound -focused within the subtle nuances of the interplay of aesthetics and dynamics of the Augmented Tonoscope will allow me to catch more than a glimpse... and enable me answer my research questions.
music was visual, imagery musical and digital computers offered the possibility of algorithmically melding the two.I see a lineage for my research in the work of these early experimental and computer in film artists and their artistic experiments in Visual Music using the technology and mediums of their time.I plan to investigate their experimental methodologies and research critical consideration of their work to inform and guide my own practice.The study of visible modal wave phenomena has a long and rich history and a significant body of empirical evidence, theory and practical application most notably: • Dr Hans Jenny -Cymatics: A Study of Wave Phenomena and Vibration Vol. 1 (1967) & Vol. 2 (1972) -Jenny emphasises the "triadic nature" of Cymatics -hear the sound, see the pattern, feel the vibrationhighlighting three essential aspects and ways of viewing a unitary phenomenon and suggesting the potential for developing a multimodal sensory instrument; • Ernst Chladni -Discoveries in the Theory of Sound (1787) -Chladni's Law describes the frequency of modes of vibration for flat circular surfaces providing the essential mathematics for modeling the virtual physics of the emulated tonoscope; The work of these pioneers has inspired contemporary research into Cymatics as a means to reveal a deeper understanding into areas such as harmonicism and spatial form: • John Telfer -Cymatic Music (2011) -An audiovisual science and music project investigating the possibilities of creating a system of visual, or rather visible music; • Benlloyd Goldstein -Cymatica (2009) -An architectural thesis investigation exploring the synthesis of spatial proportion and form generated from sound.Goldstein cites Goethe -"Architecture is the frozen music; music is the flowing architecture…" as a rationale for his research -an interesting trans-disciplinary reflection of my own research.There's inspiration and insights to be gleaned from the realisation, process and methodology of a wide range of contemporary artistic output exploring Cymatics: • Gary James Joynes/Clinker -Frequency Paintings: 12 Tones (2011); Jan Meinema and Dan Blore -Cymatics: Liquids and Sound (2009); Carsten Nicolai -Milch (2000); Thomas McIntosh with Mikko Hynninnen and Emmanuel Madan -Ondulation (2002).
) combined volumes of Cymatics: A Study of Wave Phenomena and Vibration I've noted several intriguing ideas that Jenny mentions almost in passing but I think could open interesting avenues of investigation.One such notion is the use of stroboscopic lighting to 'freeze' cymatic patterns and forms on irregular surfaces; another is to generate cymatic modal wave phenomena in luminescent liquids.To pursue this I need a flexible camera and lighting rig working in a range of light frequencies.

Figure 3 :
Figure 3: Peau Productions -PS3 Eye cameraThe PlayStation 3 Eye camera is a popular choice for interactive digital art projects.It's cheap, very modable, and delivers resolution and fps rates (at least on Windows) comparable with higher end devices.So I decided to develop my prototypes using a cheap modification of a PS3 Eye camera fitted with a vari-focal lens.There are several detailed description of how to 'Hack the PS3 Eye'

Figure 4 :
Figure 4: John Telfer -the lambdoma matrix expanded to a 32 limit.
The latter is the belief that a relatively simplistic synchronising of visual rhythms to musical beats and a matching of musical and visual tone and mood is what constitutes good Visual Music.As an experienced curator in the field of music, sound art and technology I concur with Moritz's delusion of rhythm and agree, "Too much of what passes for Visual Music relies on just such false synchronization."But while Moritz was probably right in arguing in 1986 that "Computers (and lasers and video) are hardly a panacea for most artists" and "...no machinery can offer you a sense of graphic design, a sensitivity to color, or a sensibility for choreography and timing."I don't think this is true 25 years on with the unprecedented computational power, open source programming environments and a global database of knowledge, expertise and experience at contemporary artists' disposal.