Infinite Remix Machine : Automatic Analysis and Arrangement of Musical Recordings

We live in a world where music can be remixed and re-imagined with a few simple button clicks, but at what point can this process stop requiring human input totally and work in an algorithmic fashion to explore machine creativity? The aim of this research is to investigate whether a machine can automatically analyse music and use the data to effectively combine it with other music to create a previously unheard piece.


INTRODUCTION
We live in a world where music can be remixed and re-imagined with a few simple button clicks, but at what point can this process stop requiring human input totally and work in an algorithmic fashion to explore machine creativity?The aim of this research is to investigate whether a machine can automatically analyse music and use the data to effectively combine it with other music to create a previously unheard piece.
Using freely available music and sound data, from http://www.freemusicarchive.org and http://www.freesound.orgrespectively, we are utilising a range of music information retrieval techniques (Bello & Pickens 2005, Aucouturier & Pachet 2002, Pohle et al. 2009) to analyse the data (including tempo, harmonic information, and others), and to design a set of context-aware algorithms to support automatic selection of relevant "segments" of music, and different approaches to remix the selected segments to create a completely new piece.Bello & Pickens (2005) showed that it is possible to represent a piece of music as a sequence of twelve-length vectors (representing each the presence of each semitone in a beat) obtained from a chromograph.This information can be used to estimate basic chord labels and calculate similarity to other pieces.

RELATED BACKGROUND
Aucouturier & Pachet (2002) discuss the "interestingness" of the association of music titles that came from their methods of assessing music similarity using the timbre of music to automatically identify instruments and found their system matched up songs that differed in genre, artist, dates of production, and even in cultural background and country of origin.This idea is particularly useful in this research as we are trying to remix music that is as varied as possible but to retain a high level of cohesiveness.Pohle et al. (2009) were able to achieve a 90% success rate in genre classification (their benchmark for music similarity comparisons) by combining timbral information with their own additions to traditional rhythm detection methods such as using fluctuation patterns, which can aid our algorithm in making better informed decisions when selecting appropriate audio segments in the remix process.

DESIGN AND DEVELOPMENT
The idea is to create several agents that examine the pieces of music and find salient points in the music based on musical changes such increase or decrease in tempo or sudden changes in loudness.This information is used to split music into smaller clips that can be altered and combined with other clips to create a new piece of music.We can also use the extracted data to analyse existing music and create templates in which structures of the outputted remixes can be based using machine learning techniques.Optionally, the system can take a small set of input stimuli is from the user which can "inspire" the machine to create remix on demand with certain features, such as, if the piece is fast or slow, or, or happy or sad.The system is being prototyped in Python and utilises existing audio analysis libraries (Pham 2006, Robert 2011, Brossier n.d., Cannam & Figueira 2011).
For the development of the prototype we are adopting an agent based design.By creating one agent per musical attribute (such as tempo) the system becomes scalable and features can be added as research continues as well as become adapted for high-performance (HPC) and parallel computing.In contrast, the generation section of the system looks at a "composition template", such as Figure 2, (overall outline of how the music's dynamics and tempo should change over time) and using the template to grow and mix appropriate audio segments, in order to create a new composition.

CONCLUSIONS AND FUTURE WORK
This is an ongoing project and we are currently finalising algorithms in arranging pieces based on music similarity.Final evaluation of the planned prototype includes procedure similar to a 'Turing Test' involving a group of test users listen to remixed pieces by the system and also by human DJs to see if they can be correctly distinguished as remixed by machine or by human.With such a system, it is hoped that data could be harvested from the web automatically and continuously (with permission and appropriate rights considerations) and will utilise big data techniques and HPC to process the large amount of data to offer a set of almost limitless creative possibilities.

Figure 1 :
Figure 1: Overall system architecture Music information retrieval techniques consists of a set of algorithms designed to extract music information, such as tempo and harmonic content, and segment the audio based on this information.This information is used to represent audio segments in a music-similarity space.