Keynote Paper on New Constructs in Art : Structure, Time, Correspondences and Interaction

– The paper provides a personal view of the implications of computational concepts and technologies on art practice. It draws upon 40 years of such practice by the author. As the title implies, the paper concentrates on the themes of structure within the artwork, time-based art, correspondences between different media that the computer enables, and interaction with the audience. The paper draws a line from Malevich to present day art employing computation. CONTEXT Few would dispute the role of Cézanne as the father of the early 20th century revolution in visual art. His insight into art practice is especially interesting to me: The technique of any art consists of a language and a logic [1]. That statement captures the core contextual point of the discussion to follow. More fundamentally, it underpins the nature of my thinking about art in bringing a focus to the close relationship between the act of making and the structuring of ideas. Another significant grounding point for me is in Malevich's essay of 1919, On new systems in art [2]. In it, he introduces the notion of making art with the help of " a law for the constructional interrelationships of forms " , by which he meant the language, or system of form, rather than representations of the visual world. Furthermore, he maintains, " …nowhere in the world of painting, does anything grow without a system " and, " in constructing painterly forms it is essential to have a system for their construction, a law for the constructional relationship of forms ". Patricia Railing, in From Science to Systems of Art shows how Malevich looked towards science to give a formal basis to his art. The advancement of such notions was widespread in the 20 th Century: for example, in a discussion of the influences on British Systems movement, Stephen Bann says: 107


EVA 2009 London Conference ~ 6-8 July
Ernest Edmonds _____________________________________________________________________ It is not the recurrence of the rectangle in Van Doesburg's work which needs to engage us, but the series of relationships between rectangles and the extent to which those relationships can be adequately formalised.[3].
For me, the creative life in art is in the development of systems and constructs rather than in the application of previously defined ones.Margaret Boden, in a discussion about the nature of creativity, maintains that the creative step is in changing the rules of the current style or approach rather than in finding a new application of them [4].Malevich, in assessing the effect of rules on art making took the view that "When the laws of perspective were set up in imitative art, a binding chain was established.A stall was set up in which the artist was expected to operate".Not an attractive situation for a creative mind.
My work and the significance of digital technology in its composition, is an example of an art form closely entwined with the creation of new constructs and the techniques that enable them to be realised.One such example is the introduction of time, a significant new element in visual art.Time in itself is not new to art form; it became widespread with the invention of film and caused considerable controversy in the early life of that medium.Malevich was part of that debate in the early years of Russian cinema.Very briefly, the argument was about how film should take its course.Should it be developed from theatrical conventions or from those of painting?Malevich put strong arguments forward in favour of the latter [5].In the 20th Century it seems that theatre held sway although perhaps now that dominance is being challenged.In any case, the element of time that came from the use of digital technology in the visual arts has certainly arisen from painting.Static visual art has found a new construct.This will be expanded in what follows.
Figure 1 is the first work that the author made in which software played a part.

STRUCTURE
When I use the word structure, I mean some kind of formal organisation such as the musical fugue, the geometric proportions of the golden mean, or the rhyme scheme of a sonnet.I am interested in such structures in two respects: first, the use of them is one way in which I can make global decisions about a work and reduce the specific choices.In making a work, the apparent freedom of the almost infinite range of possibilities is a difficulty.As, for example, Stravinsky claimed, true freedom comes with the application of constraints [6].After all, how is one to select from the infinite?This is, for example, the problem of facing the blank canvas or the clean sheet of paper.It is necessary to find a way of reducing the options to a manageable level.One way of doing this is to select a structure to which the work will conform.In fact, the choice or design of the structure is a major aesthetic decision itself.Creative steps may be found in inventing a new structure, for example, serial music.The second respect in which this is interesting is that constructive psychology, and the very existence of science, suggests that the search for order is a fundamental attribute of human perception: the face seen in the stain on the wall.It is possible or even probable that structures in artworks are very significant in our aesthetic experience of them.This may be so even if we do not know or notice the specifics of the structure.We often say that a work has a satisfying form, such as in a final movement to a symphony, even when we cannot quite explain what caused it.It is as if we respond to the architecture of a piece of music or a novel before we can unravel its construction.

TIME
To many, music is the highest art form.The purity and yet the power of these abstract structures in time seem to cross cultures and ages and are often at the centre of, or perhaps pointedly excluded from, religious worship.Not surprisingly, painters quite frequently aspire to being composers or musicians.Formally, the distinction between seeing and hearing aside, the key difference between painting and music might be seen to be the lack or presence of time as an integral element or dimension.Vertov's characterization of his film The Man with the Movie Camera, as an experiment in 'visual music' [7] perhaps captures the visual artist's interest in time.However, other influences have also been active in the 20th century.For example, in art that uses geometric or other systems, it has been common to produce series of works that often have a natural sequence.It is only a small step to think of them as stills from a movie.Another closely related development was the early use of computer programs to generate drawings.Works of this kind lend themselves to the automatic production of series because the computer program is a kind of general structure or form that can apply to a class of works, each a small variation of another.Calder's visit to Mondrian's studio in 1930, when he conceived of making the visual elements he saw move, was another inspiring moment.One way or another, adding time to the kind of visual art to which structure was important seemed appealing, even tempting, to me.This led to my development of what I have called 'Video Constructs' which use logic programming to construct time-based work.
My insight was this.Logic Programming [8] can be used as a method for handling structures in time by visually representing the search process.It can be used to make generative work in which the rules specified in logic control the form and order of a sequence of images.The sequence can go on forever without loops, depending on the rules.The logic specifies both the structure of each individual image and the structure in time of how the work unfolds.The details of this depend upon a particular way of using logic in computer systems known as Logic Programs.In this method, a set of logical statements (in this case about the design of images) is interpreted as a program, which instructs the computer to search for some specified goal or state (in this case of the image).An important element of Logic Programming is that it includes 'backtracking', where, when certain rules have been tried and fail to get to the goal, the computer goes back and looks for alternative ways of using the rules.In my Video Constructs, this process of backtracking is used to generate an unfolding search and the artwork, the image sequence, is a trace of this search.
In generative time-based art, the explicitly defined part of the work is the structural element including, specifically, the rules that are to be used to determine in which order and at which pace the image sequence should develop.Although random or pseudorandom elements could be employed, my work is entirely deterministic.To my mind, the complexity of these works leads to enough uncertainty without adding randomness as well.In comparison to music, this work has a relationship to the works of early Boulez and others who took the serial music concepts to a more extreme level than Schoenberg and his school, by including more musical elements in the structures that the serial forms defined [9].Just as the images have an underlying order about them EVA 2009 London Conference ~ 6-8 July Ernest Edmonds _____________________________________________________________________ based on geometrical and colour relationships, so the Video Constructs also have an order in their time progression based on the generating logics.

CORRESPONDENCES
It is not difficult to see a relationship between time-based visual art and music.In my work, the interest in time came from music in the first place.So, adding a concern for music is a step back to the starting point.In some respects, the combination of the two is normal in 21st Century Western culture.When we watch a film we accept music as a natural part of the work.More generally, the sound track is recognised as a crucial element in the quality of the film in its total sense.However, at times the music is thought of as an accompaniment to the visual element, whereas it might alternatively be thought of as having equal weight and importance.In popular music as it evolved in the 1960s and 1970s, the opposite situation has often occurred, where a visual element was added either in parallel to the music or derived electronically from it but, in either case, again as an accompaniment.Later, the 'pop video' developed as a new form that extended the music and was often as elaborate and significant as the music that it was intended to promote.Such videos extend, but primarily illustrate, the music that is their source.To me, much the most interesting integration, however, is where the music and the visual element are equal so that, for example, one can see a visual display as one instrument in piece in which other instruments, such as violins, happen to produce sound.The composition of such work can begin either with the music or the visual or swap between them.Alternatively, it might begin from some more abstract description or notation that can be mapped into either sound or image.
The idea of integrating sound and vision in art goes back at least to the early 1700s with Louis-Bertrand Caste's Colour Organ [10].Many have been inspired by Baudelaire's poem Correspondence, in which he speaks of "…Perfumes and colours, answerable sounds…Joining to form a deep, mysterious whole…".Malevich seems to have thought about it too.There is considerable interest in this integration today.The advent of the computer has enhanced the possibilities in this area enormously.There are two key reasons for this.One is its ability to control the real-time production of sound and image with considerable flexibility and speed.The second reason for the importance of the computer is that it is basically a symbol manipulation machine and so it is able to take a symbolic form (such as a score which is a symbolic representation of music) and automatically work with it by, for example, transforming it according to a rule.In my work, the idea is to operate with structures that can mediate between sound and vision, so that a unified work can be produced that integrates both.As well as fully composed pieces, it is possible to introduce improvisation or other performance attributes.For example, the sounds made by a musician can be analysed in real-time so that an extra note or a change in rhythm can be fed into the image generation system and influence its development.In this way an improvised duet between a sound instrument and an image instrument becomes possible.

INTERACTION
When we look, read or listen we know that we are engaged in an active process even if we do not physically move at all.Perceiving art is an active constructive process.In this sense, and in others, the audience is engaged in a creative pursuit as well as the artist.This was quite a hot topic in art during the 20th century and it was particularly of concern when I first started using computers in the late 1960s.The concern was partly political.It seemed to many inappropriate for artists to play the role of the all-knowing creative being handing down their work for the pleasure and education of passive masses.Artists holding this view wanted to give the audience their own part of the creative action, not just in their perceptual mechanisms but also as an explicitly recognised part of the creative process.Apart from this 'political' view it was also current wisdom that engagement and interaction had a positive part to play in any creative activity.Thus, participation in art was considered to be important.Learning by doing, interactive science exhibitions and so on were very popular concepts.However, notwithstanding 'Happenings' and exhibitions that invited the audience to play, participation was much easier to promote than to achieve.This is where the computer came in [11,12].
Considering the computer as a real-time control device, given that we can specify rules for how it is to respond to external stimuli, it can be seen to put behaviours into effect by taking in sense data and controlling output devices such as video projectors and speakers.The work done in Artificial Intelligence and on Robotics in simulating intelligent thought was an important inspiration for this kind of work.Theoretically, Cybernetics and General System Theory offered interesting ideas about how computational systems could be used to model and implement animal-like behaviours.These fields seemed interesting, therefore, in the development of engaging participative artworks.The computer offers something quite new in respect of enabling interactive artworks to be made.This new opportunity does not depend on agreeing with the form of Artificial Intelligence that believes that the programs can stand as some kind of scientific theory of human thinking, but some of the techniques developed in those studies are certainly valuable.The questions come down to quite normal ones for the artist.How to think about and how to make the work?How to enable engagement and sustain audience interest?How is one to make interaction meaningful?What makes interaction meaningful and how does that influence our understandings of ourselves?I described Video Constructs under the heading of time.These have been developed into 'Interactive Video Constructs'.The artwork reacts to events detected by sensor systems.An image analysis system that analyses the scene as the pictures are captured is incorporated into the generative program.The performance of the work, i.e. the generative path that it takes, is then reactive to what participants are doing.A Video Construct is searching through a set of rules and, as it does so, it generates the sequence of images that form the output of the work.Each image represents the state of the search at that moment.In the earlier systems, the sequence of states was entirely determined by the 'search strategy' used by the software to explore the rules.In the interactive case, however, the 'search engine', that automatically operates the search strategy, has available to it a stream of data that is a coded, or symbolic, representation of the EVA 2009 London Conference ~ 6-8 July Ernest Edmonds _____________________________________________________________________ behaviour of the viewer.This data modifies parameters in the search and leads to a perceived sense of reaction by the system to the participant.The Video Construct changes its behaviour in the light of its experience with human participants interacting with the work.As it learns it changes the way that it develops rather than simply changing the stimulus-response rules that govern its behaviour.The 'Learning Interactive Video Construct' is an art system that evolves in response to participant interaction with the work.The most recent work in this mode has been the Shaping Form series (figure 2) where the colours, shapes and behaviour of the work slowly develops over time in response to the movement and presence of people.

NEW CONSTRUCTS IN ART
Malevich's insight into science and systems came well before the advent of computers, but at a time when mathematical logicians were developing theories that proved to be very important for the invention of the computer.This work on mathematical logic was investigating particular kinds of system, known as 'Formal Systems'.These are sets of precise rules that apply to finite collections of objects rather like the rules of chess and the chess pieces.Such systems are fundamental in computer programming.A computer is basically a logical machine that manipulates systems of symbols.The symbols have formal rules that determine how they relate to one another, rather as the pieces on a chessboard can be placed in very many different arrangements, but only ones in which certain rules are obeyed.When a computer manipulates such a system, it automatically 'moves the pieces', keeping within the rules, and searches for a result, such as checkmate.Another example of such a system is a collection of shapes with defined correspondences between them, such as a Lego TM set.In effect, the computer can re-arrange the pieces in any way, but only so that the rules, that determine how they fit, are followed.
The most significant property of the computer is that it can perform any task that can be defined for any formal system to do.It is a General Purpose Logical Machine.In one sense a computer can do anything.However, that statement begs the question of how we define whatever it is we want the computer to do.The computer can do anything EVA 2009 London Conference ~ 6-8 July Ernest Edmonds _____________________________________________________________________ provided that we can define precisely how it should be done.Writing those definitions is the art of computer programming and it is well understood that programming, at least in a non-trivial way, is quite difficult.For the artist therefore, as for anyone else, using the computer is quite a challenge.However, it provides a unique opportunity to express structures and relationships in one, two and three dimensions as well as in time and between different media.
The outputs that a computer generates, be they texts, sounds, three-dimensional objects, drawings, or movements, are generated by a mapping from a set of internal symbols to a set of physical entities such as marks, sounds or actions.The internal symbols, in turn, are determined by the operation of a formal system.For all its ability to calculate quickly or put images on screens or paper, it is the operation of formal systems that is at the core of what is unique about the computer.
For artists with an interest in the structure of their work and in systems, the computer was a natural and irresistible medium to explore.Whilst a role for logical systems in art is possible and interesting, the concrete realization of formal systems in logical machines, i.e. computers brought new possibilities into play.The formal systems became dynamic as, in effect, they became expressed in computer programs.Whereas before formal systems were abstract and manipulated by human action, now a machine existed that could perform such manipulation automatically and unimaginably quickly.It became possible, for example, to build machines that could play chess.This meant that an important step forward, from Malevich's concept of systems in art, was taken by the artists who adopted programming in the 1960s.Each of them approached the subject from his or her personal point of view and not all of them would use the term system in relation to their art, but if they programmed a computer they certainly used a formal system in practice.By developing new formal systems in our work we have been developing new constructs in art.
This discussion is primarily concerned with inventing new constructs for art practice and new structures to underpin artworks.The notion of structure, as used here, is of a recoverable system in the sense that it is in theory possible to look at the final object and work out the structures that led to its creation.Whether anyone ever would, of course, is another matter.It is theoretically possible but practical often just not a reasonable thing to try to do.So an artwork formed in the ways discussed above has a clear structure and, being clearly predictable, is highly constrained in what it does.That, nevertheless, does not mean that it is boring in the sense of the audience or participants knowing actually what is going to happen next or how it works.In the example of video constructs, the explicitly determined part is the generative method defined in the program.However, the whole point is the actual output: the generation of the visual sequence, the images, rather than the program.
In one case above, the concept is to create a communications network as an artwork in which the participants become engaged in a process where they try to impose a structure, or order, on the pattern of events that they experience.In video constructs, the underlying logic is used to provide a structure in time, which is the basis of the generation of the works.In the correspondences, an underlying structure is used to unify EVA 2009 London Conference ~ 6-8 July Ernest Edmonds _____________________________________________________________________ vision and sound.For each 'instrument' the appropriate structural elements are mapped into given sounds or images.
The new constructs in art that have been discussed could only have come about as a result of the invention of the computer and the development of the understanding of computation that preceded it.Remarkably, the direction that artists such as Malevich were going in during the early part of the 20th Century pointed directly towards these, at that time, unimaginable new directions.The introduction of the computer into art practice has transformed both that practice and the forms of the resulting artworks.It has certainly revolutionised my practice both conceptually and technically.
Exploring correspondences between the audio and the visual elements of my art continues to be important.In addition, my practice is increasingly concerned with interaction, even more than it was thirty five years ago and as presented at the CG70 Conference at Brunel University.A newer explicit concern is that of location: making art systems for a variety of public and private spaces.
As a result of the work of recent decades issues about the nature and effect of interaction have been clarified.They point to new directions and new art constructs.To take a key example, it is important that the audience recognises that the art system is responding to them and so an immediate recognisable interactive reaction is required.However, simple reactive systems are not very rewarding and certainly do not lead to sustained audience engagement.Hence another dimension is required.I call this response.My current work is exploring long-term responsiveness to the audience by the art system.These audio-visual works instantly react to the presence of the audience, and are therefore interactive in the direct sense, but they also develop changing behaviours over time as a result of their interchanges with active audiences.
By their very nature, interactive works are appropriate in locations outside the traditional art gallery.I am particularly concerned with the possibilities for them in public spaces, including galleries, where the audience becomes active through its chance or unconstrained encounter with the work.I do not make use of headsets, special gloves or any other such device as these require an organised entry into the space of the artwork.The use of such devices avoids one of the issues of interaction in art systems: forming audience engagement in the first place.
The concerns of my current practice require experimental studio spaces in which to work and, most particularly, experimental locations in public spaces.Research is a significant part of this art practice.To this end, I have extended my current studios in Sydney by adding an experimental interactive space, Beta_Space, in the Powerhouse Museum Sydney.This is one step on the road to understanding the implications of public locations for interactive art.
The stimulus of the computer for the invention of new constructs in art is far from over.In fact we are only now advanced enough in the technology to really begin this exploration of, as Cézanne put it, the "language and a logic of art" that digital technology has enabled.

Figure 2 .
Figure 2. FROM SHAPING FORMS by Ernest Edmonds 2009