The Use of Human Brain Activity (Electroencephalogram) in the Making of Art

Recording EEG from awake humans has been possible for nearly 100 years, with the majority of development being in medical science for the diagnosis and treatment of epilepsy. The potential use of the EEG as a way of reading or controlling another person’s mind has however become a staple of fiction writers for even longer. In recent years with the increasing power of computers and reduced cost of processing, small cheap, non-clinical EEG systems have appeared on the market. Whilst their use has been limited mainly to the computer gaming world, artists in many disciplines have incorporated the ‘output’ generated by these devices as part of their art project. Null Object: Gustav Metzger thinks about nothing: London Fieldworks and The mutual wave machine: Lia Chavez and Matthias Oostrik are two notable examples. A question remains though as to how much of the art is the EEG itself as opposed to that of the analysis, processing and presentation directed by the artist? In answering those questions I plan to demonstrate: Live recording of human brain activity (EEG) with emphasis on high quality broad waveform (wide filter) data acquisition and its manipulation by simple measure such as eye closure and hyperventilation and sleepiness; analysis of EEG data, both online and offline and how this can be used to control a number of peripheral devices such as 2D plotter, 3D printer, light projection and sound generation. In the discussion of the demonstration and the paper I wish to: show the potential of EEG recording and creativity; point out some of the common pitfalls in recording EEG data including, poor technique, artefacts and external noise, which can drown out any true EEG activity; discuss whether the best BCI currently available is the human hand; speak of the ability to over-manipulate the data so what is left is an alias of the original and has little or no bearing on real brain activity (a real danger with post hoc analysis or processing) and question the need for minimum standards of reporting methods when creating EEG art to allow critique and reproduction in keeping with general scientific protocol


INTRODUCTION
What is an EEG? What does it measure? The EEG is a measure of electrical current generated by the activity of brain cells, mostly large vertically orientated pyramidal cell neurones of the cerebral cortex. Scalp electrodes record in the main the summation of electrical changes from the immediately underlying cortex but also some changes produced at more distant sites in the brain and also signals from non-brain structures (muscle, heart external noise). The electrical currents arise as a result of a potential difference across the neuronal cell membrane caused by changing concentrations of electrically charges ions. The movement of ion across membranes through voltage gated ion channels gives rise to current flow, which when summated from many neurones can be recorded from electrodes within the cortex, on the cortical surface (electrocorticogram, ECoG) and on the scalp. It is important to note here that most recordable activity comes from the outermost layers of the cerebral grey matter, the gyri and not the deeper structures of the limbic system that are responsible for behaviour, memory, emotion and motivation. The scalp recorded EEG differs from ECoG and is most often distorted in amplitude and morphology. Short duration and fast activities (seen as higher frequency discharges) are attenuated and sometimes lost when trying to pass through the scalp. Thus the recorded activity from the scalp does not accurately reflect the activity of the whole brain, any one electrode may be recording small currents near the electrode or larger ones from further afield, most often both. A range of electrodes covering the whole cranium is therefore needed to accurately reflect the source of scalprecorded activity.
EEG and its associated technology MEG (magnetoencephalography) are unique in that they reflect the on going function of the brain and can show changes from millisecond to millisecond. They are both relatively poor on working out where their signal arise from, the former because deep signals are not recorded and higher frequencies are filtered out by the skull and the latter because technical properties of the SQUID magnetic recording devices not picking up activity from tangential dipoles. In contrast CT (x-ray) brain scanning and MRI magnetic resonance imaging have good spatial definition of structure but not function and related techniques (PET, SPECT) that use dyes and radioactive material have poor time resolution (several seconds) as they measure changes in metabolism or blood flow. We have yet to develop the perfect instrument to say when and where brain activity occurs during certain tasks.

RECORDING
The recording equipment measures the potential difference between two electrodes attached to the scalp. These are usually non-polarisable metallic silver/silver chloride or gold plated with best contact using a conductive gel. The importance is for very low impedance (resistance V=IR) to maximise the ration of the desired signal to the external electrical noise in the system. The raw analogue signal is amplified, converted to a digital data stream (analogue to digital conversion ADC) and filtered. The amplitude of this signal measured in microvolts is plotted against time to give the familiar EEG trace. Depending on the context up to 256 electrodes may be used to cover all surface areas of brain activity. 32 electrodes usually suffice for clinical practice; some cheap gaming systems have as few as one.
With ADC each time point is given the corresponding amplitude for each electrode with respect to the ground electrode. The sampling rate for conversion must be at least twice that of the highest frequency to be displayed as well known by sound engineers (this conforms to the Nyquist formula). If sampling is to low, the resulting digital waveform will not represent the actual signal. A visual example of sampling would be to draw a silhouette of the figure above using height data from measurements 50 meters apart. Any change in height above or below the line drawn between adjacent points would be lost, in effect reducing higher frequency representation. Most clinical EEG equipment samples at a minimum of 250Hz per channel, this allows frequencies of up to 125Hz to be accurately represented. Some research systems sample at rates of over 1000Hz and are therefore able to investigate high frequency discharges seen at the onset of seizure activity.
Filtering the data is often done to ensure only signal in the expected range are displayed. Over filtering and/or using a filter at a specific (notch) frequency to remove 50/60Hz mains noise can reduce the actual EEG signal again resulting in an alias of the true data. It should also be noted that a notch filter varyingly decreases the digital signal for 10-20Hz above and below that to what is set i.e., a 50Hz notch filter will reduce some of the activity between 50 and 30Hz.
The EEG itself is usually depicted in the time domain with the amplitude of the electrical signal plotted against time for individual electrodes or for all electrodes as a topographical 'brain map'. (Figures 2 and 3). Other ways of quantitising activity involve manipulating the data into the frequency domain (usually using Fourier analysis) and plotting spectral power against frequency.
Brain activity itself is divided into frequency bands alpha, beta, gamma and delta, 8-12, >12, 4-8 and 0-4Hz respectively. The 'inappropriate' order reflecting the order of their discovery. Higher frequency beta activities and very high frequency oscillations can also be recorded with appropriate equipment, more so with ECoG recordings.  The pattern of changes in the EEG recording of a normal person differs with age and sleep state and can be manipulated with hyperventilation and pharmaceutical agents. Diseases of the brain such as epilepsy, encephalitis and dementia all affect the normal EEG patterns. For clinical use, the visual inspection of the complete recording allows any parts of the record with excessive muscle or eye movement artefact to be ignored. In much scientific / psychological research further 'cleaning' of the EEG using automated detection of artefact with component analysis and reduction of brain activity associated with mini-saccadic eye movement is performed. The need for EEG data that is clean and reliable is important given the associated clinical decision. Type of epilepsy -affects treatment, site of seizure induction -affects surgery, whether there is any brain-derived activity -affects decision to withdraw life support etc.

DEMONSTRATION
EEG recording and presentation can be demonstrated, showing some of the pitfalls. Some variable factors include: scalp preparation, electrodes wet/dry, artefacts -eyes, muscle, other, sampling errors, filtering errors, reference errors, eye open closed. We can also consider the use of single frontal electrode and control of peripheral device.

DISCUSSION
In summary the EEG represents a major distillation of the total electrical activity of the working brain with a bias towards electrical sources nearer the surface. We should bear this in mind when making claims of certain mental states and their associated EEG pattern.
If we look at music as an analogy, and take two four piece bands, the 'EEG could tell us they both have drums, guitar, keyboard and voice based on the frequency distribution of the sound but not that one is Queen playing a rock anthem and the other Billy Holiday singing a ballad. Similarly, if we distort the signal of an orchestra` by cutting out very high and very low frequencies we may miss, violins, flutes and tympani (try listening to music underwater). So with currently available equipment the ability to condense the complexity of our cortical functions and thoughts into a line of numbers in computer code, some quantum of consciousness seems unrealistic.

The Brain Computer Interface (BCI)
The BCI has been at the forefront of scientific investigation in order for the brain to directly control an external device usually via a computer. Whilst medically this has been to aid severely disabled persons to communicate or move, potential military applications and computer gaming have played a part in development. In the US the defence and research authority has been one of the largest funders of high end BCI development whereas the gaming community has led the way in development of cheap alternatives (I will return to these later).
Use of the fact that alpha activity increases when eyes are closed is the basis for at least one robotic arm (Han et al. 2018).
Leaving aside the use of BCI for injured persons, for me the perfect BCI is my hand followed closely by my mouth, the former for typing and mouse control, the later to ask 'alexasirigoogle' to play music. In computer gaming the EEG non-manual control usually relies on eye movement or blinking so is really a brain-muscle to computer interface, as is the hand!

EEG and art
Why would we want to use the EEG as a basis of an art project? The main thrust has been the idea that it taps the unconscious, the creativity of our brains unencumbered by the thoughts and doings of quotidian life. This of course is not a new idea; automatic writing developed by the surrealists and taken up by DADA artists was one attempt to tap hidden thoughts or memories not unlike psychoanalysis. It should be emphasised that the rationale for development by the surrealists was in response to the abjection of war as many of them had served as soldiers. This is probably not the reason for the search for 'unconscious reality' by today's artists although on going political ineptitude and the rise of right wing hate politics has some resonance with Europe a century ago.
The output of the scalp recorded EEG as demonstrated is limited to a range of amplitudes that change with time. From this the different frequencies of activity and its position over the head can be derived. The meaning of any of this on going electrical activity remains elusive. The search for a particular pattern or signal that represents consciousness has proved to be beyond current techniques. The fact that psychologists and neuroscientists of many specialty areas have been unable to measure consciousness even using direct recording from the cortex using electrode grids of depth electrodes combined with structural imaging would give non-experts using inadequate equipment even less of a chance.
The chance to use new techniques, however, has always been part of the artist's repertoire as evidence by Leger's early film work (Leger 1924), Hockney's fax painting (1989) and more recently virtual reality. Aside from doing something new because we can what could we hope for using the EEG as a tool, trigger or data set in the creation of art?
If art falls within a definition of 'Something that stimulates human sensation' then the brain itself needs one or more conduits to make it. The brain itself it is a seemingly inert mass of tissue without obvious aesthetic beauty that does not move (with the exception of that secondary to pulsatile blood flow), does not produce sound or smell perceivable by humans, and since the end of the ritual of eating the brain of the deceased in Papua New Guinea there is no longer a living record of its taste! (This tradition led to the brain disease Kuru, a spongiform encephalopathy similar to Creutzfeld Jacob disease -which itself has a typical abnormal EEG pattern [Wieser et al. 2006]).
If we limit our discussion to art related to vision, sound and movement, for most artists the conduits usually responsible for expressing the artistic output of the brain remain our bodies especially the hands and voice. The final artwork is then effected by the whole gamete of artist's materials or noise making instruments.
This process of 'brain activity -intrapersonal conduit -effector' describes in most cases the usual practice of an artist. Use of the EEG to read the brain activity by passes the conduit of the body to directly control the effector. It has been suggested that various emotive states can be measured using EEG and the outputs used to trigger an effector to do something. There is no doubt that there must be changes in the brain activity whilst we think etc. the question is whether they can accurately be measured or inferred by scalp EEG in general and low spatial resolution systems in particular.

New EEG recorders
There are a lot of newer, cheaper electronic devices capable of recording EEG activity. This discussion should not to be used as endorsement or dismissal of any one over another as they have been developed for different reasons and I shall not be naming any brand. Some claims of what is being measured are possible not justified given the available evidence in published papers or the manufactures literature. Researchers may wish to inform themselves and make up their own minds if they proceed with their own research. There have been some studies looking into the ability of these devices to record appropriate signals for analysis. Monori & Oniga (2018), found that one of the single channel machines was only good enough to indicate blinking patterns and not reliable for other electrical signals.
Similarly a large well funded study of measuring EEGs in viewers at an art gallery showed up to half of all data was corrupt and not fit for further analysis especially with the use of dry electrodes (Cruz-Garza et al. 2017).
Some of the devices are not safety endorsed for use in humans and as such if connected live to a person, any other mains operated equipment, including computers, sound and light installations without a suitable 'isolation' system runs the potential risk of electrocution.

What is the output from the EEG we can uses as a trigger?
Any combination of voltage high / low triggers for a single or repeat events can work in just the same way an ECG machine beeps, or not. More complex online transformation of the signal including filtering, and transformation into the frequency domain is possible. Input triggers can then be set for changes in frequencies of activity. This seems the most common mechanism in the low channel number / single channel devices although the actual algorithms and code remain secret. Offline analysis forms the bedrock of clinical and research practice. The actual transformations vary, are often related to Matlab / Brainlab applications and are usually published for scrutiny and to allow reproduction of the experimental process.
Much detailed laboratory research has shown changes in particular EEG frequencies associated with different cognitive states, usually with multi channel recording and complex artefact removal. It is well known that alpha activity increase with eyes shut (Figure 4.), theta and delta activity increase when drowsy and we should have seen the former during the demonstration. Changes in nearly all frequency ranges have been reported with many 'states of mind' and in response to manipulation of the physical environment of the subject / participant / patient. Oscillations on gamma, alpha delta and theta activity govern nearly all cognitive processes (Basar et al. 2001) so to cherry pick one paper that shows a change in one frequency means a particular process happening within the brain and claim this is what the artwork shows is most likely foolhardy and perhaps a little disingenuous. A seen in the demonstration the biggest change in posterior alpha rhythm is closing the eyes.
The potential problem with few channels and particularly frontal only devices is that most alpha activity is generated at the back of the brain and will not be recorded. Also muscle activity from scalp muscles, when sampled at low rates and filtered to below 50Hz is transformed into activity recorded between 10 and 20Hz. Therefore muscle activity (not brain wave) will make it look like increase in alpha and beta activity as shown in figure. These outputs drive the 'effector' usually a computer-controlled device with script written to operate different electronic devices. Colour / sound coding for different frequencies. Whilst the aesthetics can be pleasing, claims that we are seeing the thoughts, aura or inner state of the participant and similar are misleading. EEG signals are electrical waves not thoughts and in some cases electrical muscle activity and not even brain waves. It is also apparent that the artist or computer programmer decides which colour is to represent which activity, a decision most likely cultural as will be the choice of sound or music. Who decided calm was blue and anger red, let alone whether whale sounds are calming? No one has the same playlist. This gives us a similar problem in understanding any true meaning of the artwork as that I imagine the artist and scientist Charlie Hooker has when deciding which sound to attribute to any macrocosmic or microcosmic event picked up by a Geiger counter or microelectrode; this doesn't take away from the production but can alter the interpretation and meaning, so designed or unintended (Hooker 2011). A further predicted problem lies in the analysis of the data using artificial intelligence / deep learning. If the data we put in is not really EEG we must ask ourselves whether any correlations have real meaning.

Artist examples
'The work Aura: A holographic brainwave interface' presented at this conference a year ago used a one channel EEG recorder, the output of which, according to the authors created "a colour-coded, ethereal sculpture on the Pepper's ghost display in response to the participant's thought processes, with cooler colours represent phlegma". The manufacturer of that system clearly and accurately points out on their website "our EEG biosensors collect electrical signals -not actual thoughts." Perhaps the authors were unaware of the limitations of the system they used. Either way whilst the display of the effector output (one channel on the forehead) was aesthetically interesting, we are not be able to tell whether the input to the system was brain activity or muscle activity (I favour the latter but remain open to other ideas). Was this really a person's aura? With any 'black box' technology it is usually true that if the input is poor the output in unlikely to be any better.
When using these techniques we should be careful not to overplay the meaning of our work especially if we do not have supporting evidence.
The mutual wave machine: Chavez and Oostrik (2014) show development of the synchronicity of the participants' brain waves. It is not clear which parameters are used to measure synchronisation and triggering of the sound and light installation, the video available shows considerable movement of facial expression and would again possibly lead to an assumption that muscle activity defined the synchronicity. The use of over ten electrodes placed all over the scalp, however, means that muscle artefact containing electrodes can be dismissed form further analysis and increases the likelihood that this reflects brain synchronicity. It is know that EEG rhythms become synchronised due to the same rhythmic, visual, sound or touch stimuli and a well controlled classroom study showed increased synchronisation of pupils that shared eye contact and concluded this was secondary to attending common sensory cues during the lesson (Dikker et al. 2017). Indeed, if I flash a light at 15Hz everyone who attends to it will have a synchronous posterior 'following rhythm', a technique used with steady state visual evoked potentials as a BCI with brain injured patients who only have eye movement.
Null object: Gustav Metzger thinks about nothing (2012) also used one channel EEG to 'trigger' a computer to drive a stone carving robot, collaboration with scientists working with EEG ensured quality recordings. The resulting void perhaps representing an empty mind. It would have been interesting to see what was carved if Metzger had been thinking about atom bombs (Fischer 2017). The artists however acknowledge this and with the accompanying written essays discuss whether that creativity can be informed by science and not controlled by it (Lambert 2012). The critique of these examples, I hope highlights the need for good collaborative research if artists using science in this or similar ways wish any significant claims to stand up to a degree of scrutiny more often associated with 'the scientific process'. Also the latter of the three demonstrates controlled caution in what the process actually shows.

FINAL WORDS
Our basic EEG output is usually just a rhythm, the difference in timing and the duration and amplitude of the event. The rest is up to us. Don't overlook rhythms though, they underlie much human behaviour and emotion are probably the only things that can be seen, heard and felt whilst having the same meaning.