Airborne Inscription: Writing with drones

This paper will detail the technical and conceptual aspects behind a developing art project entitled Waveform. This project draws together the technologies of airborne drones, digital image analysis, and automatic text generation to investigate how the observable world is represented through digital sensors—a key issue in the monitoring and depiction of ecological and climactic change. Specifically, the project involves capturing airborne images of coastal shorelines, mapping and analysing the outlines of incoming waves, and then incorporating the resulting data into the generation of text resembling free-verse poetry. The process is not autonomous, and is subject to human intervention at each stage, with the generated poems being curated to engage themes concerning coast, a changing climate, and scientific knowledge-making. The intention is that the process and its outputs can provide a vehicle for deconstructing the remoteness often associated with the aerial perspective—with its connotations of erasing the ambiguities and nuances of the lived experience of a given environment in favour of the absolute and the abstract. In this regard, Waveform is presented as a practical instance of how of airborne imaging and digital sensing might be recast in ways that can resist the prevailing discourses of precision, omniscience, and control that so readily attach to them.


INTRODUCTION
Commencing in the spring of 2017 Waveform is a writing project that draws together the technologies of airborne drones, digital image analysis, and automatic text generation. The project initially involves using a camera drone to capture images of incoming ocean waves from a vertical vantage. These images are then analysed using a program that detects the threshold between land and ocean. The coordinates drawn to mark this wavering boundary are then used as a source of variables in another program that generates text resembling free-verse poetry. The source vocabulary for this generator is curated so as to produce texts that evoke aspects concerning the coastal environment, climactic change, or the practices of measurement and classification in a scientific context. Each stage of this process yields distinctive visuals that are placed together in sequence, and it is this that constitutes the project's primary output.
The goal in developing Waveform is to present an assembly of artefacts, processes, and representations that can inspire questions concerning the use of sensory systems in monitoring the natural environment at a time of profound ecological stress and disruption.
At the time of writing, Waveform is still at a relatively nascent stage of its technical and conceptual development, and has yet to be widely exhibited. It is therefore relatively modest in its current critical and creative aspirations, and is still subject to continuous improvements. Thus, this paper is presented not as an exposition of where the project may eventually sit within a greater field of literary or artistic endeavour, but as a narrative account of the various conceptual inspirations that are underpinning its development.

TECHNOLOGY
Prior to detailing the conceptual background of the project, a further word on the technology behind Waveform, the creative processes involved, and the outputs resulting.
As noted, the first stage involves using a commercial camera drone to orbit at low altitudes above ocean waves as they impact the shore, taking a series of still images depicting these interactions from a top-down perspective (see Figure 1).

Figure 1: Initial drone image (taken at 45 feet altitude).
The criterion driving the capturing and selection of source imagery for the project is predominantly the levels of contrast present in the scene-between the white of the incoming wave crests and the dark sands of the shoreline. This is crucial because it is in detecting this contrast that the image analysis software attempts to delineate the threshold between land and ocean.
To explain, the image analysis software (developed using the open source Processing toolkit) divides a grayscale source image into a grid of 64 sampling fields. These fields are then mapped according to their overall brightness values. The modal brightness value of the sampling field is then calculated, with the exact RGB value being matched according to a predefined 8-bit index pallet, so as to facilitate comparison with the other fields (see Figure 2).
The comparison process involves assessing the contrast levels between a given sampling field and its surrounding neighbours. Those passing a certain threshold along a linear path are then designated as marking the edge of the incoming wave. The coordinate values generated are then stored in a two-dimensional array, which is used to draw a wave outline upon the original source image (see Figure 3). This outlining process is uncertain by design. In cases where the source image exhibits a high level of contrast between wave and shore, the resulting line is tightly conformal. However, the very lack of a hard boundary, coupled with the relative simplicity of the analytical routine, ensures the wave outline can oscillate considerably due to the contrasts generated by the presence of rockfalls and patches of sea foam. A simple smoothing routine attempts to eliminate outlier values in order to minimise sharp spikes in the final outline.  It is the coordinate values generated in this process of drawing the wave outline which are then passed on to another program (also developed with Processing) that generates the final textual outputs. The relationship between these values and the resulting text is that they constitute a supply of random variables for the program, providing the dynamo for its linguistic constructions.
Stated concisely, the generative program is built around the principle of a Markov-chain. A source text is scanned to detect particular frequencies of word groupings within it, n-grams, before building a table recording the relative probability by which they occur in relation to one another. Selecting an arbitrary n-gram will suggest those most likely to follow, which then determines the other probable options for the next link in the linguistic chain. Making random selections from across these interlinked possibilities, weighted in accordance with their probability distribution, can result in outputs that echo the coherency of the source, but which present the semantic juxtaposition can be novel and provocative.
It is in this context that the arbitrary values supplied by the outline coordinates can be deployed for the purposes of generating coherent textual outputs. Once the initial n-gram to build the linguistic chain is chosen arbitrarily by the generative program, the rest are determined by the coordinate values. As a consequence, myriad different strings can be assembled with each execution of the program, and it is left to the artist's discretion which are finally associated with the original source image. Each string generated is automatically divided into short lines of between three to six syllables each, and presented as an overlay on the source image, along with its geographic coordinates of latitude and longitude (see Figure 4).

Figure 4:
The final generated text. The source for this string was Alfred North Whitehead's The Concept of Nature (1920).
Given the potential for many different output strings, with the consequent demands of curating only one for final display, care is necessary when selecting the source texts supplying the vocabularies used by the generative software. The choices made encompass a mixture of fictive and factual works that have the potential to engage cogently with either the maritime aspects of the source imagery, or the conceptual dimensions of the Waveform project as a whole-which will be detailed in the following section.
To cite a few examples of the texts used by the project: R. M. Ballantyne's The Ocean and its Wonders (1874), which is a lyrical maritime history; Paul Edwards A Vast Machine (2003), reflecting on the evolution and politics of climate modelling; Alfred North Whitehead's The Concept of Nature (1920), and Annie Dillard's Teaching A Stone to Talk (2003), a meditation on how the natural environment is variously captured in the fields of science and literature.

Sensing the Anthropocene
The inspiration for the Waveform project lies in the 'Anthropocene' hypothesis developed by Crutzen and Stoermer (2000), in which the Earth is characterised as entering a new geological epoch.
Here, the impact of human activity on the material ecology of the planet is seen as being fundamental, in the form of changing climactic patterns, rapid losses in biodiversity, and nuclear residues.
Although the Anthropocene has yet to be formally ratified by international bodies (as of spring 2018), and serves primarily as a classificatory marker, its very existence implies a need for humankind to reassess its relations with the observable world. In the critical and creative arts, the Anthropocene has become a term deployed with increasing frequency as a catalyst for just such a rethinking, and, indeed, a remaking of human relations with the material structures and processes that characterise the lived environment (see, e.g. Davis & Turpin 2015).
One area of concern for Anthropocene scholars is the role played by sensory systems in defining, mapping, and indeed, constituting this new epoch. As pointed out by Bolen, Scott & Yang (2016), 'ideas of the Anthropocene have been shaped by a technospheric net of innumerable satellites, cameras, and detectors, resulting in an aesthetic regime composed of data that has been used to narrate profound changes to climate, landscape, and biodiversity over the past 400 years'.
At one level, this interest involves assessing not just the basic functioning and deployment of sensory systems, but also the epistemic assumptions encoded within them, regarding the nature of that which is being observed, and the kinds of representations produced subsequently. It is here in particular that Bolen, Scott & Yang (2016) enquire: If quantification, abstraction, and the logic of evidential traces have been the means by which we've largely come to recognize our purported Anthropocene condition, then the question becomes how we might proceed so that our "sensing" is less "remote," and forge aesthetics that incorporate not only the representational, but also the lived and affective experiences of various anthropo-scenes.
The chief contention here is that sensory systems, as they have been used to map epochal changes in climate and ecology, have generated representations of the world in terms of the purely quantifiable, and the abstract-as statistical trend lines on graphs, rather than in terms of the lived experiences through which these manifest. While the utility of such actions for the modern scientific method are not disputable, the implication here is that to craft a mode of 'sensing' that draws attention to the latter can serve as part of a more responsive ecological praxis across wider culturewithin the contexts of raising awareness of the complex impacts resulting from rapid planetary change.
As an illustration of the kinds of immanent perspectives that can be missed by this data-driven episteme, Schuppli (2014) notes how indigenous peoples living in the Canadian Arctic have observed the sun setting further into the West, and the alignments of stars appearing to change: Sunlight is behaving differently in this part of the world as the warming Arctic air causes temperature inversions and throws the setting sun off kilter. Light is bending and deceiving eyes that have tracked the position of the sun for generations, using it as an index of place and a marker for direction. The crystalline structures of ice and snow are twisting and morphing, producing a new optical regime borne out of climate change and indigenous observations. A documentary film made to capture these stories was subsequently met with some hostility by parts of the scientific community, who rejected the credibility of these indigenous accounts on the basis that they had reached the wrong conclusion as to the causes of the changing light-that the Earth had tilted on its axis (Schuppli 2014). While such views might indeed be scientifically inaccurate, the experiences leading up to them remain cogent for understanding (and, indeed, empathising) with the far-reaching entanglements and implications of human collective agency as it intersects with global geophysical processes. Cosgrove (2001) documents exhaustively the varied drives behind perspectives that emphasise the global and the absolute, noting especially how, in a post-war environment: The sense of closed global space that had disturbed the strategic thinking of so many Westerners at the beginning of the twentieth century found expression in a global geopolitics of competitive imperial struggle as the frontiers of Western empires converged at the ends of the earth. Mapping the globes climates and physiography into "natural" regions could naturalize patterns of control and strategy among the imperial powers. Geopolitics suggested that unalterable geographic "realities"-the distribution of lands and seas, of landforms, natural resources, or "races"-had to be exploited if a state was to survive, compete, and prosper.
Cosgrove observes how these attitudes were facilitated by infrastructures of sensing and data collation that present a totalising, top-down view of the world-such as provided by orbiting reconnaissance satellites, and the military drive towards establishing a 'God's Eye View' over the battlespace environment. The resulting impact of the energy and resource intensive economics of geopolitical competition are seen in the key ecological and climactic markers of the Anthropocene, which thus becomes in-part a consequence of measuring and representing the world purely in the terms delineated by global political and capitalist exchange.
It is in response to such developments that Müller-Hansen (2016) asks how we might engage in the 'process of making possible a new sensorium, one that is better adapted to the world as it is and behaves?' He notes subsequently that 'Sensing has its own politics' and that 'how we construct, train, apply, and critically engage with this new sensorium' will be vital when intervening in the structures of power that are most implicated in key ecological stressors. For Müller-Hansen, the Anthropocene thesis represents 'a call to re-forge our sensory-aesthetic practices, so that we sharpen our powers of judgment with respect to the epochal transformations currently underway […] a call to re-forge our sensibility toward the Earth'.

Developing a response
Evocative though such rhetoric of a 'new sensorium' may be, questions arise immediately as to what this might concretely resemble in practice. The basic suggestion is that it will document life within a material ecology subject to accelerating change and disruption, but how this could be achieved is left open to speculation.

371
Existing practices for capturing lived experiences are certainly available here, whether in the form of humanistic analyses, sociological studies, cultural geographies, or, in the case of the creative arts, film, image, literature, and performance. One might cite the documentary discussed by Schuppli (2014) as just such an instance of the 'new sensorium' inaction.
However, another possibility implicit within the very conception of a 'new sensorium' is the creative redeployment of existing sensory systems. That is, to harness them in ways that deconstruct their established utility, the representations they generate, and the types of knowledge they crystallise.
It was in relation to exploring this possibility that the conceptual and practical basis for Waveform was established. In thinking through what form the project could eventually take, a recurrent concern was the ways in which electronic sensors are attuned to particular wavelengths, and how they work to minimise extraneous noise while identifying and separating out particular types of signal return. These acts of filtering signals across various thresholds of machinic perception are undoubtedly crucial to the practical efficacy of these systems, in terms of the depictions of the world they generate, and suggested a prime area for creative-critical interrogation.
One of the most important early instances of electronic sensing, radar, provided a vital case study in giving shape to Waveform. It is evident enough that radar is a sensor whose utility lies in seeing beyond the thresholds of visible light, as defined by the human eye, into the radio wavelengths, and then re-presenting the incoming echoes as specific vectors on a screen. The result is a picture of the aerial battlespace oriented entirely around the paradigms of clarity, predictability, and control. Link (2016) notes how the advancement of radar during the Second World War was a key driver in the development of algebraic signal processing, with the goal of separating out moving targets from unwanted interference, such as atmospheric noise, weather formations, and terrain echoes. This work fed directly into the techniques necessary for the arbitrary manipulation of electrical signals, giving rise to the conception of the 'waveform' in science and engineering. Moreover, it was through these processes of filtering that the pristine, discrete units of digital signals could be expressed within electrical circuitry, and thus providing the basis for the practical realisation of digital computing.
Where this intersects with the technical configuration of Waveform is, firstly, in Link's (2016) observation that one of the earliest, fully electronic, programmable digital computers, the UK Ferranti Mark I, was used by mathematician and engineer Christopher Strachey for an experimental text generation program in 1952, producing a series of simple, comic love letters.
One of the basic innovations of Strachey's generator, foundational to many works of digital art since, was its use of a random number algorithm to provide the essential dynamo behind its generative outputs. In a digital computing context, such an algorithm is better understood as being pseudorandom-an algebraic emulation that, unless carefully crafted, will exhibit significant levels of recursion. The reason here is that the very signal processing that enables digital computing minimises the presence of arbitrary noise, and so the only means of introducing true randomness back into the system is by attaching a sensor that detects fluctuations in the ambient environment, such as atmospheric noise, and then deploys this as a source of stochastic values.
Nevertheless, in harnessing digital computing, then still a highly militarised technology, to generate ostensibly random values for the production of nonquantifiable outputs, Strachey's generator involved the deliberate fostering of the emergent and the ambiguous back into a technical and operational context that otherwise strove for its elimination. While Strachey (1954) developed his generator as an exercise in creative programming, his account of its functioning is part a greater reflection on the potential for computing machinery to yield just these kinds of unpredictable effects-to appear to act spontaneously, and so draw attention to its status as a distinctive agent. In this sense, his generator can be read as a creative inversion of the deterministic impulses encoded within the interlinked genealogies of radar, signal processing, and digital computing. Such gestures serve to make these impulses all the more visible on reflection, and inspire enquiry concerning their effects. Strachey's generator can thusly be regarded as a pioneering instance of digital art, revealing expressive and self-reflexive potentialities within devices that were then closely associated, both functionally and operationally, with tasks of the gravest utility for the state.
It is at this point that the varied threads of this discussion can be drawn together, for it was in response to these aspects that the final configuration of Waveform was established. Stated in concise terms, the project represents an updating of Strachey's original creative gesture for the task of meditating on the global sensory infrastructures that radar, algebraic signal processing, and digital computing gave rise to. Following Cosgrove (2001), these infrastructures are implicated in the geopolitical-capitalist struggles of the past seventy years, whose planetary impacts they have only detected subsequently, and thus embedding and evidencing the Anthropocene thesis. By way of deconstructing the impulses driving this cycle, Waveform takes the form of a sensory assemblage attuned to noisy patterns in the ambient environment, using this as a source of arbitrary values for the presentation not of any clear, quantifiable data, but of poetic text. The intention of this gesture is to interrogate and unsettle the normative representations made of the world using digital sensors, to inspire enquiry as to the assumptions underpinning these practices, and to stage the kinds of reflexive alternatives to existing sensory paradigms being speculated by Anthropocene scholars.

SUMMARY REMARKS
As detailed in the opening section of this discussion, the sensory aspects of Waveform are supplied by an airborne camera drone, engaging directly with the top-down perspectives that have played a crucial role in constructing an image of a closed global space, ready-to-hand for human demands. In its close association with the military 'policing' of this space, ultimately in the name of establishing a secure environment for market expansion and trade, the drone also serves an especially charged instance of a sensory agent-a device that is implicated deeply in the very changes it observes.
Regarding the subject matter to be imaged, the outlines of incoming ocean waves, and the genealogy behind the arbitrary waveforms of science and engineering, offer an intriguing point of contrast and critical reflection, with the former supplying an evident source of random values for generative writing. More specifically, the coastal environment itself represents a key domain for monitoring the ecological and climactic markers of the Anthropocene, whether in the form of rising contamination, coastal erosion, or violent weather patterns, and thus an apposite point of reference.
Finally, as mentioned previously, the poems generated through the entire process are curated in order to suggestively engage subjects and themes appropriate to just such a critical interrogation of digital sensors and their depictions of the natural environment (see Figure 5 below for additional examples here). The way in which these poems might be interpreted and assigned meaning, as they relate to the imagery from which they are derived, is deliberately left open to the viewer's discretion. As developed, however, one goal of this aspect is to draw attention to how the 'meaning' assigned to all forms of digital sensory data, however it is rendered, represents the product of an intersection between human and machine, with both coming together as sensing and interpreting agents. Specifically, this data, often originating from devices that perceive the world in ways very different from human observers, only becomes fully meaningful and actionable when understood in relation the structures and processes through which it was derived. In cases where this is obscure or ambiguous, as in the case of Waveform, the observer is tasked with attempting to establish these links for themselves. The interpretative aporia caused by the parsing of the visual into the poetic is thus an effort at challenging any sense of transparency to data representations-that they are a direct manifestation of the phenomena under study, rather than a mediated vision. In so doing, the aim is to generate enquiry into these aspects as they manifest in more established cases, where the technologies and practices involved have been rendered normative, unremarkable, and uninterrogated, such as satellite photography or automatic image analysis.
The eventual outcome of the Waveform project will be a series of hybrid images that are to be compiled eventually into a finalised artists' book, the turning of the pages showing the 'transition' from image to text in each case, as well as offering additional notes concerning the specific vocabularies and algorithms involved in the creative process.
As observed in the introduction, Waveform is an on-going project that is still at a very experimental stage of its development. It has yet to have its initial outputs exhibited widely, and so part of its presentation at events such as EVA 2018 is to gather feedback on how it might be advanced, and to judge the potential for impact in future.  (1874), and for the right image Alfred North Whitehead's The Concept of Nature (1920).