Robot Poetics : Toward a materially sensitive drawing robot

This paper opens a discussion on robot poetics: an alternative perspective on the design of robotic technology. By considering robots as metaphorical constructs they may provide new knowledge on humanity. This position is explored relative to the on-going development of a drawing robot, which will behave sensitively to material aspects – both its own physical embodiment and worldly situation – to exhibit a visual art expression intrinsic to the robot.


INTRODUCTION
This paper opens a discussion on robot poetics: an alternative perspective on the design of robotic technology.Aristotle's Poetics is a work of literary criticism that discusses an idealised form of tragic poetry through the mediums of theatre, dance and music (Heath & Aristotle 1996).Importantly, it does not dictate or provide instruction on how to author poetic form, nor does it diminish the value of poetry in the process of its dissection.Aristotle, by analysing tragedy and speculating on its ideal form, was investigating tragedy as a mirror for a philosophical enquiry on human nature.
Similarly, the discussion in this paper aims to develop an idealised poetic approach to robotics.The summary of the position is that if a robot is considered in its entirety, such that all aspects of its physical character and worldly correspondence are thoughtfully implemented in similitude to human experience, it may be considered an ideal poetic form.In viewing such a hypothetical robot, we would be emotionally moved, seeing humanity metaphorically reflected by the robotic creation.This premise is then explored through further discussion on developing a robot to draw in response to physical materiality.

Robot poetics
To avoid confusion, this paper is not about robotic poetry, whereby poetic text is autonomously generated via computation, robots or other technology.Nor is this paper about robots as a figurative representation within poetry.This paper should be taken to discuss in what ways an approach to the design and purposing of robots could be considered poetic.Robot poetics is therefore about considering robots as manifestations of human expression, specifically robots as metaphorical constructs, and thus how robots may provide new knowledge on humanity.
Poetic theory may originate with Aristotle (Heath & Aristotle 1996) circa 300 BC.Whilst poetics has become a field of literary theory and literary discourse, Aristotle wrote expansively to include the non-literary mediums of music and dance, rhythm and melody, as poetic forms.This paper broadens poetics to include robots.However, literature is a good example to discuss what is meant by poetics and the value of poetry.
Words are symbols used to communicate, whose effectiveness is based on a shared understanding between persons.Generally, words are taken to communicate an explicit meaning with precision (consider, a legal document).In contrast, poetry has more to do with '...how a text's different elements come together and produce certain effects on the reader' (Culler, 2011).
Poetry has less to do with the explicit definition of words themselves.A part of poetry can be considered as a form of 'cerebral movement' between the symbolic and cultural connotations of words (Avanessian & Töpfer 2014), which is set in 105 motion by the interplay of worldly componentsbetween author, medium, situation and reader.Through this interplay, poetry exists as something more than just the composition itself.Whilst poetry can be analysed for its informational content and structure, the dominant effect and value of poetry is phenomenologicalits interplay and effect on human consciousness.
Poetics is therefore a study of how poetry creates the phenomenological effect.Aristotle writes that poetics depends on one's ability to perceive similarity in the world, and for the poet to express the similarity from the character of a poetic medium (mimêsis).It is through playful similarity and not exactness, that metaphor can be created and used to shift a perspective of understanding.For example, we may find emotional consolation in a turn of phrase, rather than the same feeling explicitly stated, because utilising similarity helps to elucidate a feeling, which was previously cognitively obscured.This is one of the unique values of poetry and metaphor; it does not simply reflect back to us things as we already know them, but helps us to see a fresh perspective on them.
As such, mimêsis is not a direct representation of one thing from anothera symbol of an airport on a map is unlikely to cause any emotional similitude in the viewer.Instead, mimêsis as an effect is created by thoughtfully utilising the range of nonsymbolic qualities of the creative medium.Within the medium of written poetry, a similitude of feeling can be created with the meter and the rhythm of consecutive words.In essence, any medium could be creatively explored for the ways in which its 'lateral', dynamic properties afford mimêsis.
There are several aspects of robots, which may confuse the development of an ideal form of robot poetics.First of all, robots can be constructed in simulation of people (humanoid robots, or even other bio-mimicry), and also programmed to simulate an observed behaviour.In this regard, it is important to recognise that a robot is not a human (or other organism), and therefore, any simulation forms an explicit representation, which is not intrinsically derived from the robot as medium.The inherent characteristics of any given robot can instead be explored in their own right.Second, anthropomorphism is a complex phenomenon whereby a person projects their human nature onto the observation of an object.The nuances of anthropomorphism and robot artworks are discussed in detail by (Mohammadi & Vinciarelli 2011).Toward robot poetics, the ideal would be to originate an emotional affect from the intrinsic qualities of the observed robot, rather than engage anthropomorphism from the viewer to fill in any gaps.Such an 'affective directionlism' may be extremely difficult to discern clearly, but should be sought as an ideal at inception.Robot poetics can be again be considered relative to the example of literary poetry.Like words-assymbols in a poem, robots are in constitution a clearly defined composition of discrete hardware components and explicit software descriptions.Like words, robots can be thought of as a medium to explore creatively.A robot when read, like a poem, has an intrinsic interplay with the world.Via this interplay, a robot can transcend a functionalist readingi.e., 'the robot is more than its job'.
Robot poetics should therefore aspire to engage at the level of this worldly interplay, rather than focus on any explicit or symbolic attribute of the robot implementation or given task.In this way, the greater extent of robot poetics exists in the metaphysical implications of creating a robot, prior to the reading of the robot as artefact.When approaching robotic design, the ultimate constraint is the material reality of the required technologiessoftware cannot yet take hardware beyond its initial design limitations.A robot without a material basis and without material consequence is a software agent.
There is an argument that materials can mediate or inform human intention, and that the activity of making can become metaphorical.Cazeaux (2015) posits that metaphor exists when two disparate but defined concepts are brought together to create a previously unrealised third concept.In this way, metaphor is a mechanism to gain new knowledge, and metaphor is in play when materials are manipulated: In Night Sky #19, as critic Stephanie Strain notes, the charcoal is rubbed deep into the paper in a slow, accumulative process.An electric eraser is then used to burrow through these many layers of dusty charcoal to create starry pinpricks of light as a kind of negative drawing a process that moves backwards towards the original colour and surface of the paper.The effect is the production of a range of fine greyscale variations between the black of the charcoal and the white of the paper that becomes the basis for a relationship between charcoal dust and the emission of light from distant galaxies.This might seem to be not the most inventive relationship.Doesn't it just amount to saying more charcoal equals less light, which could easily be achieved through careful shading with a charcoal stick?No, because the slow, accumulative rubbing of the charcoal into the paper and the precise, delicate acts of erasure, to the point where Celmins is working with charcoal as specks of dust, create senses of the calibrated and the particulate that interact with ideas of the celestial and the astronomical more strongly than any simple, repeated act of shading.Here we can see how 106 the articulacy of material and metaphor work in tandem: the manipulation of materials, carried out in response to a photograph of a night sky, creates particular effectssubtle shifts in tone, specks of charcoal dust, spots of intense whitethat call for description, and concepts are drawn from the field of associations surrounding the night sky (Cazeaux 2015).
Presumably, In Night Sky #19 would be different again as an ink drawing, because the artist would respond differentlyconstruct new metaphorwith new materials.Designing and implementing a robot can also be considered an exploratory, metaphorical manipulation of materials.The technologies available to robotics offer innumerable solutions to a given task.Developing robots within any discipline exercises creativity.However a robot pursued as a creative medium to express something (bottom-up design) is more poetic than a robot engineered to solve a stringent task (topdown design).
Around expression there exists an important final distinction toward robot poetics.Ju & Takayama (2009) give automatic doors gestures to welcome users and influence their emotional interpretation of objects.Burneleit et al. ( 2009) develop a toaster with an expressive behaviour to investigate human sympathy toward everyday objects.Knight & Simmons (2014) investigate non-verbal communication of robotic-intention with expressive mobile robot movement.Salem et al. (2013) investigate expressive hand and arm gestures with a humanoid robot to influence human perception of robot intention.In each of these works, a human expression is emulated in order to communicate something explicit.In order for robot poetics to not simply reflect back to us things as we already know them, the expression must be more than behavioural caricature.The expression must be formed in an emergent capacity from the latent qualities of the robot as a creative medium.This fine distinction asserts congruity between a behaviour observed and from what material basis it arises.
In order to escape incongruity, creating a robot must be approached poetically, whereby it is explored with a relaxed intention, such as when working with charcoal, paint or words, to find a nonexact and resonant similitude.The remainder of the paper aims to illustrate the aspiration of these concepts in the pursuit of developing a robot to draw with a material sensitivity.

Drawing
For the artist drawing is discovery... a line, an area of tone, is not really important because it records what you have seen, but because of what it will lead you on to see (Berger 2005).Following Berger's writings, this paper similarly uses the activity of drawing to interrogate an approach robotics, and to illustrate a form of robot poetics.There is not a right way to draw, and therefore not a singular best robotic solution.It is in the pursuit of realising drawing that robotics is simultaneously creatively explored.And whilst a robot can be explicitly programmed to emulate a human approach to drawing, we may gain new knowledge by considering how best a robotgiven its complex physical nature and digital computation would itself develop drawing ability.In this way a robot may begin to exist as metaphor, and a form of poetry.
Often art foundation courses instruct students to draw with a pencil on the end of a metre rule.This abstraction interferes with their control and perception of the activity, providing insight.Implementing a drawing robot can act as a similar insightful abstraction.Any tacit aptitude for drawing must be interrogated in order to translate the activity to electro-mechanisms, software descriptions and meaningful correspondence with reality.
Metaphor is employed to generate new knowledge; robot is the first concept, brought into union with drawing (verb) as the second concept, which can result in the new third concept of an unknown instantiation of a drawing robot.The effectiveness of the tertiary drawing robot to metaphorically elucidate new knowledge will depend on how successfully it rises to the aspirations of robot poetics.If the implementation relies heavily on explicit representations, little new knowledge will be gained beyond what is already known symbolically.Here, metaphoricity is key to poetic effectiveness (Cazeaux 2015).
A drawing robot provides a means to engage philosophically with an audience, rooted in a shared tacit understanding of drawing, and by virtue of interpreting its performancethere is no correct reading, only emotive effect.The drawings produced provide a lasting record of the robots dynamic operation, which is open to closer scrutiny.It is difficult to articulate the minute details of how we decide to draw, especially in relation to expression with certain materials.However, exhibited robots and robot drawings are generally received by the public with curiosity, discussion, and accolade, such as work by (Tresset & Fol-Leymarie 2012).In this way, creating and observing a drawing robot can provide a non-exact provocation to ask and solicit questions on the how and why of drawing, and beyond drawing, knowing (metaphysics).
The two sides of drawing, the performative act and the reading of artwork, hint at a complexity easily overlooked.Freedberg & Gallese (2007) discuss from cognitive science how mirror neurons and embodied simulation allow a viewer of artwork to emotionally feel the gestural movements used in its production.This aspect transcends a direct emotive response to what is figuratively represented.It suggests that viewing marks connects audience-toartist through their own experiences of drawing, an activity founded in childhood before any adult inhibitions (Farokhi & Hashemi 2011).In this way, drawing provides a unifying activity to investigate and express both our physical experiences of being and metaphysical experiences of knowing.In antagonism to our human-sensitivity, an intrinsically robotic approach to drawing may be recognised as something alien or other (similitude), and thus elucidate our own practices via the shift of perspective.
Drawing and mark making are significant enough to be variously studied.The signature remains a means of unique personal identification by institutions such as banks; the reader recognises the visual traces of physical idiosyncrasies of the writer.Visual traces have been studied in graphonomics for the representation of persons; hands, arms and fingers are discussed for their influence in producing spatial and temporal characteristics of mark making (Thomassen & Teulings 1985); drawing spirals is investigated as an indicator of neurological disorder (Longstaff & Heath 2006).People are perceptively sensitive to the physiology behind human mark making, and therefore any incongruity between the mark and producer.
Drawing may serve as a constructively unforgiving task to create robot poetic congruity.The drawing robot should ideally not appear fraudulenteven when the operation of the robot is explicitly divulged and dissected.People should not feel fooled or threatened.Rather, the drawing robot should stand apart as recognisably other, and robot poetic congruity is dependent on the robot not existing in emulation of innately human derived processes.
The physical experience of drawing goes beyond the direct tactile properties.The sound of a pen on paper significantly influences drawing motions (Thoret et al. 2016); and sound improved the acquisition of handwriting skill by children (Effenberg et al. 2015).These wider, less obvious aspects of drawing provide the inspiration to consider in what ways a drawing robot can be creatively and metaphorically pursued as other; technologies offer many other perceptive modalities upon which expressive drawing can be explored in an emergent, intrinsically robotic capacity.
Together these elements describe how drawing can be considered from a physically embodied perspective in addition to the visual aspect.There exists lots of computational work in modelling visual composition and stylistic elements of drawing and painting; see reviews of Non Photo-realistic Rendering (NPR) (Sayeed & Howard 2006) and Stroke-based Rendering (SBR) (Hertzmann 2003).NPR / SBR are detailed enough to include models of materiality but are rarely taken beyond a software rendering on a screen.An exception includes the robot eDavid by (Lindemeier et al. 2015), which uses visual feedback as a robot physically actions an NPR generative algorithm.An interesting question is to what degree such sophisticated and explicit models are required when a poetic approach is taken.
The shortfall of NPR and SBR being physically realised by robots is highlighted by Tresset & Deussen (2014), noting that physical and material complexities are trivially bypassed in simulation software.Their research articulates the importance of physical embodiment, by virtue of idiosyncrasies, for a rich and coherent individual style.Paul the Robot is developed for a model of the composition process and visual style in emulation of the artist Patrick Tresset.Paul the Robot is highly successful and accomplished work; however whilst otherness was achieved by virtue physical idiosyncrasies, the robot does not have its own intrinsically expressed style.
All considered, it should be possible to develop an intrinsically robotic drawing stylea wide ranging and dynamic style which emerges from the machine's proprioception of material self, and exteroception of material reality.Such a robot ought to be recognisably sensitive to the drawing materials provided, and appear to express relative to its own physical characteristics.This aspiration is an exceptionally demanding problem.There are examples of work in this direction; Berio et al. (2016) demonstrate a machine learning approach to compensate for loose position tracking of safety compliant actuators in the Baxter robot.The work definitively implements an adaptive capability relative to the physical characteristic of the robot.Another example includes Moura & Pereira (2014), where a collection of mobile robots determine drawings through the emergent quality of their dynamic spatial interactions.In both these examples, the expressive quality of the robot is largely derived in software.The following section briefly reviews the history of drawing machines in order to further discuss by precedent how machines can be broadly and creatively explored for their own intrinsic expression from materiality upward.

Drawing machines
Drawing machines have a rich history that predates digital technology.A drawing machine is different from today's general conception of a computer print because the mark is made sequentially, each mark a machine movement in turn.In contrast, print technologies such as a modern InkJet progressively scan an image on to a substrate.This scanning progression is irrespective of any visual information in the image.Such scanning dissociates the movement of the machine from the image printed.
As an early example of a drawing machine, The Writer is an automata built by the Jaquet-Droz family in the late 18th Century (Schaffer et al. 2017).The automata had the appearance of a boy at a table and was able to autonomously control its movements to dip a quill in ink, shake excess ink away, and write cursive forty characters long.The text, and therefore movement sequence, was mechanically programmable via a system of cams.Besides the engineering, The Writer is a wondrous simulation of the activity of writing coupled to human physiology.
During public exhibition in 1959, (Tate & Tinguely) Metamatic machines produced over 4000 drawings.The machines were sculptural, intentionally made with low precision, driven by an electric motor.Hence the drawn trace was a result of the physical idiosyncrasies, set in the construction of the machine.Tinguely said of his machines and process: I began to use movement simply to make a recreation.It was a way of doing a painting so that it would become infiniteit would go on making new compositions with the help of the physical and mechanical movements I gave it.Then I gradually understood that movement was an expressive possibility in itself (National Gallery of Australia 2018).
In this way, Tinguely was exhibiting the physical character of his machines expressed within their drawn marks.
The 1968 exhibition The Machine as Seen at the End of the Mechanical Age (Hultén 1968) captures a transition from the purely mechanical toward computerisation.In this era, computing technology was positioned artistically as a way to see previously unseen things.Examples include observing analogue electric signals on oscilloscope displays (Laposky 1970), and computer analysis of scanned photographs drawn by a pen-plotter (Hultén 1968).In this way, latent complex qualities of the external world were expressed from a computer-centric perspective.Fred Waldauer remarked of Frankel & Raskins 'Picture-Frame': This is probably what the computer would do, if it could think (Hultén 1968).
Departing from computerised observation, the Algorists pursued a forward approach by programming pen-plotters to express a sublime aesthetic (Verostko).The pen-plotter provides a precise platform whose behaviour is largely determined in the abstraction of code.Code is utilised in a generative capacity by way of the algorithm, meaning that the code does not make immediately apparent the final visual form of the work.In this way, algorithmic code is sought by the artist to elicit a pleasing aesthetic from the penplotter.Alfred Mohr remarked: I write computer algorithms, i.e., rules that calculate and then generate the work, which could not be realized in any other way.My artistic goal is reached when a finished work can dissociate itself from its logical content and stand convincingly as an independent abstract entity (Verostko).
The Algorists utilise the physical operation of the pen-plotter as a creative constraintto produce a linear line through movementto direct the creative pursuit of aesthetically pleasing generative algorithms.
Beyond the concise algorithm, Harold Cohen continuously developed a software programme called AARON over his career.AARON autonomously produced artworks via various machine instances.For Cohen, the emphasis was on exploring his own understanding of drawing by using a programming language as "...a second cognitive system, a second place from which to view the world" (ACM SIGGRAPH 2014).In the work of AARON, Cohen was exploring his own process of creative expression via machinetransfiguration.
Beyond this summary account, there are many more contributors and exceptions to the narrative laid out.For example: Gandy (2018) creates finely engineered mechanistic drawing machines.Tait (2018) produces analogue drawing machines programmable via electronic time sequencers; Jenod (2018) continues the art of the automata.There is also a thriving Twitter community of algorithmic artists printing via pen-plotters, e.g. the artist (Hoff).
Across this brief summary of drawing machines, there appear several loci for creative expression or inquiry.Variously, this has moved from; the physical character of machines (Tate & Tinguely); the machine as a new means of observation of the world (Laposky 1970); the constraints inherent to machine and code as medium (Verostko); and exploring the metaphysical dimension of drawing 109 practice via machine as proxy (ACM SIGGRAPH 2014).All these aspects are available in combination to an artist when working with a robot.An ideal form of poetic drawing robot implicates a considered inclusion of them all.

PRECEDING ARTWORKS
Working with a machine to develop drawn-prints was essential in developing the academic positioning of robot poetics in this paper.The processes behind the following drawings fall short of the ideal of robot poetics.However it is worth considering the work for their influence toward the conceptual framing of robot poetics, and the ongoing development of a materially sensitive drawing robot described in the following section.
The following prints were all produced on a delta 3D printer adapted to draw with pen or pencil.The drawing process is the descendent of research into using machine movement to create textured 3D prints (O'Dowd et al. 2015).The author had developed an intimate knowledge of the physical character of the machine.The machine was programmed as an expert system to exploit the machine physical idiosyncrasy.
An initial drawn print (edition 60), Figure 1, was produced by utilising (Plamondon 1993) velocitybased model of handwriting.Acceleration relative to mechanical backlash gave tonal variation.Marks were randomly generated for each print.The edition held an aesthetic consistency by the absence of a computationally defined circular region, which the machine fluidly hopped over.The machine required resetting and re-calibrating every three prints to manually compensate for the wear of the pencil, which immediately brought to the foreground the need for a material awareness by the robot.
A second drawn print (edition 10), Figure 2, utilised the Hollerbach (1980) oscillation model of handwriting to randomly generate a fixed composition of 61 computationally generated 'signatures'.By reference to signatures, this work brought to the foreground the question of 'if people can draw well, why make a robot do the same?'By using models of human movement the artwork seemed disassociated from the robot itself.A third drawn print (edition 60), Figure 3, sought to explore mark making with a visual image, in order to investigate a significantly more complex composition of marks.The print was produced with a rollerball ink pen.To generate the marks, a texture analysis algorithm was applied to a source digital image, and a Gauss error function was used to provide a variable velocity.For brevity in this positioning paper, the full technical procedure is available online (O'Dowd).The process abandoned an adherence to an empirical model of movement and instead exploited the characteristics of the machine.A tendency to converge drawn paths along areas of high edge strength lent machine an apparent obsessive behavioural quality.The reapplication of drawn lines worked tone into the image.In critique, it is easy to be distracted by the novel visual appearance of a skullthis element alludes to the way that the process does not meet a robot poetic aspiration.The visual iconography from human culture impairs our perception of any intrinsic expression of the robot itself.In this way, a robot poetic drawing robot needs to go back to the first principles of drawing -responding to materiality.

DEVELOPING A MATERIALLY SENSITIVE DRAWING ROBOT
Robots are a uniquely interesting class of machine because they afford the opportunity to make technology both embodied and situated: A situated creature or robot is one that is embedded in the world, and which does not deal with abstract descriptions, but through its sensors with the here and now of the world, which directly influences the behaviour of the creature.
An embodied creature or robot is on that has a physical body and experiences the world, at least in part, directly through the influence of that world on the body.A more specialized type of embodiment occurs when the full extent of the creature is contained within that body (Brooks 2003).
A convention of robotics, inherited from the success of robots in industry, is to design for efficiency, reliability and predictability toward a given task.This task focus has the effect of nurturing a reductive philosophy to the design of the technology.Typically a task is decomposed into a subset of problems, suitable technologies are isolated, environments are classified or formally described, and control algorithms and architectures are implemented to exhibit the desired robotic behaviours.
Drawing, being located in the arts, fundamentally inverts a task-focused convention.Drawing is not a task to be solved but a space to be explored.
Drawing is an act of communication, often highly expressive.The way a drawing is received is tangled up in complex material and physical properties, intrinsically linked to the faculties of the human mind.The way a drawing is made emerges from experiences of 'embodiedness' and 'situatedness'.A materially sensitive drawing robot is being developed to be both embodied and situated, such that an emergent drawing style intrinsic to the robot can be sought.To do this, the constitution of the robot must be explored progressively, from hardware upward.Currently, position encoders have been combined with stepper motors, and programmed for compliant motion, to enable motion to be externally perturbed.A load cell has been fitted to give the robot the capacity to respond to pressure applied to a pencil.These sensors create a closed system, a machine interacting with feedback from the world.A delta configuration has been adopted again because the moving element of the robot is very lightweight, permitting high variability in velocity changes.It is also intended that a delta configuration, by avoiding human analogous morphology and detailing, will reduce the effect of anthropomorphism.Future work will investigate algorithms to exploit various embodied and situated sense-actuation relationships to generate drawings.

Figure 4 :
Figure 4: The new, materially sensitive drawing robot in development.