An Architecture for an Expressive Responsive Machine

This paper describes work in progress, an architecture for an expressive machine, which learns, senses and responds to its environment with ‘creative’ output. This is not to mimic a human or any other known biological organism, but an attempt to investigate what it might mean for a machine to do this ‘on its own terms’. This hardware and software system forms the functional core of a large-scale interactive art installation, which plays with transduction between the material and non-material worlds, and between signals/stimuli in multiple forms. The meaning of ‘expression’ is discussed in reference to the machine. Some recent artworks from other artists are briefly reviewed, artworks which also employ a machine’s-eye view of the world. I discuss what expression might mean for these machines, and who or what might be the intended audience. Following from the artistic impetus for my work, the design rationale for the machine is presented. A modular architecture, with asynchronous messaging, allows for experimentation with various methods of pattern recognition, sensing and activation. Some sensing and expression modes are described in more detail. A neural network is trained to process live video input stream, to ‘taste’ what the machine is seeing in the world. A Responsive Markov Model is developed, with an automated method to extract vocabulary from selected sources (amongst others, Shakespeare’s sonnets and Molly Bloom’s ending monologue in James Joyce’s Ulysses ) and the ability to respond to particular stimuli, in real time, in generating new texts.

Content

Author and article information

Contributors

Laura Dekker

Conference

Publication date: July 2018

Publication date (Print): July 2018

Pages: 96-103

Affiliations

[0001]Computing Department

Goldsmiths, University of London, UK

Article

DOI: 10.14236/ewic/EVA2018.18

SO-VID: 2c74f47c-576e-4155-a31e-403fa4038bb9

License:

This work is licensed under a Creative Commons Attribution 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

Conference name: Electronic Visualisation and the Arts

Conference acronym: EVA

Conference number:

Conference location: London, UK

Conference date: 9 - 13 July 2018

Conference sponsor: Electronic Workshops in Computing (eWiC)

Conference theme: Electronic Visualisation and the Arts

History

Product

1477-9358 BCS Learning & Development

Self URI (article page): https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EVA2018.18

Self URI (journal page): https://ewic.bcs.org/

REFERENCES

M. Boden 2004 The Creative Mind: Myths and Mechanisms Routledge London
I. Bogost 2012 Alien Phenomenology University of Minnesota Press Minneapolis
R. Braidotti 2013 The Posthuman Polity Press Cambridge, UK
N Briz Glitch Codec Tutorial, Vimeo http://vimeo.com/23653867 (retrieved 26 March 2017
I. Cheng 2018 In Conversation with Nora Khan and Ben Vickers Serpentine Galleries London, UK 26 March 2018
L. Dekker 2017 Salty Bitter Sweet http://www.lauradekker.io/There_to_Here/salty_bitter_sweet.html (retrieved 26 March 2017
D. Dennett 1991 Consciousness Explained Little, Brown and Co., New York
B. du Castel, 2015 Pattern Activation/Recognition Theory of Mind Frontiers in Computational Neuroscience 26 July 2015
M. Fuller 2017 Half 10k Top-Slice Plus Five J. K. ShawT. Reeves-Evison (eds). Fiction As Method Sternberg Press Berlin
O. Goriunova (ed.) 2014 Fun and Software: Exploring Pleasure, Paradox and Pain in Computing Bloomsbury Press New York and London
D. Haraway 1991 A Cyborg Manifesto Simians, Cyborgs, and Women: the Reinvention of Nature Routledge New York
G. Harman 2014 Objects and the Arts Institute of Contemporary Arts London. YouTube. http://www.youtube.com/watch?v=QJ0GR9bf00g (retrieved 26 October 2017
J. Joyce 1914 Ulysses, Project Gutenberg http://www.gutenberg.org/ebooks/4300 (retrieved 26 November 2017
D. Link 2009 LoveLetters_1.0 http://www.alpha60.de/art/love_letters (retrieved 26 June 2017
Open Sound Control http://opensoundcontrol.org (retrieved 26 February 2017
L. ParisiM. B. Fazi 2014 Do Algorithms Have Fun? On Completion, Indeterminacy and Autonomy in Computation O. Goriunova (ed). Fun and Software Bloomsbury Press New York and London
Penn Treebank Project 2003 University of Pennsylvania http://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html (retrieved 26 October 2017
H. Pritchard 2018 Critter Compiler H. PritchardE. SnodgrassM. Tyżlik-Carver (eds) Data Browser 06: Executing Practices Open Humanities Press
J. Singer-Vine 2015 Markovify. GitHub http://github.com/jsvine/markovify (retrieved 26 November 2017
SOAR, University of Michigan http://soar.eecs.umich.edu (retrieved 26 September 2017
Sparknotes No Fear Shakespeare http://nfs.sparknotes.com/sonnets (retrieved 26 November 2017
N. Thompson 2017 The Seeker http://www.nyethompson.co.uk/the-seeker (retrieved 26 Febuary 2018
P. TressetF. Fol Leymarie 2013 Portrait Drawing by Paul the Robot Computers & Graphics 37 348 363

Comments

Comment on this article

[1] M. Boden 2004 The Creative Mind: Myths and Mechanisms Routledge London

[2] I. Bogost 2012 Alien Phenomenology University of Minnesota Press Minneapolis

[3] R. Braidotti 2013 The Posthuman Polity Press Cambridge, UK

[4] N Briz Glitch Codec Tutorial, Vimeo http://vimeo.com/23653867 (retrieved 26 March 2017

[5] I. Cheng 2018 In Conversation with Nora Khan and Ben Vickers Serpentine Galleries London, UK 26 March 2018

[6] L. Dekker 2017 Salty Bitter Sweet http://www.lauradekker.io/There_to_Here/salty_bitter_sweet.html (retrieved 26 March 2017

[7] D. Dennett 1991 Consciousness Explained Little, Brown and Co., New York

[8] B. du Castel, 2015 Pattern Activation/Recognition Theory of Mind Frontiers in Computational Neuroscience 26 July 2015

[9] M. Fuller 2017 Half 10k Top-Slice Plus Five J. K. ShawT. Reeves-Evison (eds). Fiction As Method Sternberg Press Berlin

[10] O. Goriunova (ed.) 2014 Fun and Software: Exploring Pleasure, Paradox and Pain in Computing Bloomsbury Press New York and London

[11] D. Haraway 1991 A Cyborg Manifesto Simians, Cyborgs, and Women: the Reinvention of Nature Routledge New York

[12] G. Harman 2014 Objects and the Arts Institute of Contemporary Arts London. YouTube. http://www.youtube.com/watch?v=QJ0GR9bf00g (retrieved 26 October 2017

[13] J. Joyce 1914 Ulysses, Project Gutenberg http://www.gutenberg.org/ebooks/4300 (retrieved 26 November 2017

[14] D. Link 2009 LoveLetters_1.0 http://www.alpha60.de/art/love_letters (retrieved 26 June 2017

[15] Open Sound Control http://opensoundcontrol.org (retrieved 26 February 2017

[16] L. ParisiM. B. Fazi 2014 Do Algorithms Have Fun? On Completion, Indeterminacy and Autonomy in Computation O. Goriunova (ed). Fun and Software Bloomsbury Press New York and London

[17] Penn Treebank Project 2003 University of Pennsylvania http://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html (retrieved 26 October 2017

[18] H. Pritchard 2018 Critter Compiler H. PritchardE. SnodgrassM. Tyżlik-Carver (eds) Data Browser 06: Executing Practices Open Humanities Press

[19] J. Singer-Vine 2015 Markovify. GitHub http://github.com/jsvine/markovify (retrieved 26 November 2017

[20] SOAR, University of Michigan http://soar.eecs.umich.edu (retrieved 26 September 2017

[21] Sparknotes No Fear Shakespeare http://nfs.sparknotes.com/sonnets (retrieved 26 November 2017

[22] N. Thompson 2017 The Seeker http://www.nyethompson.co.uk/the-seeker (retrieved 26 Febuary 2018

[23] P. TressetF. Fol Leymarie 2013 Portrait Drawing by Paul the Robot Computers & Graphics 37 348 363

Celebrating 65 years of The Computer Journal - free-to-read perspectives - bcs.org/tcj65