I Feel You: What makes algorithmic experience personal?

Since artificial neural networks are excelling in stylization of visual, textual and acoustic objects, we discuss the potential of such technologies in context of human-to-machine interactions in general and in an artistic context specifically. We discuss a paradigm of an uncanny-valley that was originally proposed for human-robot interaction and look at it at a new angle working with algorithms rather than with robots. We propose an exploration framework for human-algorithm interaction and illustrate basic principles of such interactions with two case studies on the verge of art and computer science. Finally, we discuss how exploration paradigm outlines the vector of further human-algorithm collaborations


INTRODUCTION
Communication is a very diverse form of human-tohuman interactions.Communication does not have to be verbal, it might be tactile or visual, in fact one can see communication between the author and her audience in any form of art, yet whenever we talk about human-to-machine interactions we rarely regard them as communicational experiences.
Despite that "social robots become more common in our society" (Broadbent 2017, Vladeck 2014) discuss possible judiciary implications associated with the arrival of truly autonomous machines and (Whitby 2008) argues that the may even lead to the breakdown of society.In this contribution, we refrain from such apocalyptic statements and rather explore the concept of communication in a context of a human-algorithm interaction, and address two key features that hinder the personalization of such interaction, namely, the problem of reproducibility and the issue of uncanny valley.This paper focusses on two case studies on the verge of art and computer science.These are Neurona -a mini-album with lyrics generated by a neural network in the style of Kurt Cobain (Tikhonov & Yamshchikov 2017a), and a Scriabin-stylized AIgenerated music-piece performed life in Moscow in summer 2017 (Tikhonov & Yamshchikov 2017b).

INTERACTION VS. COMMUNICATION
Why do we interact with machines and communicate with people?
The fact that in a context of human-to-machine interactions one rather talks in terms of userexperience or interface instead of a communication has several historical reasons.The first one is that a predictability in a narrow context was for many years (and still is) in the focus of human-machine interfaces (Weizenbaum 1966, Card, Moran & Newell 1983), whereas communication by design incorporates discovery and context shifts as integral parts of the whole process (Ohsawa 2003, Saab, Joolingen & Hout-Wolters 2005, Abe 2011).Humans expect clear, understandable and reproducible patterns when interacting with a machine, yet allow another human to be more versatile and obscure.
The second major issue that seemingly hinders human-to-machine communication is a so-called phenomenon of an uncanny valley (Mori 1970, Mori et al. 2012).This is an idea that more human-like robots would at some point cause eeriness or uneasiness in a human that interacts with them.There is a number of empirical results that consider this problem and even question its existence, see (Burleigh, Schoenherr & Lacroix 2013 Kätsyri et al. 2015), but here we are interested in this hypothesis conceptually and would show the it is irrelevant in a broader context of human-machine interaction that we address in this paper.It is substantial to note that proponents of the uncanny valley base their derivation on the inevitable anthropomorphism of the interactive machines.This premise is either stated explicitly (Mathur & Reichling 2016) or is implicitly incorporated in the reasoning that describes future technologic tendencies (MacDorman & Entezari 2015).However wider adoption of interactive AI-powered technologies shows that such anthropomorphism is neither a sufficient nor a necessary condition.Versatile personal assistants such as Alexa or Siri have limited anthropomorphic features, namely, the human-sounding voice and natural language, yet are by no means anthropomorphic in a broad sense of the word.Other AI-powered solutions tend to have no anthropomorphism at all yet humans voluntarily interact with them.One of such examples would be a game of advanced chess.In advanced chess two players accompanied by two AIs play against one another.The concept of such game was proposed in the 70s (Michie, 1972), but the fact that history of advanced chess starts in 1998, one year after a historic game of Garry Kasparov against Deep Blue, symbolizes a paradigm shift: form the twentieth century "humans against the machines" to the twenty first century "humans with the machines".In line with this idea we suggest to move from the human-robot interaction paradigm to a broader and more relevant human-algorithm interaction thus eradicating the problem of uncanny valley all together.Now let us come back to the differences between interaction and communication and look at them in the context of human-algorithm interaction.When talking either about cognitive tasks or about certain machine learning problems one often uses terms exploration and exploitation to describe different phases of interaction between an agent and an environment.Exploration phase is characterized by surprize maximization: the agent attempts to get as much new information about the environment as possible through trial and error.Exploitation, on the other hand, is characterized by lower levels of new information obtained by the agent, yet is associated with a reward maximization: as the environment becomes more familiar the agent learns how to get positive rewards out of it with minimal risks and little new knowledge about the system.These two phases also clearly illustrate the difference between communication and interaction: (i) We strongly associate interaction with exploitation.Both of them are all about results.
(ii) Interaction as well as exploitation implies predictable outcomes and limited decision space.Such restrictions make interaction non-personal, since the trajectories of different individual humans largely coincide.(iii) We strongly associate communication with exploration, since both of them are rather focused on the process itself.(iv) Communication as well as exploration implies diverse outcomes and large or even unbounded decision space.This makes communication personal, since every human can follow her own experience trajectory.
These naturally points out that if a human uses any AI-powered system with explorative intentions, her experience inevitably becomes personal as long as the state space of the aforementioned system is big enough and the means to navigate such state space are provided to the human.This framework could be very naturally transferred to the artistic experience especially in the context of generative art that fundamentally values creative process (McCormack et al. 2014).Further we give two examples of how text generation system or music generation system can be explored and become a source of personal experience, but before we have to mention some human-specific restrictions that have to be taken into account to make such experience genuinely interesting for humans.

HUMAN-LIKE RESTRICTIONS
In his lecture (Gromov, 2017) states several humanlike restrictions that outline current limitations of what is perceived by human brain as interesting, human-like or natural.Let us state these limitations here: • Non-acceptance of unstructured sets.When presented with a set without detectable structure humans tend to reject such set as a meaningless one.• No built-in sequential counting beyond four.
Any number of objects or patterns beyond four is categorized as an uncountable plural number.
• No ability to generate five consecutive iterations of the same process.
These restrictions are good examples of limitations that one needs to apply to AI-powered systems to make human-algorithm interactions fruitful and authentic.Alongside with explorative mode described by principles (iii) and (iv) such restricted system could be a source of personalized experience, since they facilitate human exploration of the AI-powered system and allow prolonged experiences that humans can perceive as interesting or insightful.
One also needs to mention that such restrictions could differ depending on the task and can themselves be a topic of an artistic or scientific exploration.Psychologists have already found a number of other possible limitations, see, for example, (Newell and Simon 1972, Faust 1982, Chi and Fan 1997, Sweller 2006) and many more.We list here the limitations outlined by Gromov since they instantly map on generative tasks that are described in this paper, however one can use other set of limitations and obtain other potentially insightful results.

CASE STUDIES
Further we list several examples of explorative interactions between a human and an AI-powered algorithm in a creative artistic context.

Poetry generation -Neurona and Neural Defence
The first examples of such explorative systems are two projects on the verge of art and computer science (Tikhonov & Yamshchikov, 2017a) in which an artificial neural network for stylized poetry generation was used to generate lyrics for mini music albums.This project was carried out in Russian in 2016, when a mini-album '404' by Neural Defence was released.In 2017 it was followed up by another mini album 'The model' by Neurona in English.Neural defence is a tribute to the cult Russian punk-rock band Civil Defence.Neurona is an analogous tribute to the cult grunge band Nirvana.Both albums got several hundred thousand listeners and attracted media attention in Russia, France, Germany and other countries.
The texts of the lyrics were generated in a way close to the explorative mode described above.Recurrent Neural Networks (or RNNs) tend to generate syntactically correct texts that can resemble target texts stylistically (Tikhonov & Yamshchikov 2018), however among correctly written texts there is inevitable noise, mistakes, semantic errors etc.Therefore, the process of text generation for Neurona as well as for Neural Defence was done in the following way: the network generated raw output and final texts were cherry-picked out of it by a human.This mode of interaction made the whole process interesting, insightful and fundamentally personal since different lines resonated with different readers.
We would like to share one of the insights that, in our opinion, pinpoints the nature of such explorative human-algorithm interactions.One of the Nirvanastylized lines ran: A god, who's always welcome to Iraq! Kurt Cobain committed suicide long before the start of the Iraq campaign, however, due to the fact that RNN was trained on a huge corpus of lyrics (not only on Nirvana ones) this line emerged among the generated lyrics.It also fitted into a broader context of a song due to the correct rhythmic structure.
When filtering the network's output, we found this line exceptionally interesting, since it resonated with our intuition that, if he was alive, Cobain would definitely address Iraq war in his lyrics.We also found it stylistically similar to Cobain lyrics and emotionally provocative.Formally speaking, all these characteristics are human attributed yet can emerge out of the explorative human-algorithm interactions.

Music generation -Scriabin stylization
The second example of such explorative interaction is a music piece generated for the opening of Yet Another Conference in Moscow, in summer 2017 (Tikhonov & Yamshchikov, 2017b).Using variational autoencoder supported by history (VRASH) as described in (Tikhonov & Yamshchikov, 2017c) a Scriabin-stylized melody was generated.Similarly to the lyrics in the first example, the generated melodies were curated and orchestrated by modern Russian artist Maria Chernova.The result of this collaboration was performed live in front of two thousand visitors of the conference in May 2017.Leaving the stylistic resemblance of the final performed piece and actual Scriabin-written pieces out of the scope of this paper we rather focus on the feedback that we got from the musicians who we collaborated with.Both key artists that were working on the project, namely, Maria Chernova, who curated and orchestrated the melody, and Petr Theremin, who was the lead performer, stated that they had a feeling of "meaningful, personal interaction with another entity".Next we discuss the phenomenon of this feeling and point out further developmental directions for creative humanalgorithm interactions.

ARTISTIC COLLABORATION WITH AI
A broad majority of people still interacts with machines along the exploitative lines (i) -( ii), yet we see an emerging trend of communication-like interactions between humans and algorithms.This explorative interaction along the principles (iii) -(iv) is either directly connected or simple serendipitously coincides with a paradigm shift in human-algorithm interactions.We think "a human AND a machine" rather than "a human AGAINST a machine".
As this AI-empowered paradigm shift occurs, a greater number of algorithms are designed to be explorative by nature.To give an example we have recorded a short demo that explains basic principles of the exploratory music generation process (Yamshchikov & Tikhonov 2018).The dataset is a cloud of dots in 16-dimensional state space, where every dot is representing one midi-file used for training.Our proposed prototype allows to navigate interactively through the plane defined by three random files within the cloud.The dots that we see on the plane are actual projections of the dataset on the chosen plane.As we navigate through the plane the model samples a new track in real time.Every track is defined by its latent vector representation, its genre and the instrument that is supposed to play it.The genre and the instrument are chosen from the nearest point in space, this point is visually emphasized with light-blue square, the current position of the latent vector in the state space is noted with a viewfinder symbol.
A person can spend hours with such AI-based explorative tool and never repeat any single piece of her trajectory.
In the last several years we have seen a number of attempts to use AI as a tool for creators.Using the approach proposed by (Gatys, Ecker & Bethge 2016) a number of visual artists applied style transfer algorithms to various pictures creating new unique experiences.In a postmodern cultural context style transfer can be a conceptual tool that is able to convey the thought of an artist in a more expressive and meaningful way, for example see (Joshi, Stewart & Shapiro 2017).Despite the artistic and scientific interest of such projects they stay within a classic exploitation human-algorithm interaction paradigm.The fundamental difference between AI-powered tools and AI-powered assistants is clearly seen when one compares image style-transfer algorithm and music generation example equipped with the state-space visualization tool.Style transfer algorithm is trained on a specific picture and when applied properly delivers predictable and reproducible stylization results that go in line with the initial intent of a particular artist, whereas generative music-exploration algorithm such as the one on Figure 2 alters the initial intent of the artists, provokes her or even contradicts with the initial plan.Moreover, due to a large training set a certain randomness incorporated into the exploration process the experience is unique for every user session.This contrast goes strictly in line with the principle differences between interaction and communication, formulated in (ii) and (iv) and illustrates a contrast between human-algorithm exploitation and human-algorithm exploration perspective.We most sincerely believe that explorative, communication-like patterns are a way to go with AIpowered systems.Paired with different sets of human-like restrictions they could be an enormous new field for research and artistic discovery that could tell us more about art, cognition, and, ultimately, about ourselves.

CONCLUSION
In this paper we draw a line between interaction-like and communication-like experiences, associating the former ones with reproducibility, limited choice and exploitation cognitive strategy, and connecting the latter to the unique, unbounded explorative scenarios.We suggest to broaden a framework of human-robot interactions to the domain of algorithms by relaxing the limitations of the so-called uncanny valley that is firmly associated with the anthropomorphic machines.We rather suggest to focus on human-algorithm interaction.With two case studies of generative artistic projects we demonstrate how the exploration approach can fundamentally personalize such human-algorithm experience.Finally, we outline possible vector for further development of such explorative algorithms that might be of interest in the context of artistic statements as well as in the cognition research.

ACKNOLEDGEMENTS
Authors would like to thank Mark Wilkinson for his help and attention to details, Andrey Sebrant and Matvey Kireev for their constant support and enthusiasm, Ilya Edrenkin for his technical consultations and meaningful advice, Pavel Gertman and Maria Chernova for their artistic inputs and Yurii Zelenkov for his poetic heuristics.

Figure 2 :
Figure 2: A screenshot of an interactive visualization for 16-dimensional state space.The plane is defined by three tracks marked as red squares, light blue square highlights the closest track to the latent vector, the viewfinder shows the position of latent vector for generation.