Resuscitating a Dead Rabbit

This paper is about the computer-generated Little Grey Rabbit stories of Professor Eric Mendoza. This (and its sister-work, High Entropy Essays) dates from the early 1960s and is one of the first examples of a fiction-writing computer program. The programming is innovative for its time and still has value as an approach to the challenge of creating literature using a computer. This approach is contrasted with contemporary AI methods of text generation. fiction.


INTRODUCTION
This is Professor Eric Mendoza is the creator of two very innovative literary computer works. His High Entropy Essays are bogus physics essays and may be the first computer-generated texts to masquerade (unsuccessfully) as human-authored. His Little Grey Rabbit stories might be the first examples of computer-generated fiction. Therefore, they have importance in the history of electronic literature that requires acknowledgment.
Added to Mendoza's historical importance in this area, his work has some vital lessons for contemporary computer-generated literature. This paper explores this aspect through the recreation of Mendoza's Little Grey Rabbit stories. A further claim is that Mendoza's approach to programming these stories may have weaknesses, but that it also has strengths. The main strength is the syntactic and semantic coherence it enables in the output texts. This is in contrast (it will be seen), with much recent AI-created literature.

A DEAD RABBIT
The place of Mendoza's work in the chronology of computer-generated literature is not certain. According to Ryan's (2017) research the first computer storywriter is the work of Joseph E. Grimes in 1963 (this was also anticipated in 1960 by SAGA II, a computer-generated screenplay writer). So, the claim for Mendoza's priority needs further research. But the example that Ryan gives of Grimes' program's output is that of a very abbreviated and schematic outline of a story. So, Mendoza's may not be the first, but it is certainly the more sophisticated.
Probably, nothing of the original computer programs that wrote Mendoza's stories survive, or the computers on which these programs ran. The computer language itself in which the original programs were written is unknown (Mendoza does not state it) and may have been unusable. It could have been an early version of Fortran (Lavington describes its use at this time in Manchester University computers -see below). Mendoza cannot be asked, as he died in 2007 (The Times  2007).
What we have is some examples of texts written by Mendoza's programs and some descriptions of aspects of the structure of the programs. With High Entropy Essays, several examples of the essays survive and a description of how they were generated in the form of a flow chart. It is because of the survival of Mendoza's descriptions of the working of this program, and that of Little Grey Rabbit, that it has been possible to attempt reconstructions. These are available online (http://www.in-vacua.com/software.html). They are written in the computer language PERL.
Reprogramming of the Little Grey Rabbit stories indicates that a text-generating artwork can survive as a description of its structure independent of the language and machine on which it ran, or runs.
Mendoza's Little Grey Rabbit stories exist as a vocabulary (a word list), diagrams of probability matrices, two examples of the first two paragraphs of the stories, and Mendoza's commentary. These were published in an essay in the 1968 catalogue of Cybernetic Serendipity (Mendoza 1968), the first exhibition of computer art (ICA Gallery, London).
Mendoza's work in this area seems to date back to the early 1960s. It may have been programmed to run on a Manchester University computer. Mendoza refers to "our staff" at the Physics Department at Manchester University at a date "about ten years ago", when discussing, in 1968, the origins of High Entropy Essays (Mendoza 1968, p.58). Therefore, it could be placed around 1958 or after. Certainly, Manchester University, which was a leader, along with Cambridge University, in British computing, had pioneered computer-created text with Christopher Strachey's (1952) Love-letters (see Link). We lack definite evidence of the date of Mendoza's work, but it seems to have been roughly contemporary with that of Grimes (see above). The fact that the idea for High Entropy Essays originated in the late 1950's does not prove that the computer program was created then. But Mendoza (1999, p.145) was aware his texts were innovative. He referred to the program that produced High Entropy Essays as "novel for its time".
High Entropy Essays and the Little Grey Rabbit stories are related works. Mendoza states he "began by writing Little Grey Rabbit stories" (Mendoza 1968, p.58). From a computer programming point of view, High Entropy Essays is simpler: "it is as if there were a number of boxes each containing subjects or verbs or objects and so on, and the computer went along choosing one word or phrase at random out of each and then strung them together in a sentence" (Mendoza 1968, p.58). This is an example of the output of the newly programmed version (very similar to the examples Mendoza published) of High Entropy Essays: Effectively quantised particles define the binding energy however quantised waves can be calculated at zone boundaries in solids phonons are reflected in all directions before planck j.j. thomson tried to deal with the blamer serixies in germany debye proceeded anharmonically It may be observed that the basic structure is that of five distinct sentences. With Little Grey Rabbit, Mendoza states, in contrast, "the style was literary and the number of impossible combinations that had to be guarded against was large". The way this is achieved is by the use of probability matrices. So, the sun might move across the wood, the sun might shine across the wood, the cloud might move across the wood, but the cloud cannot shine across the wood. The probability matrices set impossible choices to a probability of zero. Possible combinations are set to different weightings according to preference.
The attempt at subterfuge mentioned earlier in this paper, involving High Entropy Essays, is described by Mendoza (1968) as an attempt to pass off its randomised text as a real physics exam paper. The deception was discovered. But the episode is a very early version of a practical demonstration of the Turing Test (Clements 2016) involving computer-generated text. As such, it predates Bulhak's (1996) more successful ruse by several decades. This is the first paragraph of an example (there are two) Mendoza published of the Little Grey Rabbit stories: The sun shone over the woods. Across the fields softly drifted the breeze while then the clouds which calmly floated all afternoon moved across the fields. (Mendoza 1968, p.60) Mendoza's Little Grey Rabbit stories imitate the classic children's stories of the same title by Alison Uttley, first published in 1929. But the examples published by Mendoza, unlike Uttley's, lack dialogue and story development. It is easy to suppose why: the programming demands are certain to be much greater. Mendoza (1968, p.58) was aware of the lack of narrative development in the stories ("nothing actually happened"), although he does not comment on the absence of dialogue.
It is perhaps because of such limitations, it might be claimed harshly, that Little Grey Rabbit stories and High Entropy Essays are (arguably) a history of 'firsts' but also of dead ends. Certainly, Mendoza seems to have published no more of the stories, and only one more, in 1973, of the essays (Mendoza 1999).
The stories by Mendoza show little real engagement with, then contemporary, Cybernetic theory. This is in fact the norm with early text writing computer programs. As a rule, they show no learning or adaption. This behaviour is to be found in some early computer works. For instance, there is Gordon Pask's (1953) Musicolour (Clements 2007). But it is not an aspect of Mendoza's programming of electronic text. These computerworks do not develop and grow. Each run of the program has no relation to what went before or will come after. This, it will be argued, is different to contemporary AI based initiatives in text generation.
Other methods of text generation have been tried, with varying success. These include Markov chains, and Recursive Transition Networks. Markov processes and recursive grammars are already quite well explored in the literature (see Pierce 1971, andBulhak 1996). But the use of AI to generate literature is newer and less well visited critically. It is for this reason that this paper tries to draw some comparisons between Mendoza's approach and that of AI in this field.
Little Grey Rabbit starts with a model of what a story looks like and instructs the computer to produce a version similar to it. This is unlike contemporary AI-generated literature that starts with datasets of literary examples, which then the AI software learns how to imitate using patterns it finds in those dataset texts.
Mendoza's approach is essentially to have template sentence structures, with a little scope for some variants. These templates are then populated with words, from a vocabulary, which are chosen in such a way that impermissible combinations are avoided. This way of constructing a text is not unique. It is similar to the basic method of one of the more notable of examples of computer-written literature, The Policeman's Beard is Half Constructed (Racter 1984). Attributed to a computer program named Racter, it uses a technique of obscuring its text choice methods by employing random selection processes (Barger 2020) in a way that is comparable to Little Grey Rabbit. But Mendoza's work is much earlier.
Grimes' story generator (in fact, also a fairy-tale writer as is Mendoza's) seems to also use a series of random selections, according to Ryan (2017).
What, in the history of computer-text, sets Mendoza's approach apart is the use of the probability matrices, in Little Grey Rabbit, to control the likelihood of words occurring in combination. This distinguishes his work from some notable early examples of computer-generated text, such as Strachey's 1952 Love-letters, and Theo Lutz's 1959 Stochastic Text (Funkhouser). The latter uses random selections of lines from a set. Strachey's program interpolates words from lists into variable sentence structures. But neither adjusts its sentences to fit the appropriate actions of different protagonists.
In the probability matrices, probabilities are weighted by giving them a numerical value. In Figure 1 it may be seen that, for example, only Old Grey Owl 'flew' and this was three times as likely than that he 'sniffed at'. There was zero probability that he 'ran', for instance. The next term (an adverb) is controlled by another probability matrix. So, if Old Grey Owl 'flew', there is a probability with a weight of 2 that he did so 'slowly' but a probability with a weight of 1 he did so 'quickly'. And so on through the construction of the text by the program. It is as if there is a hat and this hat has slips of paper only with words and phrases on it on it that are possible. The more slips on which a word or phrase appears, the more chance of it being chosen.

RABBIT REVIVED
This is a variant version of the paragraph above, generated by the newly written program: The sky looked down quietly across the meadows. Over the woods gently moved the breeze while then the breeze which softly moved all day drifted across the meadows. This paragraph has a determinant sentence structure. This sentence structure has a lexicon of possible terms to deploy and allows for some minor alternatives in its sentence form (note the first adverb, "quietly"; the adverb is chosen from a list and not always included). But if a fundamentally different text were wanted the program would need to be substantially rewritten.
The presence of repetitions, it will be noted, also features in Mendoza's own examples "[A]cross the fields"). This results from similar actions being available in the program for different parts of the text.
Another limitation of the Mendoza approach is the length and complexity of the program required to create even brief texts. It needed hundreds of lines of code to write the first short paragraph of the Little Grey Rabbit stories. AI programming also requires lengthy coding. But it is also more adaptable. This is an excerpt of the program that produced the above version of Mendoza's text: The program can be adapted to write further new, but structurally similar, versions: Nocturne adapts the sentence and paragraph structure of the first paragraph of Little Grey Rabbit. But a new text structure, without a pre-existing model, could be created from scratch. This makes the point that new texts without precedent can be created, albeit with substantial coding work. But Nocturne retains much of Little Grey Rabbit's sentence structure.
The stars glimmered above the meadows. Over the woods calmly the shadows drifted while then the dark clouds, which softly glided across the meadows, moved over the hillsides (Clements 2019).
Run again, the program produces: The heavens revolved over the woods. Across the fields gently the stars floated while then the dark clouds, which drifted behind the trees, towered across the fields.
And so on. As noted, the outlay in terms of programming resources required to make even short texts is considerable. This is the first line of the story's second paragraph, as presented in one of the two examples given by Mendoza: Squirrel who scampered through the trees quickly ran off and off noisily ran Little Grey Rabbit (Mendoza 1968, p.60). This is a newly programmed version of the same sentence: Little Grey Rabbit who ran out of the door gaily scampered off and off slowly flew Old Grey Owl (Clements 2019).
Another run of the program writes: Hare who hurried down the path to the woods then scampered off and off happily hurried Squirrel (Clements 2019).
To make the observation again, several hundred lines of code were needed to produce this, and similar sentences, of less than twenty words. This is a strong limit on the utility of the method.
But the strength of Mendoza's approach is that the programmer decides in advance the form of the finished text. This is in contrast to AI approaches, where the program attempts to find patterns in datasets. One is a 'top down' approach, the other is 'bottom up'. A great advantage of the latter is that one dataset can be swapped for another (for example, Tolstoy for Shakespeare). The disadvantage is that, despite a substantial investment in terms of programming effort, the results still tend to be rather opaque. This might be perhaps less of a problem in poetry where, conventionally, obscurity and incongruity may be more acceptable and expected. Nevertheless, in poetry the reward for much programming effort can still be unimpressive. For example, this text presented as poetry: What is this life converge in a late to depart blue years day a moon in little years putting putting i was blue without my victories day a ragtag bell without day a moon to our cache years the letters snaps blue day a moon day day they were never best friends into the (Agrawal 2020).
Described as drama (Karpathy 2020), this is pseudo-Shakespeare: Second Senator: They are away this miseries, produced upon my soul, Breaking and strongly should be buried, when I perish The earth and thoughts of many states.
AI text-generating software is more adaptable, as numerous datasets can be inputted to create a large number of different texts. For example (this time the genre is that of technical manuals), Philip M. Parker serves to illustrate the idea: He's created a computer system that can write books about specific subjects in about 20 minutes. The patented algorithm has so far generated hundreds of thousands of books. In fact, Amazon lists over 100,000 books attributed to Parker (Hill).
Literary research in AI is relatively recent. It is probably too soon to make a final judgment about its results and prospects.

LIFE AFTER DEATH
But the assessment of Mendoza's method is perhaps easier to make. The apparent weaknesses of Mendoza's technique are its labour-intensive programming requirements and its inflexibility (if you want a new style or genre, you need a new program). But it has, as noted, some strengths. A text does not need the input of pre-existing examples for its creation: new texts do not require examples on which to be based. But to make any text, detailed programming will be necessary, with all the time and effort that suggests. The program will need to set out all the sentence structures wanted with all the vocabulary (nouns, verbs, adverbs) and so on.
The advantage of approaches similar to Mendoza's might be that the programmer is the intelligence (therefore not artificial) that plans out the text's form: she or he makes literary decisions about the sort of text required. The computer is not asked to work it out using computational methods. At present, AI seems more successful at scientific research (see, Sample), where aims are more verifiable and specific than is the case with literature.
However, it may be speculated, the difficulty of producing a complete story might have helped condemn Little Grey Rabbit stories to obscurity. But the potential of its compositional approach may still remain to be realised.