Exploring Audience Reading Behaviour through Interactive Public Displays

This paper focuses on how people read and respond to text on static and dynamic display screens in public spaces. It aims to explore what is effective and engaging about the interactive experience in reading, as well as exploring the engagement aesthetics of literary art in the digital age. One of the questions we address is whether the cognitive approaches to reading can inform the design of interactive art systems, to promote readable and engaging experience. In order to explore this question, we designed an interactive art system based on the Landscape Model of Reading. Being informed by the model, our system reflects both on the linguistic value and the playful element of texts in interactive art settings. The system is integrated into an interactive installation, which uses found objects as text materials, and creates a novel engaging environment for reading. We explored participants’ reading behaviour through large displays, by comparing different forms of static and dynamic presentations in a dynamic-interactive environment. Our results show that readers find it easier to read text on static-neutral backgrounds compared to dynamic-interactive backgrounds.


INTRODUCTION
Text is displayed in public spaces in various forms; in kiosks (Wendy, J. & David, S., 2010) to inform people about specific subjects and products, to advertise products on display screens . In museums and galleries, text is displayed to inform spectators about an exhibition through effective interpretive labels (Bitgood, S. 2000), or as a media artwork to deliver a novel reading experience (Arellano, D. & Helzle, V., 2013).
Text as an artwork has been a long-standing topic of interest for artists, which can be traced back to artworks from Surrealist painters such as Rene Magritte. His experimentation on words and images depicts the non-relation of a painting and its title. Looking at Magritte's paintings, spectators are prevented to be both the reader and the viewer at the same time (Foucault, M., 1983). In his influential work entitled The Treachery of Images, Magritte paints the statement this is not a pipe below an image representation of a pipe, provoking the thoughts of its spectator. In the 1960s, the language and art movement of Conceptual Art tradition continues to search for the relationship between an image, its text, and its interpreter. One of the most influential works of this tradition dates back to conceptual artist Joseph Kosuth's installation entitled One and Three Chairs (Kosuth, J., 1965). In this installation, Kosuth displays the dictionary meaning of the word chair printed on a poster, as well as displaying a wooden chair and a photograph of that chair, in order to invite spectators to question what is the actual representation of an object.
Today there is an overt relationship between the images of objects and the representations we attain to them. The commonality of the public behaviour of people sharing fragments of their lives on social media, is not a novel experience anymore. With the ever-growing advances in digital technologies, and artworks becoming physically interactive, text is appearing within interactive art systems. Examples include Listening Post by Mark Hansen and Ben Rubin (Hansen, M & Rubin, B., 2002-2004, an art installation that displays text recruited from chat rooms; Illuminated Manuscript by David Small (Small, D., 2002), a work that explores new types of reading in electronic media; and works such as Text Rain (Achituv, R & Utterback, C., 1999), Still Standing (Nadeau, B. & Lewis, J., 2005), and XFR: Experiments in the Future of Reading (RED PARC, 2001). This research is quite interdisciplinary in its nature. It is concerned with reading behaviour and draws insights from linguistic theories and new media criticism about how people read text as an artwork in large displays, as well as evaluating reading behaviour based on methods for evaluating interactive art experience and public displays. Goodman, K. (1967) describes the process of reading as a psycholinguistic guessing game that enables an interaction between thought and language. In reading, the reader does not identify all the available language cues, but selects the most productive cues that are enough to interpret, or to guess, the following of the text. By doing so, the reader predicts the content of the text, and either asserts a true or false prediction. The reader either confirms the text by consulting to her prior knowledge, or corrects the false prediction and terminates the reading process. Goodman's approach is an example of top-down processing, where the crucial information of the text comes from the readers' minds, from their prior knowledge.

THEORIES OF READING COMPREHENSION
Unlike the reading process described by Goodman above, bottom-up approach to reading assumes that the crucial information of the text comes from lowerlevel recognition skills and the readers' background knowledge is not of a high value. As described in Davoudi, M. & Moghadam, H. (2015), bottom-up processing emphasises on the hierarchical structure of reading; reading is a step by step process and it requires recognition of phonemes, words, sentences then clauses. Readers do not skip any part of the text because each of the lower-level recognition skills, such as orthographic, semantic, syntactic and phonological processing, are crucial for comprehension.
One of the criticism for both top-down and bottom-up approaches is that, there is a one-way process of reading in which lower-level recognition skills does not interact with higher-level recognition skills. One way to overcome this shortcoming is to combine both of the strategies. Rumelhart, D. E. (1977) proposed an interactive model of reading, to combine the bottom up strategies with top down approach. The interactive model of reading was based on the idea that the meaning does not only come from the text itself, or from the interpretation of the reader. The meaning is constructed by the combination of the two; it results from the interaction between the text and its reader. Rumelhart's model was one of the earlier attempts to explain how readers construct meaning in a more comprehensive way. Its importance lies in the fact that the model emphasises an interaction between a reader and the text; the reader is not a mere consumer of the text, and there is a constant information exchange between higher-level and lower-level recognition skills.

Interactive Models of Reading
Another cognitive model explaining how people comprehend text by focusing on the mental representations is the Landscape Model of Reading (LMR), which is the cognitive framework we build our interactive art system on.

Landscape Model of Reading (LMR)
Landscape Model of Reading (LMR) describes reading process as gradually building mental text representations. The model conceptualises the cognitive process of reading as a cyclical process, with each cycle corresponding to a clause or a phrase. During each reading cycle, concepts are activated according to the following four sources; (a) text input that is currently being processed, (b) the immediately preceding reading cycle, (c) the memory representation constructed for the text read so far, and (d) the reader's background knowledge (Van den Broek et al. 1996, Van den Broek et al. 1999.
The model explains the process of reading by building a coherent mental representation of text. That is said, for a reader going through a text, there are at least two processes that take place; lowerlevel and high-level processes. Low-level processes are based on decoding; recognising the orthographic properties of a written text. Higher-level processes are more complex than mere recognition; they are based on representations and making connections. For a reading process to be successful, readers need to successfully construct a mental representation of the text they read.
The model defines successful comprehension of text as reader's ability to allocate reader's attention to the most important aspects of a text. During reading, as the reader proceeds from one cycle to another, individual concepts fluctuate in their activation and text cues allocate attention throughout reading (Gaddy, M. L. et al., 2001).
LMR assumes that working memory and long-term episodic memory are in use during reading. In each reading cycle, the model assumes one parsed text statement as an input. This text statement corresponds to a clause in a sentence with a main verb and it can be depicted as a larger unit of propositions than the smallest atomic proposition (Tzeng, Y., 2007). When the reader reads the text, each element in the text assumed to be activated in the working memory. Based on these claims, we formed a recall questionnaire to measure reading comprehension, shown in Table 1.

Recent
Human-Computer Interaction (HCI) research on public displays (Sahibzada, H. et al., 2017;Müller, J. et al, 2012;Parra, G. et al., 2014) focuses on designing engaging interactive systems. The main research questions examine the modality of display screens, and its relation to memorable content. Sahibzada, H. et al. (2017) discusses the use of public displays for interactive advertising. They examine whether passers-by pay more attention to non-interactive or interactive displays. Interaction modes are encouraged in two ways; passers-by are either shown their silhouettes and they can navigate in a map through their movement, or they use their personal mobile devices to navigate. The study discusses that passers-by were reluctant to use their mobile devices, and were more interested in the advertisement in the silhouette condition.

Interaction modes; full-body or mobile interaction?
The findings from this paper indicates that, in terms of engagement, spectators are attracted by their silhouette on a display screen to navigate in a place, more than using their mobiles to be informed about a place.

Interaction modes; mirror image or silhouette display?
Some other studies have also found that changing body movement is an effective strategy in noticing interactivity of public displays Parra, G. et al., 2014). Main discussion in these studies is around whether mirror image of the spectator or the silhouette is more attractive. Müller, J. et al. (2012) discusses effective ways passers-by notice the interactivity of public displays. They designed an interactive installation to observe how people react to interactive visual feedback in the following interaction modes; (1) mirror image, (2) silhouette, or (3) avatar. Their findings are informative for the purpose of designing novel public displays; they suggest that the mirror image was found to be the most effective way in conveying the interactive visual feedback.

More engaging, but is it memorable?
Different full-body interaction modes might provide an engaging experience to spectators, however, it is not clear whether extensive full-body interaction facilitates remembering the content of the display. Panhey, P. et al. (2015) investigates cognitive effects of interacting with large displays. Their focus is on whether more movement results in higher recall, and whether more positive emotions throughout interaction results in higher recall. To test this, they designed an interactive game in 3 different interactive conditions, and compared it with the non-interactive base. Interactive conditions included (a) interaction with a mobile device, (b) through touch screen, and (c) full-body interaction. Participants were asked to complete a recall test, a game experience and user experience questionnaires after each condition. Their game performance show that participants performed best in the mobile condition. They recalled most in the non-interactive condition, followed by mobileinteraction, full-body interaction and touchinteraction as the least recalled. This study suggests that full-body interaction doesn't necessarily facilitate remembering what was shown in the display screen.
Based on these findings on interactive public displays, we aim to explore whether mirror-image display, as a full-body interaction mode, is an engaging interaction mode to prompt public reading behaviour. We will also compare the full-body interaction mode to a non-interactive mode, to explore the relationship between interactivity and memorable content. Alt, F. et al. (2012) provides guidelines on evaluation methods for research on public displays. They brought together over 500 studies that were concerned with public displays, and the common evaluation methods included; ethnographic studies, questionnaires, lab studies, field studies, and deployment based research. One such questionnaire is the Spatial Presence Experience Scale (SPES) (Hartmann, T. et al., 2016). Hartmann, T. et al. (2016) introduced a new measurement of self-reports that is particularly designed for a short and easily applicable spatial experience scale for diverse media settings. The SPES questionnaire suggests that spatial presence can be observed in two dimensions; self-location and possible actions. The questions focus on exploring the experience related to these two dimensions. For the purpose of our study, we adapted the following 4 questions from SPES questionnaire, the first two relating to self-location and the latter two to possible actions.

EXPERIMENT
The aim of this experiment is to explore how readability of text can inform the design of interactive art systems. More specifically, it aims to explore whether silhouette display, as a full-body interaction mode, is an engaging interaction mode prompting public reading behaviour. We will also compare the full-body interaction mode to a noninteractive mode, to explore the relationship between interactivity and memorable content.
As discussed in Section 2.1.1., Landscape Model of Reading suggests that readers build coherent representation of text while they go through a text. In order to successfully achieve this, they make a causal representation of events in a given text. In cases of violation in these representations, readers slowdown in their interpretation. In our experiment, we used short stories that represent sequences of events, and we displayed them in a readable and less readable forms.
Our first independent variable then is readability of text, in two levels as readable text, and less readable text. A readable text refers to a text that is represented based on not-violated causal network of events in a story (text displayed in Figure 2). Less readable representation refers to when there is an unexpected sequence in the text, such as unnecessary blank spaces that violates the traditional representation of text (text displayed in Figure 1).

Figure 1: Example display screen in M-TH condition (text with blank spaces on mirror image)
Drawing insights from the research on display screens from the section 3., we investigate whether mirror image display is an effective strategy while conveying a memorable content. Our second independent variable is engaging display, in two levels as a static white background and dynamic mirror image of the spectator. Static white background refers to a background image that is white and non-moving (background displayed in Figure 2), while dynamic mirror image of the spectator refers to a background display in which the spectator could see their mirror image, and could manipulate the displayed text through their movement (background displayed in Figure 1).
We aim to answer whether the introduction of the dynamic display on static text has a positive impact on the meaningful experience of the spectator. In order to answer these questions, we designed a novel interactive art system that encourages reading on large displays. The system is integrated into an interactive installation, Hand-Me-Down, which uses found objects such as a wooden frame and print photographs to create a novel and engaging environment for reading. We then evaluated spectator responses in a lab study, to further inform our design practices to be tested in a museum setting in a future study.

Materials
For the physical build of the interactive installation, we designed an interactive frame. A found wooden frame was hacked by adding an Arduino Uno and Adafruit Radio Frequency Identification (RFID) card reader behind it. We used 12 found print photographs and placed RFID tags behind each. Each RFID tag corresponded to a different short story, related to the images on the print photographs. Each photograph had a unique radio frequency identification tag, wirelessly connecting to the RFID reader. The read data was collected by the Arduino through serial pins. After successfully displaying unique key numbers of each cards, Arduino communicated with Processing software over the serial port for displaying the text and background image on the computer screen. A projector was used to display each condition on the wall to mimic public displays.

Experiment Design
The pilot study is a 2x2 within subject design, with four conditions. Independent variables are readability and engaging display, with the varying levels shown in Table 3.

Hypothesis
We suggest that, H1 -text with blank spaces on white background (TH) should be read slower than text on white background (T) due to triggering less connected nodes within the story as suggested by LMR. Moreover, H2-text on mirror image (MT) should be more engaging and memorable due to being more readable than text with blank spaces on mirror image (M-TH). We expect to see the condition M-TH as least memorable since it requires intensive motion and not legible text compared to other conditions, and text on white background (T) to be the most readable text compared to other conditions.

Procedure
For the experiment, participants were tested individually in one session. This is to have insights about individual's reading behaviour on large displays. The participants were instructed that they would read short stories, and that they would be asked a Yes/No question after each story. They were asked to read at their own speed for comprehension.
The experiment consisted of 4 sessions, with four different displays based on the four conditions above. Stories were presented in 4 different lists, each list with 4 different sets.
In each session, participants were presented 4 photographs, that have unique radio frequency numbers, and a frame, that reads radio frequency numbers and starts the projector. The participants were instructed to tap the photographs on the frame, to start reading the projected text and image on the wall.
At the end of the all 4 sets, each participant was asked to fill a 5-point Likert-scale questionnaire, that had 8 questions, shown in Table 1 and 2.

RESULTS
The descriptive statistics showed us that there were observed differences in between conditions for the question 1 (Q1) and 2 (Q2). We ran an inferential statistics test, Wilcoxon Signed-Rank test as we had within-subject design on nonparametric data. We chose this test, to see whether the distributions are significantly different across conditions. As Kerne, A. et al. (2014) discusses, for non-parametric, ordinal data, Wicoxon tests provide a rigorous basis, as they investigate whether the distributions are significantly different across conditions for each metric, unlike using t-test to parametrically compare means.
As illustrated in Figure 3, for the first question (Q1), there is a significant difference between the We see a similar pattern in the comparison of the conditions of TH-MTH. There is a significant difference between when text with blank spaces is displayed on white background, compared to when text with blank spaces is displayed on mirror image background (Wilcoxon Signed-Rank test; W = 49; z = 2.47 p = 0.0135). The participants could follow the text easily in both of the text conditions, compared to both of the mirror image conditions. The reason for this might be the fact that participants found the mirror image more distracting compared to a neutral background display.
There is a significant difference between conditions TH and MT. Text on white background with blank spaces is found to be followed more easily, compared to text on mirror image display (Wilcoxon Signed-Rank test; W = 41; z = 2.06 p = 0.0394). This finding suggests that although the text was less readable, participants found it easy to follow it. This might be because the background distraction is more powerful than a manipulation in text, in terms of distracting the attention of the reader.
We haven't found any significant difference between conditions for questions related to the use of space (Q5 to Q8). There might be three reasons for this; (1) the interactivity of the display screen was not clear enough for the participants to realise that it was a live environment. Some participants focused on reading and did not realise that they could actually make changes on the display screen. Another reason (2) might be that, the chosen questionnaire questions did not reflect the experience of the readers. While filling the questionnaire, some participants asked what is meant by 'true location, story environment, environment of the presentation'. This might be an indicator that the wording in the questionnaire was not clear and straightforward. Another reason (3) might be that, reading environments are perceived static, no matter whether they are dynamic-static or dynamic-interactive. We need to further investigate the use of space in reading environments, in order to support these claims.