Moving Across The Boundaries : Visual Communication Repositioned In Support of Interaction Design

This paper is a theoretical contribution to the research area of Aesthetics of Interaction, but from a Visual Communication perspective. In order to convince those that still see Visual Communication as merely style and artifice, and an internalized and subjective design process, I will use the theses of Dourish ‘Embodied Interactions’ and McCullough’s ‘Digital Ground’ to connect to current HCI research. A Pragmatist philosophical position will be adopted from which to explore this Phenomenological area. This will present the design discipline from a fresher perspective of intellectual, considered and rhetorical discourse, into a richer understanding of the discipline by dispelling two unhelpful myths. Then an argument can be made to reposition Visual Communication as a stronger influence upon Interaction Design. Academic, Aesthetics, Interaction Design, Phenomenology, Practice, Semiotics, Visual Communication.


INTRODUCTION
This paper focuses upon resolving a current bifurcation of two influential disciplines upon Interaction Design between Human-Computer Interaction (HCI) and Visual Communication.Over the last quarter of a century the development of Interaction Design has grown under quantitative, functionalist influences.But in Bill Moggridge's original vision for the emerging design discipline of Interaction Design, it wasn't all to be about procedural scientific processes.
Moggridge wanted the new discipline to: "be concerned with subjective and qualitative values, (Interaction Design) would start from the needs and desires of the people who use a product or service, and strive to create designs that would give aesthetic pleasure as well as lasting satisfaction and enjoyment."(2007, p14).
With a purely procedural, quantitative influence the balance has dipped away from 'aesthetic pleasure' towards functionalism and usability.In doing so aesthetics has been viewed as 'inversely proportional' to usability (Ahmed et al., 2009).One of the issues with different disciplines, especially with a quantitative/qualitative divide, is agreeing what constitutes knowledge, methodology and axioms.In one discipline they can be seen as acceptable and in another discipline unsound.
"The boundaries of a discipline mark not only what falls within its breadth but also what the field excludes, what it bars."(Lupton & Miller, 1999, p66).
As the visual aspects of design do not reveal themselves through quantitative research methods, there has been a dismissive attitude to aesthetics because the emotional aspects of its influence can't be measured but judged (Dewey, 1980, 320).
But through new theses that explore embodied interaction (Dourish, 2004) and the emerging aesthetics from the experience (Fiore et al., 2005) (Petersen et al., 2004), that functionalist dip is being rebalanced towards an embodied and an aesthetic experience (Dewey, 1980) (Shusterman, 1992).Visual Communication, as a design discipline, has had problems in the past working with interactivity (Wood, 2009) but with Harrison's new HCI Phenomenological paradigm (2007) to connect with, it can help towards re-balancing the influence equilibrium, and meet HCI at least halfway, especially in researching the Aesthetics of Interaction.This paper continues my theoretical ground for a practice-based PhD in Visual Communication on the Aesthetics of Interaction and explores how Visual Communication can now be repositioned in support of Interaction Design.

Aesthetics
Aesthetics cannot be understood by quantification or reduced to simplified procedural scientific processes.
First impressions of aesthetic experience within humans may be affective (Tractinsky, 2004, p13) -we feel the experience before we understand it.Aesthetic experience can therefore be explained as spanning the mind and body into a fully embodied experience.It is Phenomenological.In re-framing aesthetics within a Pragmatist philosophical rather than a Classical position, the research into Aesthetics of Interaction can begin to share a common ground between HCI and Visual Communication.The research by Tractinsky, Petersen and Harrison is helping assuage some of the schism and misunderstandings between two disciplines of influence upon Interaction Design.The use of a Pragmatist philosophical position on aesthetics rather than a Classical or Analytical position in this paper will place Visual Communication beyond the aesthetic consideration of just the surface.Classical Aesthetics is analytic and places the human as a processor constructing disembodied independent realities that are based on perception and critical cognition.This places a creator of the aesthetic work as someone identifying and creating works that can be appreciated through immediacy, and understood from direct perception by the viewer.This disembodied, cognitive approach to aesthetics leaves out an important factor that Dewey in the early 20th century argued should be considered.Experience.With Dewey's thesis, later continued by Shusterman, he argued that we do not see aesthetics solely cognitively but experience it emotionally as embodied subjects.Therefore a Pragmatic position is more able to accommodate an embodied experiential understanding of aesthetics beyond the surface and into use, taking into account emotion, intellect and engagement.

As
Frascara (2004, p4) argues Visual Communication is a more embracing term than Graphic Design to define the design discipline.It places the emphasis upon the method (design), the objective (communication) and the medium (visual) of this discipline, rather than just the creation of graphic forms (outcomes).By adopting this disciplinary term to argue for its repositioning, I can avoid a perceived problem of graphic design as 'decorationists/dictators of style ' (Laurel, 2003).Graphic design is now so "deeply ingrained in the texture of daily life that (it's) taken for granted" (Crowley, 2004).This has led to an unfair misconception that Visual Communication can only become involved at the end of any engineering or construction process to do the 'aesthetic bit', the artifice.This misconception leaves out so much of the intellectual design process, and the emotional and social contexts (Kolko, 2010, p102) that Visual Communication draws from.Visual Communication is beyond mere decoration or artifice.It is more than an aesthetic of form or surface.The graphic designs created are not merely schematics for production or construction, but for use.The designs are not to be consumed by mere visual fetishisation (Shusterman, 1992, p16), but from engagement.The design outcomes are varied in scale, use, and material, but are rhetorical in construction.Graphic designs are not 'passive'; they are designed to make the 'receiver/user/viewer' act.They may be so ingrained in our everyday, westernised life that they are taken for granted as 'visual noise', but the designs are used everyday in some form or other.The combination of text and image manipulated by a Visual Communication designer for maximum effect, communicates a message semiologically using signs and codes.Depending upon the message, and the semiological manipulation of the design, the understanding of it can be constructed with a denotational or connotational meaning.Barnard builds upon Barthes when he says that a denotational meaning requires at least a low-level, culturally specific knowledge from the 'receiver/ user/viewer'.The connotational meaning is deeper and requires the 'receiver/user/viewer' to contribute to the construction of the meaning (p36).This is an affective contribution; it is emotional, cognitive, interpretive, and phenomenological.It cannot be reduced to a procedural process of measurement.It is an aesthetic experience that goes beyond the surface into use.

Phenomenological Position
When discussing Aesthetics of Interaction the HCI point of view must be positioned.Harrison et al. (2007) describe three paradigms of HCI.The first paradigm located itself firmly within an objective and functional view of design.The second paradigm focused upon a "central metaphor of mind and computer as symmetric, coupled information processors".The first two paradigms are a traditionalist point of view, but Harrison et al. argue that HCI has now entered into the third intellectual paradigm in the 21st century.HCI is developing from a position of 'objective knowledge' into a position from where knowledge arises from 'situated viewpoints'.They term this a 'Phenomenological Matrix'.This places the Interaction Designer into a phenomenological position of design enquiry, and nearer Visual Communication methodologies.They build upon McCullough's 'Digital Ground' and Dourish's 'Embodied Interaction' theses.Dourish suggests that meaningful experiences are situated within, and shaped by a person's immersion within the experience.McCullough develops the argument that actions are shaped by their contexts and that opportunities for participation become available through cognitively encountering them in a grounded, situated way.Shaped peripherally from possible affordances rather than directly imposed choices, the meaning of an experience changes through interaction, where the understanding of the experience is cognitive, temporal, physical and social (Anderson, 2003).Behaviour therefore is also changed to accommodate the unfolding experience, and Visual Communication, like HCI, has an opportunity to inform aspects of the Interaction Design.

Communicating Visually
Frascara's reframing of graphic design as an outcome releases Visual Communication to demonstrate it is deeper and richer than just the perceived design of the 'artifice'.Within its internalized design processes beats the semiological heart, where contextually the relational configuration of image and text is reliant on a direct relationship with the person interpreting it.The relationship is situated within the internal and external life of the graphic design -within "itself, the medium, the place and the time" (Bergstrom, 2008, p82).Frascara suggests that this is the design not of a product, object or visual, but the design of a "communicational situation" within which the design impacts on the knowledge, attitudes and behaviour of the 'receiver/user/ viewer'.Therefore there is an interactive element to graphic design, between message and 'receiver/user/viewer', through the use of the design over duration of time (p13).Bergstrom uses the terms internal and external to explain this.The semiological relationship of how the text and images are laid out in the design; the choice and use of typography; and the art direction of tone, colour, composition, flow and balance, are all the internal variables of Visual Communication that a designer manipulates.
There is a misconception that within a design the meaning is set by the designer, in fact what this refers to is a denotational meaning.An image or phrase in one social context may mean one thing to a 'receiver/user/viewer', and in another context something else.But the connotational message the target 'receiver/user/viewer' interprets leads only to construct the intended meaning.This visual perception is "pervaded by our attitudes, values and experiences" (Bergstrom, 2008, p80), and affect the process of interpreting the message.It is within this crucial area that the rhetorical nature of the visual communication engages the 'receiver/ user/viewer' proactively in understanding the designed message, relevant to their cultural context, to aid successful reception.Bergstrom refers to the designer operating from a 'perspective of proximity' to decide upon the internal variables of a design and how they will perform once externalised in space and time.

Understanding and Interpreting
From a 'perspective of reception' the 'receiver/user/ viewer' visually perceives a connotational meaning through their own cognitive and emotional interpretation based upon their own previous experiences.Through this external involvement of the 'receiver/user/viewer' in the reception, interpreting and comprehension of the message, behavioural change can be induced leading to an embodied action.Barnard presents the discipline of Visual Communication as a "signifying system, within a much larger system" as a visual constructor for how a "society constructs and communicates meaning" for itself (2005, p67).The visual organisation of the design itself must aesthetically be appropriate and congruent to "establish clear relations of importance, inclusion, connection, and dependence", and then to "guide the sequence in the perception of a message" (Frascara, 2004, pp67-68).Aesthetics attracts and retains attention to communicate possible actions to facilitate interaction.It is certainly a Visual Communication aim to seek interpretation of a connotational meaning by a 'receiver/user/viewer', rather than their passive acceptance of understanding the surface denotational message.This is crucial to aid the construction of meaning that will then elicit the embodied action and change in behaviour that the design seeks.Understanding itself, Shusterman insists, should be understood as "corrigible, perspectival, pluralistic, prejudiced, and engaged in active process", and that understanding "initially grounds and guides interpretation, while the latter explores, validates, or modifies that initial ground of meaning".He further insists that even understanding on a highly intelligent level is "unreflective, unthinking, indeed unconscious" whilst proper interpretation is deliberate, critical and conscious thought characteristically involving a "problem-situation".Interpretation acknowledges that there may be other interpretations or meanings, whereas understanding merely accepts without engaging further (p133).Understanding is acceptance, but interpretation leads to behavioural change.

Facilitating the Behavioural Change
The cultural identity of the 'receiver/user/viewer' is influential to construct the meanings.Barnard argues that semiological communication is a "cultural phenomenon, not an engineering problem", and an aesthetic choice is culturally connected and carries meaning (p28).The rhetorical nature of Visual Communication, can afford a change in behaviour of the 'receiver/user/viewer', evoking a cognitive and emotional response.Performed at the semiotic level of communication, by manipulating the internal attributes within a design, the external properties of how the graphic design is experienced.Connotationally the target 'receiver/ user/viewer' will interpret and construct the meaning.The meaning is open to reception on an individual basis but the designer attempts to frame the message to be interpreted.Depending upon the individual they may not experience the full meaning.The aesthetics of surface attracts and retains attention, but it is within the aesthetics of use that the communication is made.It is through individual response based on personal interpretation of a connotational message, that Visual Communication communicates to the population.Everyday a piece of graphic design will attract and retain attention leading to a communication being made.As a result, the Westernised world, makes everyday decisions -the context (internal and external) shapes actions.This is mostly performed subconsciously (hence being perceptively ingrained in visually cultured societies) on the periphery of everyday life.This is where Visual Communication's repositioning through Dourish's embodied interactions, and McCullough's situated digital grounding of it is evidenced.
McCullough argues "The disciplines of architecture and interaction design both address how contexts shape actions.Architecture frames intentions.Interactivity, at its very roots, connects those mental states to available opportunities for participation.These processes are ambient.Their benefits are to be found in the quiet periphery, and not in the seductive objects of attention."(p47) Visual Communication, beyond the aesthetics of surface ("seductive objects of attention") also performs this role when aesthetics of use is taken into account.Visual Communication's ground is not in framing intentions like Architecture, but framing decisions.

MOVING ACROSS THE BOUNDARIES
There is a strong socio-cultural aspect to successful Visual Communication, and this cannot be measured within an objective and functional view of design, nor as coupled information processing.These are Harrison's first two paradigms of HCI and are from a position of 'objective knowledge'.New HCI research from Harrison's third paradigm, a 'Phenomenological Matrix' (a position from where knowledge arises from 'situated viewpoints') build upon McCullough's 'Digital Ground' and Dourish's 'Embodied Interaction' theses. Petersen et al. (2004) develop a framework in which to explore aesthetics phenomenologically.
Using the Pragmatist philosophical work of Dewey and Shusterman they locate an understanding of interactive aesthetic experience in the context in which it is experienced.As the paper has shown, from a Pragmatist philosophical position, an aesthetic experience is appropriated bodily and intellectually, emotionally and cognitively, and released during a dialogue with the interaction system or artefact.Petersen outlines three central aspects to define an aesthetic experience: a socio-cultural approach to aesthetics, designing for mind and body, and the instrumentality of aesthetics.In this final section I will demonstrate where Visual Communication is well suited to being repositioned in support of Interaction Design, and in doing so reduce the bifurcation with HCI.

Social-cultural Approach to Aesthetics
HCI has come to see that to understand interaction from an emotive, experiential, aesthetic position it must do so from a phenomenological socio-cultural context.Human interaction with an interactive system is participative, embodied within the physical world, and its constructed actions are situated in time and place.Therefore improvisation (or freedom) of use of the system or artefact leads to a variance between users of what experience they gain.This is due to their own intellectual appropriation from the consummation of the experience, and drawn from their social and cultural positions.From a semiological perspective this positioning, according to Barnard, generates meaning through the interaction between 'senders' and 'receivers', the interpretation of the meaning "is not separable from the interaction of communication" (2005, p25).Frascara argues that Visual Communication designers make "substantial contributions to the clarity, effectiveness, beauty, and economic viability" of communication."To attract, to retain, and to communicate are three essential functions of every message, and aesthetics plays an important role in all three of them, but it should never become a distraction" (Frascara, 2004, p85).Interaction is not communication, and communication is not simply interaction.That needs to be clarified.I am not suggesting that Visual Communication is easily mapped over Interaction Design just because of the involvement of the 'receiver/user/viewer' in completing the experience.But for HCI to continue explore the Aesthetics of Interaction from a Phenomenological position, Visual Communication needs to become accommodated as a legitimate source for research, and not dismissed because its modes of learning, obeying, knowing and conforming are not quantifiably understood, or translate to an engineering design criteria of reproducible design rules.

Designing for Mind and Body
Lim et al. in their paper on Interaction Gestalt (2007) draw upon embodiment in parallel with symbolic representations (semiotics of the interface) where aesthetics is appropriated through both the analytical mind, and embodied experience.Therefore the design of an aesthetic, interactive experience needs to be inter-dependent with both body and mind working together.From a Pragmatist perspective that aesthetic experience is shaped not only through visuals, touch, smell, and hearing, but also from the past experiences of the 'receiver/user/viewer'.But past experiences can all at times be contradictory, ambiguous or complex.The importance for Visual Communication designers to organise the "perceptual, emotional, and cognitive processes to be followed by the viewer" (2004, p65) go beyond the issues of aesthetics of form.The context in which it is experienced and interpreted involves cognitive and emotional responses to complete the behavioural change in the 'receiver/user/viewer', where they develop a relationship "not only with the message but also with the source" (Ibid., p73).As I have outlined in the previous section, within Visual Communication the designed message has an internal and external relationship between the design, the medium its represented through, and the time and place its experienced in.The sociocultural context itself does not "induce actions so much as shape perceptual selectivity" (McCullough, 2005, p34), enabling the intellectual appropriation from the consummation of the experience to be made from interpretation from past experiences, rather than just unreflective understanding.Through embodiment, McCullough argues, the valuation and expression of the properties of the experience, long since "dismissed as unmeasurable" (p44) can be developed.Before Visual Communication can be repositioned to aid this development, the problem with understanding what aesthetics is needs to be finally discussed.

Instrumentality of Aesthetics
There is a Kantian tradition of seeing aesthetics as having no use or function beyond a pure intrinsic value (often reduced to the shorthand term 'beauty' or 'beautiful').This is due to a functionalist assumption that everything is reducible to a 'means-end' equation, an equation to define a capacity to produce a single desired result or effect.This mode of thinking has permeated HCI as thinking about aesthetics in those narrow terms allows for measurement and conclusion of success/failure.Even in Design research, those from an 'Engineering Design' position dismiss the usefulness of Visual Communication's contributions (Love, 2010).Both Dewey, followed by Shusterman, have attempted to correct this error of thought on aesthetics.Shusterman argues that the function and value of aesthetics lies not in a specialised 'means-end' but in a more global way of serving a variety of ends.He says that aesthetics enhances, invigorates and vitalises our immediate environment thus "aiding our achievement of whatever further ends we pursue.[Aesthetics] is thus at once instrumentally valuable and satisfying in itself" (p9).Therefore the instrumentality of the aesthetic is not predefined by the designer but is emergent within use.An improvisation of use of an interactive system or artefact leads to a variance between each 'receiver/user/viewer' of the experience they gain, due to their own sociocultural and intellectual appropriations.They can all feel a different degree of aesthetic experience due to their own individual behaviour.The instrumentality of aesthetics "is connected to experiential quality and value" (Petersen et al., 2004, p271).Frascara is useful here to support this view of aesthetics.The 'means-end' functionalist expectation of aesthetics can only lead to distracting decoration.This is where the misconception of the 'artifice' view of Visual Communication originates.But aesthetics contributes to three essential functions of visual communication: to attract, to retain attention and to communicate.Aesthetics serves the purpose within an experience to keep active in the mind of the 'receiver/user/viewer' their immediate options for continuing the experience.Therefore as Tractinsky ( 2004) has revealed, and as Visual Communication has always understood, higher aesthetic values aid the enjoyable continuation of an experience until consummation.Aesthetics operates within a Phenomenological Matrix (or as Visual Communicators already understand it -a context) within which the designed interaction's function, limitations, and possibilities can be understood.Visual Communication, Interaction Design and especially HCI cannot be truly understood without a context that includes "social, political, physical, cultural, and commercial contexts in which it operates" (Frascara, 2004, p86).Therefore aesthetics and usability are connected together in an orthogonal inter-relationship synthesising an enjoyable experience, rather than aesthetics having a negative, detrimental affect upon efficient functionality.It is not a case of 'means-end' but as Moggridge stated in the opening quote to this paper, a "lasting satisfaction and enjoyment".

CONCLUSION
In order to argue for Visual Communication to be repositioned in support of Interaction Design I have had to dispel some myths and misconceptions about what it does, and what the function of aesthetics is.The two main myths dispelled in this paper were that Visual Communication just did the 'aesthetic bit', and that aesthetics had no use or function beyond 'beauty'.Approached from a procedural, analytical, engineering position (where experience isn't discussed), aesthetics could not be reduced and measured as a functionalist equation of 'means-end'.Therefore the usefulness of aesthetics and Visual Communication design are immeasurable.But if we take experience, emotion, and interpretation into account we need to explore those aspects within an aesthetic experience.Then it becomes a cultural phenomenon not an engineering problem.Using Pragmatist philosophy to examine the Aesthetics of Interaction from a situated and culturally connected embodiment of the experience, aesthetics then becomes emergent from the experience through three factors.A sociocultural context based upon the audience prior experiences; the current experience appropriated from an embodied interaction within a situated temporal-spatial physical place; and finally the means-to-many-ends that only emerges from the interaction.This makes this a Phenomenological study of aesthetically experiencing interaction and closer to Visual Communication than to traditional HCI.Thanks to some exciting research from HCI, some researchers are building the theoretical bridge towards Visual Communication.Despite what detractors may say, Visual Communication is a rhetorical communicational situation from where the knowledge, attitudes and behaviour of its intended audience can be altered through the outcome of its graphic designs.Through semiology, the designers' manipulation of signs and codes, with the internal and external contexts of combined text and image, involve the target audience in interpreting the intended message of the design.This is received through similar embodied processes as used in interactive systems and artefacts.This takes Visual Communication beyond aesthetics of surface and into aesthetics of use.Like Interaction Design, Visual Communication is also concerned with designing an enjoyable experience.Its graphic designs are not passively consumed, but are used (even if its on a subconscious level).Aesthetics serves in both cases to attract, and retain attention so that the immediate options can be communicated.This is not the 'aesthetic bit' added on as decoration, but it is an orthogonal factor with functionalist usability, that together synthesise into an enjoyable aesthetic experience.HCI cannot understand this without repositioning Visual Communication back into a supporting discipline to Interaction Design.