The Variations of Recognition Primed Decision-Making and How it Informs Design Decision-Making

Motivation - The Recognition Primed Decision-Making model (RPD) (Klein, 1999) has been used to describe Design Decision-Making (DDM). Originally the RPD model was a multi-stage model but it has matured into three variations to better describe differing situations that one could encounter. These variations have not been reflected in the DDM literature. Research approach - Theoretical Thematic Analysis was conducted, based on six semi-structured interviews with Interaction Designers. Findings - Three themes were identified: 1) the use of analogies, 2) the impact of constraints, and 3) the use of mental simulation. Value - This study aims to update the current understanding of DDM to be more in line with the current NDM literature. Take away message - The third variation of the RPD model is best at encapsulating the DDM process exhibited by participants, although none of the variations could account for results such as the persistence of the initial analogy.


INTRODUCTION
The act of design is composed of several different cognitive skills ranging from mental modelling to solution generation.While all of these cognitive skills are important the amount of attention that is devoted to each is not equal.Paradoxically, decision-making has been identified as an important area of design cognition whilst research in this area has been lacking in comparison to other areas (Zannier et al., 2007).The aim of my doctoral research is to investigate in more detail the area of Design Decision-Making and specifically to examine the impact of analogies.In this paper I report on an exploratory study, which aims to investigate the current variations of the RPD model in the context of Design Decision Making (DDM) and provides the foundation on which my further research will be based.
To date, there have been several models proposed to understanding DDM (e.g.Hammond et al., 1983) but the Recognition-Primed Decision-Making model (RPD) has been proposed as particularly promising.The issue though is that while the RPD model has evolved, these refinements have not been reflected in the Design Decision-Making literature (e.g.McLennan & Omodei, 1996).

Design Decision Making
There are a wide array of activities that can full under the term of decision-making, but the one common underlying thread is understanding how and why people came to the conclusions that they do.Traditionally there have been two main theoretical perspectives that have been used to understand decision-making: the Rationalist and the Naturalistic perspectives (Piegorsch et al., 2006).
The fundamental basis of Rationalistic Decision-Making (RDM) is that one can generate a series of alternatives and use clear and discernible criteria to evaluate which of the solutions maximises the utility and minimises the cost.In contrast, the Naturalistic paradigm emphasises the importance of situational pressures and how people deal with complex cognitive functions in realistic settings (Klein, 1997).Naturalistic Decision-Making (NDM) suggests that, due to the pressures and limitations of our environment, engaging in a linear evaluation of the available options is not realistic; rather, individuals use cognitive shortcuts, such as satisficing, to quickly reach a decision (Orasanu & Connolly, 1995) A growing concern though is that researchers are applying the wrong theoretical perspective when examining the activity in question (Yates, 2001).People have applied both perspectives to understand DDM such as Cross (RDM;1994) and Klein (NDM;1997) and even a mixture of both (Zannier, 2007).An example of taking the wrong theoretical perspective to DDM is the application of RDM.Simon (1996) criticises this rationalist paradigm by pointing out that this perspective is impractical in the practical context of design.The idea of a finite number of solutions to a given design problem that can be linearly evaluated and an optimal design solution selected, is unrealistic.Designers simply do not have the cognitive resources to linearly evaluate a potentially endless number of solutions to find the optimal design.
Considering the emphasis that NDM places on understanding how people come to decisions in the context of their natural environment it would seem like a more appropriate lens through which to view DDM.Both Klein (1998) and Orasanu & Connolly (1993) have compiled several factors which define a decision making situation which is best described by NDM: namely a situation which is characterised by time pressure, high-stakes, ill-defined goals, dynamic environments and input from multiple team members.Both groups of researchers state that, while not every situation will be defined by all of these characteristics An NDM-appropriate decision making situation will conform to the majority of them.In a similar vein, design, while not defined by all of those characteristics, does conform to the vast majority of the NDM identifiers.Characteristics of time-pressure and high-stakes have been discussed by Lawson (2006), while the ambiguous nature of design goals has been discussed by Rittel & Webber (1973).
Understanding DDM is more than just applying a particular theoretical lens to the design process.To fully understand DDM we need to be able to describe and model the decision-making process that designers engage with.Within the umbrella of NDM there are several proposed models (Lipshitz, 1995).Of all the differing models within the NDM framework the Recognition Primed Decision-making (RPD) is most associated with the NDM framework, with RPD being seen as the 'prototypical NDM model' (Lipshitz et al., 2001).Due to its prominence within the NDM framework RPD is a viable model to begin to understand the decision making process that Interaction Designers engage in.

Recognition Primed Decision-Making
The central goal of RPD (Klein, 1997) is the description of how people can use their experience to arrive at decisions without having to engage in an arduous mental search for the optimal solution.When talking to experienced decisionmakers, Klein found, that, when faced with a situation in which a quick and decisive course of action was necessary the first thing people did was to assess the situation and, on the basis of that assessment, select a 'best course of action'.This series of actions in the RPD model is characterised by three distinct phases: situation recognition, serial action option evaluation, and mental simulation (Mitchell et al., 1997).
Situation recognition involves classifying the situation as either unique or common based on previous knowledge.Based on this initial matching, a series of goals that will satisfy the decision point are identified.Serial option evaluation involves compiling a potential plan of action to address the goals that are required.If there is more than one potential plan of action then each plan is evaluated, one after the other, until a satisfactory one is decided upon.Mental simulation, involves mentally walking through the proposed action plan to evaluate the potential outcomes and the potential problems.Based on this analysis the proposed action plan will be executed if no problems are identified, or modified to address any identified issues.

The Three Variations of RPD
These three stages formed the foundation of the original RPD model (Klein et al., 1988) (see figure 1).As the RPD model matured it became evident that not all situations share the same defining characteristics.In his later work Klein discussed three variations of the RPD model which aimed to capture some of the differing frameworks that we use under differing situations (Klein, 1999).Three variations were proposed which represented the three most common types of situations that one could face: A familiar situation which has an accompanying familiar set of actions, a situation in which the decision-point is unclear and ill-defined but yet once it has been refined has a clear course of action, and a situation which we recognise as familiar but has an ever-changing course of required actions.The focus of the first variation is the situation in which a 'simple match' is made between the familiar situation and a familiar course of action.This match is based on identifying the situation as either a typical, which has a clear solution, or as familiar, in which they have encountered a similar situation in the past.Since the situation is identified as typical/familiar, the decision maker will recognise a course of action which has successfully accomplished this task in the past.This identified course of action is reached by understanding the priorities of the situation, what information needs to be taken into account, the expected outcome, and the typical actions which have solved the situation in the past.The proposed solution, under the first variation, is a single and familiar analogy coupled with a clear and familiar set of actions to solve the initial decision.Not all problems that people encounter are typical or familiar.The second variation entails assessing a situation where no typical scenario exists.Since either no familiar analogy is known, or various analogies are identified, a greater attention to the diagnosis stage of the model is required.When no typical case is evident, more information must be gathered to refine the range of possible analogies down to a manageable level of one or two.This evidence-gathering stage is further complicated by the fact that the individual may have misinterpreted the problem and chosen an ill-suited match from which to build a potential solution.It is possible that the decision-maker will not realise this until a point is reached at which it is clear that the proposed solution to this problem will not, in fact, be a viable answer.
While the second variation of the RPD model deals with an ambiguous problem coupled with a simple solution, the third solution focuses on the other end of the decision process: a simple analogy coupled with an ever-changing set of solution-focused actions.Under these conditions the individual must engage in a continual process of mental simulation in order to continually adjust the proposed solution as difficulties in the proposed solution arise.Mental simulation allows the decision-maker to cope with this ever changing action plan in an efficient and economical fashion.
To date any use of the RPD model in the area of DDM has been the original RPD model (Klein, 1987).While these newer variations have been applied and validated across several different fields of research (Klein, 1999), these variations have not been examined in the context of DDM.The aim of this study is to examine the use of these variations in the context of the DDM processes of Interaction Designers.

METHOD
There are several approaches to analysing qualitative data, On one hand, one can adopt a data-driven approach in which a rich-description of what occurred is obtained and themes emerge from the data.On the other hand is a theoretical perspective, which investigates in detail one particular aspect of the data which is identified by the research question.As a specific research question was being investigated, in this case the relation between the RPD variations and DDM, a Theoretical Thematic Analysis (TTA) was conducted.A TTA is a multi-stage coding process which looks for themes in a data set in the context of a particular research question (Braun & Clarke, 2006).

Participants
Six Interaction Designers with various backgrounds and levels of experience (see table 1) were interviewed.These semi-structured interviews ranged in time from 35 to 50 minutes with an average duration of 40 minutes.These recorded interviews were then transcribed verbatim.

Participant Job
Background Experience (Years) To probe the decision-making process employed by Interaction Designers, an adaptation of the Critical Decision Method for Eliciting Knowledge (CDM) was employed (Klein, et al., 1989).CDM commences with the subject being asked to recollect an important decision that they recently had to make.The interviewer asks a series of probing questions based on the information provided by the participant.To reflect the graphic nature of Interaction Design, the CDM methodology was modified so the starting decision point was a recent interface they had worked on.Each interview commenced with the participant being asked to describe the design decisions that were involved in the provided screenshot, as well as the rationale behind each of those decisions.In keeping with CDM, a series of themes were covered to provide both breadth and depth to the interview (see table 2).

Initial Decision
What was the biggest decision you had to make when putting together this interface?
Origin of Idea How did you come up with the initial idea for your design?
Experience How much do you think your experience played a factor in your design process?
Environment Did any external factors play an important part in the decision making process?
User How did you consider the user in your design?

Analysis
The preliminary phase of the analysis was to examine the data at a semantic level.At this stage the explicit meaning of the transcripts is analysed to generate an initial coding list.The codes are generated by identifying significant quotations and associating a label which aimed to encapsulate a relevant theme or idea (Braun & Clarke, 2006).Each code that is generated can appear multiple times throughout the interview under various quotations and sentences that reflect that particular theme.There are several levels of granularity that be used to break down the transcript into discrete codes.In the context of this study the most appropriate approach was a sentence by sentence level of analysis.As per Braun & Clarke's recommendations (2006) a sentence by sentence analysis was conducted.
Once the surface level meanings had been encapsulated in the code list, the analysis then progressed to examine the data for latent themes, i.e. the underlying ideas, by examining the relationships between the codes generated at the semantic level of the analysis.By creating relationships between individual codes, these relationships cluster together to form broader themes.The aim of this clustering is to give shape to the data by organising the codes under broader theoretical concepts.From these relationships a model of DDM emerged.

RESULTS AND DISCUSSION
From this analysis three were identified that encapsulated the DDM process across all of the participants.The thematic map in figure 3 summarises the clustering of individual codes into the broader latent themes that will be further discussed.What follows is a detailed discussion of the relationships between the identified themes and the RPD variations.To illustrate the relationship between the identified themes and the data, a summary of participant 2's decision-making process is used as a consistent example as he/she was the most articulate in terms of verbalising their thoughts.
What emerged from the TTA were three stages that encapsulated how the participating Interaction Designers reached the design decisions that they did: using an initial analogy as the starting point, modifying the initial analogy based on various constraints, and evaluating the proposed design through mental simulation.These three stages support the RPD model of DDM, as discussed by Klein (1999).Analogy use and mental simulation appear as important stages in the DDM process and also play key roles in the RPD model.Discrepancies between the data and the traditional view of RPD also emerged, with the persistence of the initial analogy across all stages of the DDM process being the largest difference.The initial stage of the RPD model is to quickly narrow down the possible solution space by identifying an analogy or prototype that most closely matches the observed situation.This stage was identified as the commencement point for all design decisions that were discussed.All participants discussed how the design they created was based on an analogous within-domain example.All participants mentioned this concept when engaging in the decision-making process with no participants citing the concept of their design coming from an original/inspirational source.To better illustrate the usage of an analogy to scale down the problem space, a summarised account of participant two's decision-making process is presented below.

Example 1 -Digital Library Search Interface -Stage 1
Participant 2 was faced with designing a new search function for a digital library.The primary design goal for this project was to reduce the complexity of searching an elaborate academic catalogue.From the beginning of the decisionmaking process the interface used by Google seemed to be the solution to emulate, in that it produced potentially complicated results but yet retained a simple and non-threatening interface.As a way to build on this simplicity and retain some of the power of complex searching of the original digital library, the initial search would take place on the simple Google-like search bar but refinements would be suggested in the form of modifiable drop down boxes as a way to obtain a simple query but maintain the leverage of an advanced search.
When participant 2 was faced with a complex situation that was fraught with ill-defined goals and time-pressures he/she streamlined the problem space by identifying an analogy that matched requirements of the situation, in this case the Google search interface.
"Ah yeah, I also think I was trying to copy Google as well.Trying to be evocative of a plain, long plain box and one single button" -Participant 2 This initial analogy let the designer begin the decision-making process without having to engage in a rational linear evaluation strategy.While this Google analogy was not the optimal option at the time, it was deemed satisfactory in terms of the design requirements.
These initial analogies are more than just starting points in the decision-making process in the RPD model.These starting analogies persist throughout the entire design process, with characteristics of the initial analogy being apparent in the final design solutions: "There are a few bits and pieces that have been kind of borrowed from systems ... this IVR thing -the reason why I thought of the visual thing was because I had found some instructions for something that had it mapped out in a kind of hierarchical way and so that gave me inspiration for visualising it that way" -Participant 4 From the above quotation we can see that the starting example of an old IVR (Interactive Voice Response) manual provided the starting point for the decision, but was characterised throughout the design process.This persistence of a key base concept seems to have strongly influenced the final outcome of the DDM process.In conjunction with this persistence of the original design analogy, at no point in the interviews did designers discuss comparing several design analogies to find the most appropriate.No evidence for a linear assessment to obtain an optimal solution was exhibited, as is consistent with Ball et. al. (1998) where results indicated that engineers generated very few, if any, alternative solutions.This persistence of an initial idea overriding the evaluation of plausible alternatives is also echoed in Drake's idea of a Primary Generator (Drake, 1979).Drake states that a primary generator is the situation in which a designer commences the design process with a simple idea, and latches on to this idea for the remainder of the design process, at the cost of evaluating alternative solutions to the problem.
Results indicate that interaction designers rely heavily on the use of analogies as a way to commence the DDM process.In all the discussions with the interaction designers, it seemed that the resulting end decision was a modified variant of this original analogy.The first stage of the DDM process seemed to commence with the selection of a baseline design analogy.The second stage then involved modifying this initial analogy, based on several different types of constraints.

Constraints and the Changing Action Plan
After the initial analogy has been identified in the first stage , the second stage is the modification of this analogy to not only better suit the initial design goal but, as well, to accommodate various 'constraints'.The term 'constraints' in this context refers to any influence which fundamentally changes the initial course of the decision.The impact of constraints on the decision-making process of designers has been discussed before.Hammond et al. (1983) identified constraints as one of the major influences on the formation of a design decision.Even outside the domain of DDM, the importance of constraints on the design process has been widely discussed with all parties agreeing on the powerful impact that different constraints have on shaping the design process (Lawson, 2006).The issue, though, is that, while the importance of constraints has been identified by the design research community, the impact of constraints on RPD is far from clear.
To illustrate the impact that constraints can have in the DDM process, a continuation of the previous example is provided.While the first stage of Participant 2's DDM process was to choose the Google search interface analogy as a way to streamline the viable solution space, this second example illustrates how various constraints influenced the development of the digital library interface.

Example 2 -Digital Library Search Interface -Stage 2
After the initial Google-like analogy had been selected, Participant 2 was then confronted by the immediate practicalities of trying to execute this Google-like interface.The most immediate problem with simply executing this design analogy was working within the constraints of the programming language specific to the digital library.The ranking of the results that Google returns was simply not possible to accomplish based on the existing structure of the programme.Programming in that particular language was also affected by various time crunches such as upcoming deadlines as well as certain family situations.There were other issues that led to several major modifications such as the task itself, which was simply too complex to handle ambiguous entries into the search box.In the end Participant 2 determined that, to avoid the complexity of the search results and ambiguity of the search queries, a partially filled search bar would guide the user and narrow the potential ambiguous terms that the user might enter.
From the passage we can see that the initial idea of a Google-like search interface was not able to meet the demands of the various constraints imposed on the design process.The constraints of the existing system, in conjunction with time constraints and the fallibility of the user, lead to a modification of the initial analogy.From these we can see that the limitations that influenced the DDM process ranged from technical constraints to environmental constraints to user based constraints.These represent just some of the constraints that emerged from the data as important influences.
"So we were really constrained by what we could do as in how drastic we could be with the redesign because it's not like you've got these nice little plugs you can just plug new bits into" -Participant 4 The first, or second variations, of RPD is supported by the data, since the common element between the two is that a simple course of action, which when executed, will achieve the original goal.In contrast the third variation of the mode emphases the evaluation of an ever-changing decision process, as does our data.The discussed constraints emphasised the difficulty of generating a design solution as the flux created by these constraints led to an ever-changing environment.

Mental Simulation
The Third stage identified was the evaluation of the proposed design using the act of mental simulation.After the initial analogy has been selected and modified based on different types of constraints, the designer evaluates the proposed solution by walking themselves through the use of the proposed solution and examining how well that solution addresses the design goal that was set out at the onset of the process.

Example 3 -Digital Library Search Interface -Stage 3
Once a solution was proposed Participant 2 engaged in evaluating the feasibility of the solution.As discussed in example 2, Participant 2 created a possible solution by having a partially filled in search bar.As Participant 2 imagined himself/herself in the role of the user he realised that, if the user was going to have to move around or even retype some of the predefined search terms to customise their query, this solution was simply not feasible or realistic.As this evaluation returned a negative match between the solution and the design goal this led to a revision of the proposed design.In the end the final design was a combination of the initial Google-like search bar in conjunction with a series of drop-down boxes that aimed to refine the search after the initial query had been run.This solution proved to be a modification of the original Google analogy to accomplish the design goal, while controlling for all of the influencing constraints.
The majority of statements within the design evaluation theme reflected the act of mental simulation, with the designer imagining him/herself as the user and participating in a mental walkthrough of how the user could, or could not, accomplish their goals, given the current course of action in the decision process.In the latter portion of the analysis a linkage between the codes 'design evaluation', 'thinks as user', and 'user task completion' became evident.Throughout the narrative of the interviews these three codes were constantly being mentioned in the context of the evaluation of their proposed solutions.This linkage further illustrates the process that interaction designers used to evaluate their proposed solutions by conducting a process of mental simulation in which they placed themselves in the role of the user.This linkage among the codes further reaffirms the presence and evaluative use of mental simulation as an important component in a cyclical process of adopting a starting analogy, refining that analogy based on constraints, and evaluating the potential success of the proposed solution.This iterative approach, strongly echoes the RPD framework; in which the act of mental simulation is used to evaluate a potential course of action and based on that mental simulation either a refinement of the course of action may occur or a completely new cycle maybe proposed.
The notion of mental simulation as a way to refine the ambiguous course of action is a defining characteristic of the third RPD variation.As discussed earlier the previous two models place an emphasis on being able to identify a clear and concise course of action that will successfully achieve the initial decision point.

CONCLUSIONS
The themes that have emerged from the multi-staged theoretical thematic analysis provide evidence that the third variation of the RPD model (in which a simple analogical solution is proposed and a course of action is difficult to develop due to the ever-changing nature of the situation) is best at encapsulating the design-decision-making process used by the participating interaction designers.While the third variant best fitted the exhibited decision processes, it was not a complete description.Discrepancies, which cannot be adequately accounted for by any of the three RPD variations, were found, with the persistence of the initial analogy being the largest divergence.This is an important finding in and of itself as this issue plays a large part in DDM but to date little work has been done to investigate it.My doctoral work aims to investigate in more detail the impact that these initial analogies have on the DDM process.
Over the course of this thesis a mixed method approach will be employed, using such quantitative methods as the Design Fixation (DF) paradigm (Jansson & Smith, 1991) to further the extent that fixation influences the DDM process, as well as qualitative methods such as Protocol Analysis to gain a deeper understanding of the underlying cognitive processes utilised in DDM.date two follow up studies have been conducted which have aimed to address DFs methodological flaws in its current state before DF can be used as a reliable means to experimentally control the types of analogies used in the DDM of Interaction Designers.The next series of studies will take the refined DF paradigm and use it in a qualitative context by using think-alouds to understand the differences between working with previous examples and using internally generated exemplars.This mixed method approach will allow us to investigate the impact that these initial analogies have on the types of decisions Interaction Designers reach; this will in turn allow us to help designers avoid potential 'blind alleys' in their thinking.

Figure 3 .
Figure 3.A Thematic Map illustrating the clustering of semantic codes to identify latent themes Analogy as a Starting Point in the Decision-Making Process