Sensemaking and Visualization for Situation Cognition

Motivation – The study describes an on-going effort to develop empirically-based operational visualization concepts and their implementations to support sensemaking skills in ill-structured problem situations. Research approach –Military officers with command experiences are used to conduct sensemaking exercises in simple and moderately complex scenarios. Findings/Design –The study is a 2X 2 between-subject design with two task scenarios and two sensemaking tools conditions. The use of Analysis of Variance revealed mean differences between aided and unaided groups and across task scenarios. Research limitations/Implications – The results indicate the need to use environment visualization aids to support sensemakers who normally deal with ill-structured problem situations. The limitation of the study is studying the different types of cue prompting and their presentation modalities (e.g., a combination of auditory and visual modes). Originality/Value –The existing literatures on situation awareness do not clarify the relationship between display and visualization, and sensemaking performance in ill-structured problem situations. Take away message – Attaining decision superiority in technology-enabled work place is enabled primarily by visualization and display tools which can make sensemaking tasks less cognitively taxing.


INTRODUCTION
Sensemaking is a process, a design, or a technique of fusing information, achieving situation awareness (Endsley, 1995;Klein, 1989), and interpreting the information in context so as to derive knowledge for actions.Sensemaking has also been viewed as a paradigm, a tool, a process, or a theory of how people reduce uncertainty or ambiguity; or socially negotiate meaning during decision making events (Ntuen, 2006).Weick (1995) states that sensemaking refers to how meaning is constructed at both the individual and the group levels.Sensemaking involves the collective application of individual "intuition"-experience-based, sub-consciously processed judgment and imagination-to identify changes in existing patterns or the emergence of new patterns (Weick & Sutcliffe, 2001).As an NKDM tool, sensemaking is a deliberate, conscious process of fitting data into a frame (Seick, et al., 2004).The frame may be a story, script, map, or other form of representation.The intention is to reduce complexity and simplify the world in relation to a particular goal.Referring to Figure 1, this is the information fusion process.
Frenette (1999) has noted that "humans only use information resources in the context of their own experience -where they come from, what they are struggling with and where they are going."Additionally, Suchman (1987) explained that purposeful actions, such as information seeking for task executions (e.g., judging, planning, deciding) may be planned based on our mental constructs which Klein (1989) refers to as mental simulation mitigated by recognition primed process.Sensemaking, is however "inevitable situated actions", since the sensemaker's actions are shaped by the ad hoc and local contingencies of a situation.Within this context, sensemaking is a cognitive phenomenon with different ways of seeking and obtaining information (and processing the information into knowledge), and of judging relevance of information under various circumstances.It has a domain dependent property; i.e., what makes sense for a physician may not necessarily make sense for the military commander.However, according to Ntuen's (2008) stages of sensemaking, the process has identical ingredients, except that unlike decision making, it is not symmetric but is often subject to occasional collapses due to the changing nature of the problem environments (Weick & Suchcliffe, 2001).
Theoretically, sensemaking is aided by many cognitive tools; primarily among these is situation awareness (Figure 1).Situation awareness (SA) theory, developed by Endsley (1985), has influenced our interpretation of information-and situation-understanding. Endsley defines SA as a three tier process:  (a) Perception of the status, attributes, and dynamics of the individual task-relevant elements in the environment (Level 1, SA); (b) Comprehension of current situation, based on a synthesis and understanding of perceived elements and relevancy of goal (Level II SA); and (c) Projection of future actions in the system, at least in the short-term (Level III SA).SA by itself is a by-product of visualization that occurs internally and externally.However, the existing literature does not clarify the relationship between SA (as explicated by display and visualization) and cognitive performance in information-and situation-understanding tasks, especially in austere, dynamic, complex networks of adversaries.

SITUATION COGNITION
As shown in Figure 1, SA, sensemaking (SM) and situation understanding (SU) is a tripartite cognitive construct that can only be understood in context.This context is intrinsically embodied within the task environment, human cognition, and aided tools via displays.Thus, situation cognition has been viewed as a sequence of situated acts (Clancey, 1997;Suchman, 1987).Because situations may change over time, the cognitive processes required to adapt to such changes must be dynamic.This change is dependent on constructive memory which holds that memory is not a static imprint of a sensory experience, but is subject to continuous changes due to new information stimuli (Dietrich and Markman, 2000).The sensory experience is stored and the memory of it is constructed in response to any demand on that experience.Shattuck and Miller (2004) describe a dynamic model of situated cognition (DMSC) in which data flows from the environment, through sensors and other machine agents, to the human agents in the system.This approach overcomes the biases which are inherent in analytical methods focusing almost exclusively either on machine agents or on human agents.The DMSC posits that there are various stages of technological and cognitive system performance.On the technological side, all the data in the environment, data detected by technological systems (e.g., sensors), and data available on local command and control systems (C2; e.g., workstations) are included.In this situation, both technology and human cognition are embodied; sensemaking or decision making tasks are rarely deliberate; they are dynamic since situation information is evolving in time and context.

VISUALIZATION AND THE THEORY OF SITUATED COGNITION
Visualization tools present information and arrange knowledge in a visual representation.An advantage of visual representation is that it makes important information more accessible and easier to remember and process.Visualization has been studied by noted psychologists and philosophers for years (Fodor, 2000;Searle, 1983).Among other things, it has been observed that visualization occurs internally in the mind and is externally mediated by ecological information factors (Gibson, 1966).The mind itself carries abstract information such as thought, ideas, perceptions, feelings, and memories.The mind is responsible for shaping of meaningful spaces for situation understanding through logical assignments of concepts and their saliencies.In the process of visualization, the mind expresses this in terms of imagination, precepts, and creativity through the ability and capacity to conceive ideas and concepts that precede cognition.Card, MacKinlay and Shneiderman (1999) observe that information visualization is useful to the extent that it increases our ability to perform some cognitive activities; and, "the real power comes from devising external aids that enhance cognitive abilities; for the sole purpose of knowledge discovery, decision making, and explanation."Visualization tools are therefore useful in supporting direct cognition of information in context of a domain environment.It is useful for gaining insights into external cognitions in context.As noted by Eppler and Burkhard (2003), knowledge visualization allows visual representations to improve the creation and transfer of knowledge between people by sharing what they know and what they need to know through perspective making and sharing.As noted by Novak and Wurst (2004), "in order to make sense out of information and construct knowledge, one needs to contextualize it within one's own existing knowledge and thought world …(which is the art of visualization)."

SUPPORTING SENSEMAKING WITH VISUALIZATION MODELS
The AVALANCHE model for theorizing, designing, and experimenting with situation cognition has been developed (Ntuen, 2008) for studying visualization in the context of evaluating cognitive readiness in networked-enabled battlefield tasks.AVALANCHE is an acronym for Aggie Visualization Architecture for Learning to Anticipate Novel Cognitive Human task Environments (See Figure 2).AVALANCHE is a constellation of many visualization modules that include Sensemaking Support Software (S3), theory domain, and experimental domain.The S3 module allows for collaborative sensemaking, supporting 1-5 participants who may be co-located or geographically dispersed; the theory domain allows the users to formulate a series of hypotheses from the sensemaking tasks; and the experimental domain allows the user to select situations of interest for evaluation.Currently, we have completed the development of SASOSIM, an experimental domain for simulating stability and security operations within AVALANCHE.The model captures the Army's FM 3.0 doctrinal statements in terms of four cognitive levels of abstraction hierarchy: 1) the operator's experience as retrospective knowledge; 2) a situation awareness model that enables spatial knowledge and affordance for noticing information; 3) a knowledge discovery model built around a network of human visualization and pattern recognition mental models; and 4) a situation understanding model to enable the process of translating information in context to actionable intelligence.

Background
The purpose of the experiment is to evaluate sensemaking performance in two task scenarios with and without visualization aid.These two scenarios will henceforth be called S1 and S2, and the sensemaking affordance tools will be "Unaided" and "Aided", respectively.The SASOSIM environment with tasks that mimic stability and security operations is used.SASOSIM addresses the issues related to joint military and civilian emergency response operations.Stability and Security Operations (SASO) are created by a system whose state changes are either predictable or unpredictable with respect to time, location, and changing patterns of information.The pattern of information can reconfigure their interactions such that the emerging behaviors become difficult for the human to understand, thereby placing some burdens on the human cognitive resources.Table 1 below illustrates a simple two-dimensional matrix information interaction created for the study.Two scenarios from Table 1 were used: predictable environment with random information (S1), and unpredictable environment with deterministic information (S2).Note that there is no orthogonal relationship between S1 and S2.Carolina.All participants had command experience in a platoon and have taken part in SASO tasks.11 participants were Majors and 5 were Captains.5 participants did not complete the tasks and were eliminated from the analysis.

Materials
Two case scenarios were created and approved for use by a military subject matter expert.These vignettes are shown in Exhibit 1.
Two simple cases from S3 are presented: The possible strategies for the situations as suggested by experienced commanders are shown in Table 2.

S1:
Explosions have been reported at a market place in Bagdad, Iraq; five minutes later, another explosion of a similar description is reported at a mosque about 60 miles along the Sunni neighborhood.Many coalition force members are reportedly trapped and many civilians are fleeing their homes.Past intelligence estimates had warned of possible attacks on the perimeters of the Sunni corridor and downtown Bagdad.S2: Incidents in Najaf, Iraq have been unpredictable.Various groups usually claim responsibility for attacks.Some of the known ones are: Al-Zawahari Army (30%), Al-Qaida (50%), and ethnic conflicts (20%).So far, nobody can estimate the attack strategies which can use IED, kidnapping, suicide bombing, or sniper attacks.
A recent intelligent report is that two top politicians have been taken hostage and a reconnaissance aircraft used by one of the coalition forces has been shot down.Exhibit 1. Case scenarios used in the study.

Apparatus
The SASOSIM visualization tool runs on a 90-Megahertz Intel Pentium-class processor with Microsoft Windows Server 2003, Windows XP Professional, Windows XP Home Edition, and Windows Millennium Edition (Windows Me). Figure 2 shows an example screen capture for a SASOSIM session.

Procedure
Individual and not team sensemaking was assessed.Each participant logs into S3 with a protected password, and reads the SASO incident presented to reflect the situations S1 and S2 (left hand side of Figure 2).The right side of Figure 2 allows the experimenter to show maps, situation cues, and can use a white board to illustrate possible locations the enemies (red bar), possible plans of engagement (blue), and the enemy pursuit routes or avenues of approach.S3 also allows the user to search for information using text browsing on the web.For example, the user may want to look for doctrinal information on the Army's web page or a CNN news report on a terrorist bombing in Iraq.S3 has both text and voice capabilities.For this experiment, the voice was disabled.
The eleven participants were randomized into "Unaided" and "Aided" groups as shown in Table 3.The schedule was flexible to accommodate the participants time.The "Aided" group was briefed for 75 minutes on the use of S3 by the experimenter during a coordinated group session.The "Unaided" group was told to read the case and develop a sensemaking plan similar to that provided by the subject matter expert on Table 2.The staggering between S1 and S2 presentations were randomized to minimize a usual transfer of learning from one context to another.The "Aided" user can use maps, annotations, and graphics to illustrate facts or clarify arguments.As shown in Exhibit 1, the participant can visualize areas of interests and the likely IED routes (Exhibit 2).The risk levels are classified using data fusion coded symbols.All participants were presented the two cases and given a maximum of 5 minutes to simulate sensemaking under stress.

Data Coding
The experiment is a 2X2 between subject design with two independent variables: two levels of sensemaking visualization tool (Aided, Unaided), and two levels of task scenarios (S1 and S2).Three dependent measures were observed.These are: (a) Percentage of plan match to the Subject Matter's solution in Table 2 (Pmacth), time to finish task (Time in mins.), and the number of cue prompts (CuePrompt) used.A cue prompt is a condition when the participant asks for more information or uses the S3 to browse for information (for the Aided group).

Analysis and Results
The data was analyzed using Statistical Analysis Software (SAS).The primarily analyses were mean and standard deviations of the dependent variables, Analysis of Variance (ANOVA) to test for differences in mean performance as well as possibilities of interaction effect between scenario and affordance to sensemaking exercise.Figures 3-5 shows the mean plots for percentage match (Pmatch) between the participants plan development and the standard plan shown in Table 2, planning times, and the frequency of cue prompts.

Effect of Scenario and Sensemaking Support
By using the percentage of plan match with standards, there were statistical mean differences between the group with sensemaking support (Aided) by score accuracy (the t-statistics, t 0.025,9 = 2.262), p < 0.0001.Similarly, the scenario mean scores were different with p < 0.0043.There was no interaction effect between the scenario and sensemaking support (p = 0.9834).Note that the Aided had a mean percentage match for both scenarios.The worst score was the unaided scenario 2 (See Figure 3).
With planning time, every participant finished within the allotted 5 minutes window.There was evidence of mean differences by scenario (p = 0.0004), sensemaking support (p < 0.0001), and interaction effect between sensemaking support and problem scenario (p = 0.0214).Figure 4 illustrates this possible interaction.The aided group had the lowest planning time, especially in scenario S1.The unaided scenario S2 had the longest planning time of 10.32 minutes.
With cue prompts, there was no interaction effect between scenario and sensemaking support (p = 0.7158).However, there were significant differences in mean number of prompts by scenario (p = 0.0020) and by sensemaking support (p = 0.0319).Figure 5 illustrates this situation.It was noted that an aided sensemaker under scenario S2 will likely seek the same amount of information as the unaided sensemaker.However, the unaided group under S2 had the most prompts.

CONCLUSION
This paper has described an on-going effort to develop empirically-based operational visualization concepts and their technical implementations to support sensemaking skills in complex and chaotic information environments.Particularly for military applications, the Army's FM3.0 doctrinal information on "Visualization, Detection, and Decide" requirements has to be validated through training for cognitive readiness in austere and asymmetrically complex environments.
The scenario problems used are simplified versions of what usually happens in SASO environments.However, they are very atypical in that expert military commanders approved of the scenarios as "good for training" because of their openended natures.The two scenarios are representatives of situated cognition tasks that mimic the components of retrospective knowledge corpus.As the results indicate, aiding the sensemaker with visualization tools improves planning times, increases sensemaking accuracies, and reduces unnecessary information seeking through prompting or browsing.It is noted with consistency that as the task scenario increases, as in S2, the performance degrades noticeable even with aided sensemakers.Scenario S2 is an example task domain that usually requires deliberate planning.Here, reflective cognition is used often and requires more sensemaking time and frequent cue prompting.The results indicate the need to use environment visualization aids to support sensemakers who normally deal with ill-structured problem situations.Note that in this analysis, there are many issues that have not been addressed.Some of these are: (a) discrepancies between sensemakers in selecting a plan of action; (b) effect of negotiation on team planning time and cue prompting frequencies; (d) performance assessment in team sensemaking; and (e) studying the different types of cue prompting and their presentation modalities (e.g., a combination of auditory and visual modes).All these are tasks that are being investigated.

Figure 1 .
Figure 1.A theoretical framework of sensemaking, situation awareness, and situation understanding in context.
Figure 1.AVALANCHE system for supporting visual cognition Search and rescue coalition forces • Secure terrain/ routes and checkpoints to trap the enemy • Conduct noncombatant evacuations for civilians • Provide humanitarian aids • Use search and rescue, and patrol units • Search, track, and rescue the shot down friendly aircraft • Close borders and conduct house-to-house search for the kidnapped politicians • Use: Stryker units search and rescue units, patrol units, talk to civilians to get insight into adversary locations.

Figure 3 .
Figure 3. Mean plot of plan match Figure 4. Mean plot of sensemaking time

Table 1 . The dimensions of system complexity based on information states
ParticipantsVoluntary participants were 16 military instructors from the Air Force-and Army-Reserve Officers' Training Corps (ROTC) at North Carolina A&T State University and some members of reserve component units in Greensboro, North