Developing Pattern Languages: New Ways of Communicating Naturalistic Insights for System Development and Evaluation

Motivation – The Naturalistic Decision Making (NDM) community benefits from having its insights used in software development projects. Research approach – We are investigating the pattern form as a way to incorporate findings from naturalistic research into software development. The socio-technical level of description – how technology impacts groups, teams, and organizations – is the focus of our work. Findings/Design – Pattern form offers the advantage of focusing interpretation and connecting to a solution outline over more traditional ways of making findings available. These advantages also make the pattern form good for use in the evaluation of systems where the need for solutions is immediate. Research limitations/Implications – The pattern form is not a panacea and will not replace NDM participation in development projects. Originality/Value – Patterns have not been used for evaluation at the socio-technical level. Take away message – Patterns may be useful for communicating NDM insights in development projects, particularly those at the socio-technical level.


INTRODUCTION
Getting systems engineers and designers to make use of NDM research results and insights when they are creating systems is challenging.We are investigating the pattern form as a vehicle for effectively presenting findings from naturalistic research to those involved in software development.The software development community has embraced the pattern concept (Buschmann, Henney, & Schmidt, 2007) for software internals, software development processes, organization, and user interface design (Dearden & Findlay, 2006).Since the pattern form is accepted in allied disciplines we are investigating whether this form is effective in presenting the rich contextual understandings that characterize expertise and the findings from naturalistic research.

COMMUNICATING INSIGHT
The best way to create systems that support the reality of cognitive work is to have NDM researchers involved in system development throughout the lifecycle.For a myriad of reasons this does not happen and the quality of support for cognitive work suffers greatly as a result.Some of these failings have been captured by "anti-patterns" (Laplante, Hoffman, & Klein, 2007).We see the pattern form as a rich, structured way of representing NDM insight that will help engineers and designers incorporate those insights.We agree with Stanard, Wampler, Conrad & Osga (2006) that patterns will have the biggest impact on systems that have limited NDM or Cognitive Systems Engineering researcher involvement.These systems should be the ones in already well understood domains.
The pattern form, particularly when organized into a pattern language, has two main benefits in our view.The first benefit is focusing interpretation.Often the ways of communicating NDM insight requires interpretation in order to apply them.This interpretation is easy for NDM researchers to do but hard for engineers and designers who do not have the same expertise.Concrete stories of work solutions are a prime example of this.These stories are very evocative for researchers, but are interpreted very differently by engineers and designers.Another common form of communicating insight is collections of laws, heuristics, or general principles such as the Cognitive Performance Indicators (Long & Cox, 2007).These also require much interpretation to apply to full effect.
The second benefit of the pattern form is that a solution is indicated.By documenting the problem to be addressed, the forces at work, and how to resolve them, the pattern form focuses the interpretation needed to successfully incorporate the insight into system design and development.Solutions are not present in work stories or general principles.Outlining the solution in the pattern saves the developers from having to invent their own solution.Presenting a solution limits pattens to addressing problems for which there are accepted solutions.

WHAT IS A PATTERN?
A is a form for communicating general and tested design knowledge.At its core, a pattern expresses a design solution to resolving apparently conflicting forces in a context.We include a rough outline of a pattern from our own socio-technical observations to suggest what might be contained in a pattern.

Name: Divide And Share
Context: In control rooms, the work is often divided up between the members of the control team.This provides focus for the team members and saves them from having to explicitly coordinate ownership of every issue that comes up.On teams where people have overlapping expertise, problems that are in the overlap can be addressed by more than one person.

Pattern description:
In control rooms, work is divided up between control team members based on a long term view of the workload of operators and operator's skills.Organizations often train their operators to have overlapping expertise.This increases team resilience.Often situations arise where operator workload is not equalized by the default divisions of operator focus.These cases require the operators to renegotiate who is addressing which problems.The other operators must be able to take the secondary problems from the overloaded operator with minimum overhead and the whole team must remain coordinated so that all the problems do get addressed.

Solution:
The system focuses the operator on the part of the system that is the primary responsibility.The system provides sufficient overview for the operator to make sense of the whole system and other operators actions within the system.The system supports the operator in receiving additional work outside the focus area from other operators, and in offering to take on parts of a problem another operator may be having.The system ensures that sufficient context is available to the receiving operator, and that the owning operator can keep track of the state of the problem and actions by the receiving operator.

EVALUATING USING PATTERNS
Patterns have been presented most often as a resource for design.They can also be a resource for evaluation.NDM researchers are often engaged in projects after design is complete and evaluation has commenced.At this point in the development process, there is little room for more research or extensive design activities.Problems without solutions will not be addressed in the current release.Patterns allow the critique and the solution to be presented together, increasing the chances that the system will be changed.Patterns are a particularly useful evaluation tool because they couple domain understanding with the design solution.The domain understanding is present in the description of the context and the pattern.Patterns can be used both in early reviews and later more formal evaluations.

CONCLUSION
Patterns are a way of presenting NDM insight for use in software development.They will not replace having NDM researchers involved in software projects.We are evaluating patterns as a way to raise the level of discourse in development teams.If development teams are working with patterns based on NDM insight as building blocks the resulting software will be better whatever the level of involvement of NDM researchers.This should result in fewer systems with antipatterns (Laplante et al, 2007)