Expert Judgment in Contaminated Land Assessment

Motivation – Engineering judgment forms an integral part of the contaminated land risk assessments required to develop land in the UK (ODPM, 2004). This study investigated how the complex, uncertain site information is used by expert engineers in these assessments. Research approach – Site reports with different cue combinations were assessed and the findings were compared quantitatively to the predictions of three theories of decision making: Recognition-primed decision making (RPD), fast and frugal heuristics and the Lens model. Findings / Design – Support was found for a process of combining several sources of information simultaneously, rather than a matching heuristic in which cues are tested serially. Experts appear to focus on a limited set of information rather than treating all cues equally as recommended in current guidance (Defra, 2004). Research limitations / Implications – The site reports and assessment format, although reflective of real projects, was simplified to meet time constraints of the participants. The sample size used was sufficient to fit the chosen models to the average expert response, though insufficient to enable predicting of individual responses. Both of these aspects could be addressed with further research. Originality / Value – An understanding of expert judgment in contaminated land assessment can be used to improve future training and guidance required to address the current industry skill shortage (ASC, 2008). Take away message – Despite the apparently holistic approach to contaminated land risk assessment described in the qualitative responses to this study, the numeric analysis of the expert’s rating of risk showed that it was based on a limited number of cues.


INTRODUCTION
Contaminated land assessment is a complex process in which expert engineers must judge the level of risk posed on a site.This requires the identification and assessment of "pollutant linkages", comprising a contamination source, a receptor and a pathway linking them.Experts use reports which describe the detailed site data: site history, geology, hydrogeology, and site investigation data including chemical test results.The commercial context in which data are collected leads to inevitable gaps and uncertainties.Therefore engineers must make judgments about risk informed by their own expertise, based on complex and incomplete information; this study investigated the risk assessment process.
Several theoretical alternatives have been proposed to explain expert decision making.RPD (e.g.Klein, 1998) proposed that experts are sensitive to cues and perceiving familiar combinations leads directly to a single course of action.Gigerenzer (e.g.Gigerenzer & Goldstein, 1996) proposed that fast and frugal heuristics are used to reach a decision.For example, the Matching Heuristic tests the most valid cue to see if a decision can be reached immediately.If not, the next most valid cue is tested and so on until a decision can be made.The emphasis on recognising cues is a similarity in both approaches, but the parallel assessment of cues in RPD and serial testing with the Matching Heuristic are a clear difference.Finally, the Lens model (e.g.Hammond & Stewart, 2001) proposed that judgments are made using a weighted combination of the cues.Experts' judgments of risk in twenty-seven contaminated land cases were elicited.The three decision theories were compared to the expert's accounts of the risk from contamination to establish the "best fitting" decision process.

METHOD
Thirty contaminated land assessors participated in the study, with a mean experience of fifteen years.The study required each participant to assess a portfolio of hypothetical sites for a housing developer seeking to invest in land for development.The portfolio of information presented to participants closely mirrored the type of reports in practice.For each site, participants reviewed a brief site description and limited chemical data to judge the potential risk from contamination.The site information consisted of cues reflecting the basic information required to assess "pollutant linkages".Based on industry guidance and expert interviews, each cue was designed to be a high, moderate or low risk indicator, although they were not explicitly presented as such to participants.The participants rated each case as high, moderate or low risk and provided their level of confidence in their response.

RESULTS & DISCUSSION
The risk assessments made by the thirty experts were averaged for each site.Linear regression analysis was then used to identify the significance of each the available cues for the average risk assessments given by participants.The relative significance of the cues formed the basis for applying the different theories of decision making described above to establish the best fitting model.Initial findings show that the cues concerning the contamination source influence the experts' judgment of the level of risk much more than the pathway and receptor cues.However, the average of the source related cues correspond more closely to the risk assessment than the serial approach of testing the most valid cue followed by the next valid cue etc.Finally, a Lens model was created by regressing the cues onto the responses and a selected combination of source cues that most influenced the judgement of risk (not all cues were influential).Table 1 shows the fit of each model against the average risk scores given by participants.

CONCLUSION
These findings suggest that expert judgments in this context are made using a weighted combination of cues, rather than a fast and frugal heuristic based on recognition of individual cues.The above analysis also suggests that experts do not seem to use all the information available, which contrasts with current official guidance.However, analysis of the qualitative open questions in the study suggests that experts' judgments of risk are more holistic, and involve constructing a story about the site to explain the information presented.This study has practical significance: a clearer understanding of the risk assessment process will enhance and accelerate training of contaminated land assessors to address the current UK skills shortage.