Towards a More Evidence-Based Risk Assessment for People in the Criminal Justice System: the Case of OxRec in the Netherlands

Abstract

Risk assessment tools are widely used throughout the criminal justice system to assist in making decisions about sentencing, supervision, and treatment. In this article, we discuss several methodological and practical limitations associated with risk assessment tools currently in use. These include variable predictive performance due to the exclusion of important background predictors; high costs, including the need for regular staff training, in order to use many tools; development of tools using suboptimal methods and poor transparency in how they create risk scores; included risk factors being based on dated evidence; and ethical concerns highlighted by legal scholars and criminologists, such as embedding systemic biases and uncertainty about how these tools influence judicial decisions. We discuss the potential that specific predictors, such as living in a deprived neighbourhood, may indirectly select for individuals in racial or ethnic minority groups. To demonstrate how these limitations and ethical concerns can be addressed, we present the example of OxRec, a risk assessment tool used to predict recidivism for individuals in the criminal justice system. OxRec was developed in Sweden and has been externally validated in Sweden and the Netherlands. The advantages of OxRec include its predictive accuracy based on rigorous multivariable testing of predictors, transparent reporting of results and the final model (including how the probability score is derived), scoring simplicity (i.e. without the need for additional interview), and the reporting of a wide range of performance measures, including those of discrimination and calibration, the latter of which is rarely reported but a key metric. OxRec is intended to be used alongside professional judgement, as a support for decision-making, and its performance measures need to be interpreted in this light. The reported calibration of the tool in external samples clearly suggests no systematic overestimation of risk, including in large subgroups.