Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches

Abstract

Background

Embedded approaches to implementation research (IR), whereby health system decision-makers participate actively in the research process, are gaining traction as effective approaches to optimise the delivery of health programmes and policies. However, the evidence base on the processes and effectiveness of such collaborative research remains inchoate. Standardised approaches to evaluate these initiatives are needed to identify core elements of ‘embeddedness’, unveil the underlying pathways of change, and assess contribution to evidence uptake in decision-making and overall outcomes of effect. The framework presented in this paper responds to this need, designed to guide the systematic evaluation of embedded IR.

Methods

This evaluation framework for embedded IR approaches is based on the experience of a joint initiative by the Pan American Health Organization/Alliance for Health Policy and Systems Research, which has supported 19 IR grants in 10 Latin American and Caribbean countries from 2014 to 2017. The conceptualisation of this framework drew on various sources of information, including empirical evidence and conceptual insights from the literature, interviews with content experts, and a prospective evaluation of the 2016 cohort that included semi-structured key informant interviews, document analysis, and a research team survey to examine key aspects of embedded research.

Results

We developed a widely applicable conceptual framework to guide the evaluation of embedded IR in various contexts. Focused on uncovering how this collaborative research approach influences programme improvement, it outlines expected processes and intermediate outcomes. It also highlights constructs with which to assess ‘embeddedness’ as well as critical contextual factors. The framework is intended to provide a structure by which to systematically examine such embedded research initiatives, proposing three key stages of evidence-informed decision-making – co-production of evidence, engagement with research, and enactment of programme changes.

Conclusion

Rigorous evaluation of embedded IR is needed to build the evidence on its processes and effectiveness in influencing decision-making. The evaluation framework presented here addresses this gap with consideration of the complexity of such efforts. Its applicability to similar initiatives is bolstered by virtue of being founded on real-world experience; its potential to contribute to a nuanced understanding of embedded IR is significant.