A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions

Abstract

Background

Overviews of reviews of healthcare interventions (overviews) integrate information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overviews may identify multiple SRs that examine the same intervention for the same condition and include some, but not all, of the same primary studies. Different researchers use different approaches to manage these “overlapping SRs,” but each approach has advantages and disadvantages. This study aimed to develop an evidence-based decision tool to help researchers make informed inclusion decisions when conducting overviews of healthcare interventions.

Methods

We used a two-stage process to develop the decision tool. First, we conducted a multiple case study to obtain empirical evidence upon which the tool is based. We systematically conducted seven overviews five times each, making five different decisions about which SRs to include in the overviews, for a total of 35 overviews; we then examined the impact of the five inclusion decisions on the overviews’ comprehensiveness and challenges, within and across the seven overview cases. Second, we used a structured, iterative process to transform the evidence obtained from the multiple case study into an empirically based decision tool with accompanying descriptive text.

Results

The resulting decision tool contains four questions: (1) Do Cochrane SRs likely examine all relevant intervention comparisons and available data? (2) Do the Cochrane SRs overlap? (3) Do the non-Cochrane SRs overlap? (4) Are researchers prepared and able to avoid double-counting outcome data from overlapping SRs, by ensuring that each primary study’s outcome data are extracted from overlapping SRs only once? Guidance is provided to help researchers answer each question, and empirical evidence is provided regarding the advantages, disadvantages, and potential trade-offs of the different inclusion decisions.

Conclusions

This evidence-based decision tool is designed to provide researchers with the knowledge and means to make informed inclusion decisions in overviews. The tool can provide practical guidance and support for overview authors by helping them consider questions that could affect the comprehensiveness and complexity of their overviews. We hope this tool will be a useful resource for researchers conducting overviews, and we welcome discussion, testing, and refinement of the proposed tool.