Systematic reviews and meta-analyses have become increasingly important in health
care. Clinicians read them to keep up to date with their field,1,2 and they are often
used as a starting point for developing clinical practice guidelines. Granting agencies
may require a systematic review to ensure there is justification for further research,3
and some health care journals are moving in this direction.4 As with all research,
the value of a systematic review depends on what was done, what was found, and the
clarity of reporting. As with other publications, the reporting quality of systematic
reviews varies, limiting readers' ability to assess the strengths and weaknesses of
those reviews.
Several early studies evaluated the quality of review reports. In 1987, Mulrow examined
50 review articles published in 4 leading medical journals in 1985 and 1986 and found
that none met all 8 explicit scientific criteria, such as a quality assessment of
included studies.5 In 1987, Sacks and colleagues6 evaluated the adequacy of reporting
of 83 meta-analyses on 23 characteristics in 6 domains. Reporting was generally poor;
between 1 and 14 characteristics were adequately reported (mean = 7.7; standard deviation = 2.7).
A 1996 update of this study found little improvement.7
In 1996, to address the suboptimal reporting of meta-analyses, an international group
developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses),
which focused on the reporting of meta-analyses of randomized controlled trials.8
In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred
Reporting Items for Systematic reviews and Meta-Analyses), which have been updated
to address several conceptual and practical advances in the science of systematic
reviews (Box 1).
Terminology
The terminology used to describe a systematic review and meta-analysis has evolved
over time. One reason for changing the name from QUOROM to PRISMA was the desire to
encompass both systematic reviews and meta-analyses. We have adopted the definitions
used by the Cochrane Collaboration.9 A systematic review is a review of a clearly
formulated question that uses systematic and explicit methods to identify, select,
and critically appraise relevant research, and to collect and analyze data from the
studies that are included in the review. Statistical methods (meta-analysis) may or
may not be used to analyze and summarize the results of the included studies. Meta-analysis
refers to the use of statistical techniques in a systematic review to integrate the
results of included studies.
Developing the PRISMA Statement
A 3-day meeting was held in Ottawa, Canada, in June 2005 with 29 participants, including
review authors, methodologists, clinicians, medical editors, and a consumer. The objective
of the Ottawa meeting was to revise and expand the QUOROM checklist and flow diagram,
as needed.
The executive committee completed the following tasks, prior to the meeting: a systematic
review of studies examining the quality of reporting of systematic reviews, and a
comprehensive literature search to identify methodological and other articles that
might inform the meeting, especially in relation to modifying checklist items. An
international survey of review authors, consumers, and groups commissioning or using
systematic reviews and meta-analyses was completed, including the International Network
of Agencies for Health Technology Assessment (INAHTA) and the Guidelines International
Network (GIN). The survey aimed to ascertain views of QUOROM, including the merits
of the existing checklist items. The results of these activities were presented during
the meeting and are summarized on the PRISMA Website.
Only items deemed essential were retained or added to the checklist. Some additional
items are nevertheless desirable, and review authors should include these, if relevant.10
For example, it is useful to indicate whether the systematic review is an update11
of a previous review, and to describe any changes in procedures from those described
in the original protocol.
Shortly after the meeting a draft of the PRISMA checklist was circulated to the group,
including those invited to the meeting but unable to attend. A disposition file was
created containing comments and revisions from each respondent, and the checklist
was subsequently revised 11 times. The group approved the checklist, flow diagram,
and this summary paper.
Although no direct evidence was found to support retaining or adding some items, evidence
from other domains was believed to be relevant. For example, Item 5 asks authors to
provide registration information about the systematic review, including a registration
number, if available. Although systematic review registration is not yet widely available,12,13
the participating journals of the International Committee of Medical Journal Editors
(ICMJE)14 now require all clinical trials to be registered in an effort to increase
transparency and accountability.15 Those aspects are also likely to benefit systematic
reviewers, possibly reducing the risk of an excessive number of reviews addressing
the same question16,17 and providing greater transparency when updating systematic
reviews.
The PRISMA Statement
The PRISMA Statement consists of a 27-item checklist (Table 1; see also Text S1 for
a downloadable template for researchers to re-use) and a 4-phase flow diagram (Figure
1; see also Figure S1 for a downloadable template for researchers to re-use). The
aim of the PRISMA Statement is to help authors improve the reporting of systematic
reviews and meta-analyses. We have focused on randomized trials, but PRISMA can also
be used as a basis for reporting systematic reviews of other types of research, particularly
evaluations of interventions. PRISMA may also be useful for critical appraisal of
published systematic reviews. However, the PRISMA checklist is not a quality assessment
instrument to gauge the quality of a systematic review.
Box 1
Conceptual issues in the evolution from QUOROM to PRISMA
Figure 1
Flow of information through the different phases of a systematic review
Table 1
Checklist of items to include when reporting a systematic review or meta-analysis
From QUOROM to PRISMA
The new PRISMA checklist differs in several respects from the QUOROM checklist, and
the substantive specific changes are highlighted in Table 2. Generally, the PRISMA
checklist “decouples” several items present in the QUOROM checklist and, where applicable,
several checklist items are linked to improve consistency across the systematic review
report.
Table 2
Substantive specific changes between the QUOROM checklist and the PRISMA checklist
(a tick indicates the presence of the topic in QUOROM or PRISMA)
The flow diagram has also been modified. Before including studies and providing reasons
for excluding others, the review team must first search the literature. This search
results in records. Once these records have been screened and eligibility criteria
applied, a smaller number of articles will remain. The number of included articles
might be smaller (or larger) than the number of studies, because articles may report
on multiple studies and results from a particular study may be published in several
articles. To capture this information, the PRISMA flow diagram now requests information
on these phases of the review process.
Endorsement
The PRISMA Statement should replace the QUOROM Statement for those journals that have
endorsed QUOROM. We hope that other journals will support PRISMA; they can do so by
registering on the PRISMA Website. To underscore to authors, and others, the importance
of transparent reporting of systematic reviews, we encourage supporting journals to
reference the PRISMA Statement and include the PRISMA web address in their Instructions
to Authors. We also invite editorial organizations to consider endorsing PRISMA and
encourage authors to adhere to its principles.
The PRISMA Explanation and Elaboration Paper
In addition to the PRISMA Statement, a supporting Explanation and Elaboration document
has been produced18 following the style used for other reporting guidelines.19-21
The process of completing this document included developing a large database of exemplars
to highlight how best to report each checklist item, and identifying a comprehensive
evidence base to support the inclusion of each checklist item. The Explanation and
Elaboration document was completed after several face-to-face meetings and numerous
iterations among several meeting participants, after which it was shared with the
whole group for additional revisions and final approval. Finally, the group formed
a dissemination subcommittee to help disseminate and implement PRISMA.
Discussion
The quality of reporting of systematic reviews is still not optimal.22-27 In a recent
review of 300 systematic reviews, few authors reported assessing possible publication
bias,22 even though there is overwhelming evidence both for its existence28 and its
impact on the results of systematic reviews.29 Even when the possibility of publication
bias is assessed, there is no guarantee that systematic reviewers have assessed or
interpreted it appropriately.30 Although the absence of reporting such an assessment
does not necessarily indicate that it was not done, reporting an assessment of possible
publication bias is likely to be a marker of the thoroughness of the conduct of the
systematic review.
Several approaches have been developed to conduct systematic reviews on a broader
array of questions. For example, systematic reviews are now conducted to investigate
cost-effectiveness,31 diagnostic32 or prognostic questions,33 genetic associations,34
and policy-making.35 The general concepts and topics covered by PRISMA are all relevant
to any systematic review, not just those whose objective is to summarize the benefits
and harms of a health care intervention. However, some modifications of the checklist
items or flow diagram will be necessary in particular circumstances. For example,
assessing the risk of bias is a key concept, but the items used to assess this in
a diagnostic review are likely to focus on issues such as the spectrum of patients
and the verification of disease status, which differ from reviews of interventions.
The flow diagram will also need adjustments when reporting individual patient data
meta-analysis.36
We have developed an explanatory document18 to increase the usefulness of PRISMA.
For each checklist item, this document contains an example of good reporting, a rationale
for its inclusion, and supporting evidence, including references, whenever possible.
We believe this document will also serve as a useful resource for those teaching systematic
review methodology. We encourage journals to include reference to the explanatory
document in their Instructions to Authors.
Like any evidence-based endeavour, PRISMA is a living document. To this end we invite
readers to comment on the revised version, particularly the new checklist and flow
diagram, through the PRISMA website. We will use such information to inform PRISMA's
continued development.
Note: To encourage dissemination of the PRISMA Statement, this article is freely accessible
on the
Open Medicine website and the
PLoS Medicine website and is also published in the Annals of Internal Medicine, BMJ,
and Journal of Clinical Epidemiology. The authors jointly hold the copyright of this
article. For details on further use, see the PRISMA website. The PRISMA Explanation
and Elaboration Paper is available at the PLoS Medicine website.
Supporting Information
Figure S1
Flow of information through the different phases of a systematic review (downloadable
template document for researchers to re-use)
Text S1
Checklist of items to include when reporting a systematic review or meta-analysis
(downloadable template document for researchers to re-use)