Systematic reviews and meta-analyses have become increasingly important in health
care. Clinicians read them to keep up to date with their field ,, and they are
often used as a starting point for developing clinical practice guidelines. Granting
agencies may require a systematic review to ensure there is justification for further
research , and some health care journals are moving in this direction . As with
all research, the value of a systematic review depends on what was done, what was
found, and the clarity of reporting. As with other publications, the reporting quality
of systematic reviews varies, limiting readers' ability to assess the strengths and
weaknesses of those reviews.
Several early studies evaluated the quality of review reports. In 1987, Mulrow examined
50 review articles published in four leading medical journals in 1985 and 1986 and
found that none met all eight explicit scientific criteria, such as a quality assessment
of included studies . In 1987, Sacks and colleagues  evaluated the adequacy
of reporting of 83 meta-analyses on 23 characteristics in six domains. Reporting was
generally poor; between one and 14 characteristics were adequately reported (mean = 7.7;
standard deviation = 2.7). A 1996 update of this study found little improvement .
In 1996, to address the suboptimal reporting of meta-analyses, an international group
developed a guidance called the QUOROM Statement (QUality Of Reporting Of Meta-analyses),
which focused on the reporting of meta-analyses of randomized controlled trials .
In this article, we summarize a revision of these guidelines, renamed PRISMA (Preferred
Reporting Items for Systematic reviews and Meta-Analyses), which have been updated
to address several conceptual and practical advances in the science of systematic
reviews (Box 1).
Box 1: Conceptual Issues in the Evolution from QUOROM to PRISMA
Completing a Systematic Review Is an Iterative Process
The conduct of a systematic review depends heavily on the scope and quality of included
studies: thus systematic reviewers may need to modify their original review protocol
during its conduct. Any systematic review reporting guideline should recommend that
such changes can be reported and explained without suggesting that they are inappropriate.
The PRISMA Statement (Items 5, 11, 16, and 23) acknowledges this iterative process.
Aside from Cochrane reviews, all of which should have a protocol, only about 10% of
systematic reviewers report working from a protocol . Without a protocol that
is publicly accessible, it is difficult to judge between appropriate and inappropriate
Conduct and Reporting Research Are Distinct Concepts
This distinction is, however, less straightforward for systematic reviews than for
assessments of the reporting of an individual study, because the reporting and conduct
of systematic reviews are, by nature, closely intertwined. For example, the failure
of a systematic review to report the assessment of the risk of bias in included studies
may be seen as a marker of poor conduct, given the importance of this activity in
the systematic review process .
Study-Level Versus Outcome-Level Assessment of Risk of Bias
For studies included in a systematic review, a thorough assessment of the risk of
bias requires both a “study-level” assessment (e.g., adequacy of allocation concealment)
and, for some features, a newer approach called “outcome-level” assessment. An outcome-level
assessment involves evaluating the reliability and validity of the data for each important
outcome by determining the methods used to assess them in each individual study .
The quality of evidence may differ across outcomes, even within a study, such as between
a primary efficacy outcome, which is likely to be very carefully and systematically
measured, and the assessment of serious harms , which may rely on spontaneous
reports by investigators. This information should be reported to allow an explicit
assessment of the extent to which an estimate of effect is correct .
Importance of Reporting Biases
Different types of reporting biases may hamper the conduct and interpretation of systematic
reviews. Selective reporting of complete studies (e.g., publication bias)  as
well as the more recently empirically demonstrated “outcome reporting bias” within
individual studies , should be considered by authors when conducting a systematic
review and reporting its results. Though the implications of these biases on the conduct
and reporting of systematic reviews themselves are unclear, some previous research
has identified that selective outcome reporting may occur also in the context of systematic
The terminology used to describe a systematic review and meta-analysis has evolved
over time. One reason for changing the name from QUOROM to PRISMA was the desire to
encompass both systematic reviews and meta-analyses. We have adopted the definitions
used by the Cochrane Collaboration . A systematic review is a review of a clearly
formulated question that uses systematic and explicit methods to identify, select,
and critically appraise relevant research, and to collect and analyze data from the
studies that are included in the review. Statistical methods (meta-analysis) may or
may not be used to analyze and summarize the results of the included studies. Meta-analysis
refers to the use of statistical techniques in a systematic review to integrate the
results of included studies.
Developing the PRISMA Statement
A three-day meeting was held in Ottawa, Canada, in June 2005 with 29 participants,
including review authors, methodologists, clinicians, medical editors, and a consumer.
The objective of the Ottawa meeting was to revise and expand the QUOROM checklist
and flow diagram, as needed.
The executive committee completed the following tasks, prior to the meeting: a systematic
review of studies examining the quality of reporting of systematic reviews, and a
comprehensive literature search to identify methodological and other articles that
might inform the meeting, especially in relation to modifying checklist items. An
international survey of review authors, consumers, and groups commissioning or using
systematic reviews and meta-analyses was completed, including the International Network
of Agencies for Health Technology Assessment (INAHTA) and the Guidelines International
Network (GIN). The survey aimed to ascertain views of QUOROM, including the merits
of the existing checklist items. The results of these activities were presented during
the meeting and are summarized on the PRISMA Web site (http://www.prisma-statement.org/).
Only items deemed essential were retained or added to the checklist. Some additional
items are nevertheless desirable, and review authors should include these, if relevant
. For example, it is useful to indicate whether the systematic review is an update
 of a previous review, and to describe any changes in procedures from those described
in the original protocol.
Shortly after the meeting a draft of the PRISMA checklist was circulated to the group,
including those invited to the meeting but unable to attend. A disposition file was
created containing comments and revisions from each respondent, and the checklist
was subsequently revised 11 times. The group approved the checklist, flow diagram,
and this summary paper.
Although no direct evidence was found to support retaining or adding some items, evidence
from other domains was believed to be relevant. For example, Item 5 asks authors to
provide registration information about the systematic review, including a registration
number, if available. Although systematic review registration is not yet widely available
,, the participating journals of the International Committee of Medical Journal
Editors (ICMJE)  now require all clinical trials to be registered in an effort
to increase transparency and accountability . Those aspects are also likely to
benefit systematic reviewers, possibly reducing the risk of an excessive number of
reviews addressing the same question , and providing greater transparency
when updating systematic reviews.
The PRISMA Statement
The PRISMA Statement consists of a 27-item checklist (Table 1; see also Text S1 for
a downloadable Word template for researchers to re-use) and a four-phase flow diagram
(Figure 1; see also Figure S1 for a downloadable Word template for researchers to
re-use). The aim of the PRISMA Statement is to help authors improve the reporting
of systematic reviews and meta-analyses. We have focused on randomized trials, but
PRISMA can also be used as a basis for reporting systematic reviews of other types
of research, particularly evaluations of interventions. PRISMA may also be useful
for critical appraisal of published systematic reviews. However, the PRISMA checklist
is not a quality assessment instrument to gauge the quality of a systematic review.
Flow of information through the different phases of a systematic review.
Checklist of items to include when reporting a systematic review or meta-analysis.
Reported on Page #
Identify the report as a systematic review, meta-analysis, or both.
Provide a structured summary including, as applicable: background; objectives; data
sources; study eligibility criteria, participants, and interventions; study appraisal
and synthesis methods; results; limitations; conclusions and implications of key findings;
systematic review registration number.
Describe the rationale for the review in the context of what is already known.
Provide an explicit statement of questions being addressed with reference to participants,
interventions, comparisons, outcomes, and study design (PICOS).
Protocol and registration
Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address),
and, if available, provide registration information including registration number.
Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics
(e.g., years considered, language, publication status) used as criteria for eligibility,
Describe all information sources (e.g., databases with dates of coverage, contact
with study authors to identify additional studies) in the search and date last searched.
Present full electronic search strategy for at least one database, including any limits
used, such that it could be repeated.
State the process for selecting studies (i.e., screening, eligibility, included in
systematic review, and, if applicable, included in the meta-analysis).
Data collection process
Describe method of data extraction from reports (e.g., piloted forms, independently,
in duplicate) and any processes for obtaining and confirming data from investigators.
List and define all variables for which data were sought (e.g., PICOS, funding sources)
and any assumptions and simplifications made.
Risk of bias in individual studies
Describe methods used for assessing risk of bias of individual studies (including
specification of whether this was done at the study or outcome level), and how this
information is to be used in any data synthesis.
State the principal summary measures (e.g., risk ratio, difference in means).
Synthesis of results
Describe the methods of handling data and combining results of studies, if done, including
measures of consistency (e.g., I2) for each meta-analysis.
Risk of bias across studies
Specify any assessment of risk of bias that may affect the cumulative evidence (e.g.,
publication bias, selective reporting within studies).
Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression),
if done, indicating which were pre-specified.
Give numbers of studies screened, assessed for eligibility, and included in the review,
with reasons for exclusions at each stage, ideally with a flow diagram.
For each study, present characteristics for which data were extracted (e.g., study
size, PICOS, follow-up period) and provide the citations.
Risk of bias within studies
Present data on risk of bias of each study and, if available, any outcome-level assessment
(see Item 12).
Results of individual studies
For all outcomes considered (benefits or harms), present, for each study: (a) simple
summary data for each intervention group and (b) effect estimates and confidence intervals,
ideally with a forest plot.
Synthesis of results
Present results of each meta-analysis done, including confidence intervals and measures
Risk of bias across studies
Present results of any assessment of risk of bias across studies (see Item 15).
Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses,
meta-regression [see Item 16]).
Summary of evidence
Summarize the main findings including the strength of evidence for each main outcome;
consider their relevance to key groups (e.g., health care providers, users, and policy
Discuss limitations at study and outcome level (e.g., risk of bias), and at review
level (e.g., incomplete retrieval of identified research, reporting bias).
Provide a general interpretation of the results in the context of other evidence,
and implications for future research.
Describe sources of funding for the systematic review and other support (e.g., supply
of data); role of funders for the systematic review.
From QUOROM to PRISMA
The new PRISMA checklist differs in several respects from the QUOROM checklist, and
the substantive specific changes are highlighted in Table 2. Generally, the PRISMA
checklist “decouples” several items present in the QUOROM checklist and, where applicable,
several checklist items are linked to improve consistency across the systematic review
Substantive specific changes between the QUOROM checklist and the PRISMA checklist
(a tick indicates the presence of the topic in QUOROM or PRISMA).
QUOROM and PRISMA ask authors to report an abstract. However, PRISMA is not specific
This new item (4) addresses the explicit question the review addresses using the PICO
reporting system (which describes the participants, interventions, comparisons, and
outcome(s) of the systematic review), together with the specification of the type
of study design (PICOS); the item is linked to Items 6, 11, and 18 of the checklist.
This new item (5) asks authors to report whether the review has a protocol and if
so how it can be accessed.
Although reporting the search is present in both QUOROM and PRISMA checklists, PRISMA
asks authors to provide a full description of at least one electronic search strategy
(Item 8). Without such information it is impossible to repeat the authors' search.
Assessment of risk of bias in included studies
Renamed from “quality assessment” in QUOROM. This item (12) is linked with reporting
this information in the results (Item 19). The new concept of “outcome-level” assessment
has been introduced.
Assessment of risk of bias across studies
This new item (15) asks authors to describe any assessments of risk of bias in the
review, such as selective reporting within the included studies. This item is linked
with reporting this information in the results (Item 22).
Although both QUOROM and PRISMA checklists address the discussion section, PRISMA
devotes three items (24–26) to the discussion. In PRISMA the main types of limitations
are explicitly stated and their discussion required.
This new item (27) asks authors to provide information on any sources of funding for
the systematic review.
The flow diagram has also been modified. Before including studies and providing reasons
for excluding others, the review team must first search the literature. This search
results in records. Once these records have been screened and eligibility criteria
applied, a smaller number of articles will remain. The number of included articles
might be smaller (or larger) than the number of studies, because articles may report
on multiple studies and results from a particular study may be published in several
articles. To capture this information, the PRISMA flow diagram now requests information
on these phases of the review process.
The PRISMA Statement should replace the QUOROM Statement for those journals that have
endorsed QUOROM. We hope that other journals will support PRISMA; they can do so by
registering on the PRISMA Web site. To underscore to authors, and others, the importance
of transparent reporting of systematic reviews, we encourage supporting journals to
reference the PRISMA Statement and include the PRISMA Web address in their Instructions
to Authors. We also invite editorial organizations to consider endorsing PRISMA and
encourage authors to adhere to its principles.
The PRISMA Explanation and Elaboration Paper
In addition to the PRISMA Statement, a supporting Explanation and Elaboration document
has been produced  following the style used for other reporting guidelines –.
The process of completing this document included developing a large database of exemplars
to highlight how best to report each checklist item, and identifying a comprehensive
evidence base to support the inclusion of each checklist item. The Explanation and
Elaboration document was completed after several face to face meetings and numerous
iterations among several meeting participants, after which it was shared with the
whole group for additional revisions and final approval. Finally, the group formed
a dissemination subcommittee to help disseminate and implement PRISMA.
The quality of reporting of systematic reviews is still not optimal –. In
a recent review of 300 systematic reviews, few authors reported assessing possible
publication bias , even though there is overwhelming evidence both for its existence
 and its impact on the results of systematic reviews . Even when the possibility
of publication bias is assessed, there is no guarantee that systematic reviewers have
assessed or interpreted it appropriately . Although the absence of reporting such
an assessment does not necessarily indicate that it was not done, reporting an assessment
of possible publication bias is likely to be a marker of the thoroughness of the conduct
of the systematic review.
Several approaches have been developed to conduct systematic reviews on a broader
array of questions. For example, systematic reviews are now conducted to investigate
cost-effectiveness , diagnostic  or prognostic questions , genetic associations
, and policy making . The general concepts and topics covered by PRISMA are
all relevant to any systematic review, not just those whose objective is to summarize
the benefits and harms of a health care intervention. However, some modifications
of the checklist items or flow diagram will be necessary in particular circumstances.
For example, assessing the risk of bias is a key concept, but the items used to assess
this in a diagnostic review are likely to focus on issues such as the spectrum of
patients and the verification of disease status, which differ from reviews of interventions.
The flow diagram will also need adjustments when reporting individual patient data
We have developed an explanatory document  to increase the usefulness of PRISMA.
For each checklist item, this document contains an example of good reporting, a rationale
for its inclusion, and supporting evidence, including references, whenever possible.
We believe this document will also serve as a useful resource for those teaching systematic
review methodology. We encourage journals to include reference to the explanatory
document in their Instructions to Authors.
Like any evidence-based endeavor, PRISMA is a living document. To this end we invite
readers to comment on the revised version, particularly the new checklist and flow
diagram, through the PRISMA Web site. We will use such information to inform PRISMA's
Flow of information through the different phases of a systematic review (downloadable
template document for researchers to re-use).
(0.08 MB DOC)
Click here for additional data file.
Checklist of items to include when reporting a systematic review or meta-analysis
(downloadable template document for researchers to re-use).
(0.04 MB DOC)
Click here for additional data file.