AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

The number of published systematic reviews of studies of healthcare interventions has increased rapidly and these are used extensively for clinical and policy decisions. Systematic reviews are subject to a range of biases and increasingly include non-randomised studies of interventions. It is important that users can distinguish high quality reviews. Many instruments have been designed to evaluate different aspects of reviews, but there are few comprehensive critical appraisal instruments. AMSTAR was developed to evaluate systematic reviews of randomised trials. In this paper, we report on the updating of AMSTAR and its adaptation to enable more detailed assessment of systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. With moves to base more decisions on real world observational evidence we believe that AMSTAR 2 will assist decision makers in the identification of high quality systematic reviews, including those based on non-randomised studies of healthcare interventions .

Related collections

Most cited references 25

Record: found
Abstract: found
Article: not found

Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

Joel Lexchin, Lisa Bero, Benjamin Djulbegovic … (2003)

To investigate whether funding of drug studies by the pharmaceutical industry is associated with outcomes that are favourable to the funder and whether the methods of trials funded by pharmaceutical companies differ from the methods in trials with other sources of support. Medline (January 1966 to December 2002) and Embase (January 1980 to December 2002) searches were supplemented with material identified in the references and in the authors' personal files. Data were independently abstracted by three of the authors and disagreements were resolved by consensus. 30 studies were included. Research funded by drug companies was less likely to be published than research funded by other sources. Studies sponsored by pharmaceutical companies were more likely to have outcomes favouring the sponsor than were studies with other sponsors (odds ratio 4.05; 95% confidence interval 2.98 to 5.51; 18 comparisons). None of the 13 studies that analysed methods reported that studies funded by industry was of poorer quality. Systematic bias favours products which are made by the company funding the research. Explanations include the selection of an inappropriate comparator to the product being investigated and publication bias.

0 comments Cited 468 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: not found
Article: not found

The hazards of scoring the quality of clinical trials for meta-analysis.

Peter Jüni (1999)

Although it is widely recommended that clinical trials undergo some type of quality review, the number and variety of quality assessment scales that exist make it unclear how to achieve the best assessment. To determine whether the type of quality assessment scale used affects the conclusions of meta-analytic studies. Meta-analysis of 17 trials comparing low-molecular-weight heparin (LMWH) with standard heparin for prevention of postoperative thrombosis using 25 different scales to identify high-quality trials. The association between treatment effect and summary scores and the association with 3 key domains (concealment of treatment allocation, blinding of outcome assessment, and handling of withdrawals) were examined in regression models. Pooled relative risks of deep vein thrombosis with LMWH vs standard heparin in high-quality vs low-quality trials as determined by 25 quality scales. Pooled relative risks from high-quality trials ranged from 0.63 (95% confidence interval [CI], 0.44-0.90) to 0.90 (95% CI, 0.67-1.21) vs 0.52 (95% CI, 0.24-1.09) to 1.13 (95% CI, 0.70-1.82) for low-quality trials. For 6 scales, relative risks of high-quality trials were close to unity, indicating that LMWH was not significantly superior to standard heparin, whereas low-quality trials showed better protection with LMWH (P<.05). Seven scales showed the opposite: high quality trials showed an effect whereas low quality trials did not. For the remaining 12 scales, effect estimates were similar in the 2 quality strata. In regression analysis, summary quality scores were not significantly associated with treatment effects. There was no significant association of treatment effects with allocation concealment and handling of withdrawals. Open outcome assessment, however, influenced effect size with the effect of LMWH, on average, being exaggerated by 35% (95% CI, 1%-57%; P= .046). Our data indicate that the use of summary scores to identify trials of high quality is problematic. Relevant methodological aspects should be assessed individually and their influence on effect sizes explored.

0 comments Cited 381 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)

Beverley Shea, Lex Bouter, Joan Peterson … (2007)

Background Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews. Methodology AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently. Conclusions The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were −0.19±1.38. This translates to a minimum detectable difference between reviews of 0.64 ‘AMSTAR points’. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.

0 comments Cited 196 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Beverley J Shea: Role: senior methodologist, clinical investigator, and adjunct professor

Barnaby C Reeves: Role: professor

George Wells: Role: director and professor

Micere Thuku: Role: research associate

Candyce Hamel: Role: senior clinical research associate

Julian Moran: Role: research student

David Moher: Role: senior scientist, associate professor, and university research chair

Peter Tugwell: Role: senior scientist and professor

Vivian Welch: Role: clinical investigator and assistant professor

Elizabeth Kristjansson: Role: professor

David A Henry: Role: professor and senior scientist

Journal

Journal ID (nlm-ta): BMJ

Journal ID (iso-abbrev): BMJ

Journal ID (publisher-id): BMJ-UK

Journal ID (hwp): bmj

Title: The BMJ

Publisher: BMJ Publishing Group Ltd.

ISSN (Print): 0959-8138

ISSN (Electronic): 1756-1833

Publication date Collection: 2017

Publication date (Electronic): 21 September 2017

Volume: 358

Electronic Location Identifier: j4008

Affiliations

[1 ]Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada

[2 ]Bruyère Research Institute, Ottawa, Canada

[3 ]School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

[4 ]School of Clinical Sciences, University of Bristol, Bristol, UK

[5 ]University of Ottawa Heart Institute, Ottawa, Canada

[6 ]The Hospital for Sick Children, the Genetics and Genome Biology Program, Toronto, Canada

[7 ]Department of Medicine, The Ottawa Hospital, Ottawa, Canada

[8 ]Centre for Research in Educational and Community Services, School of Psychology, Faculty of Social Sciences, University of Ottawa, Canada

[9 ]Centre for Research in Evidence-Based Practice, Bond University, Gold Coast, Australia;

[10 ]Dalla Lana School of Public Health, University of Toronto, Toronto, Canada

[11 ]Institute for Clinical Evaluative Sciences, Toronto, Canada

Author notes

Correspondence to: B J Shea bevshea@ 123456uottawa.ca

Article

Publisher ID: sheb036104

DOI: 10.1136/bmj.j4008

PMC ID: 5833365

PubMed ID: 28935701

SO-VID: 28a7148e-85d8-403d-b4c5-9e3ac2f7d41f

License:

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

Read this article at

Abstract

Related collections

Annual Reviews: Coronavirus Research

Most cited references 25

Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

The hazards of scoring the quality of clinical trials for meta-analysis.

External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)

Author and article information

Contributors

Journal

Affiliations

Author notes

Article

History

Categories

Comments

Comment on this article

Similar content 344

Cited by 2,713

Most referenced authors 1,954