168
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

      other

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          The number of published systematic reviews of studies of healthcare interventions has increased rapidly and these are used extensively for clinical and policy decisions. Systematic reviews are subject to a range of biases and increasingly include non-randomised studies of interventions. It is important that users can distinguish high quality reviews. Many instruments have been designed to evaluate different aspects of reviews, but there are few comprehensive critical appraisal instruments. AMSTAR was developed to evaluate systematic reviews of randomised trials. In this paper, we report on the updating of AMSTAR and its adaptation to enable more detailed assessment of systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. With moves to base more decisions on real world observational evidence we believe that AMSTAR 2 will assist decision makers in the identification of high quality systematic reviews, including those based on non-randomised studies of healthcare interventions .

          Related collections

          Most cited references25

          • Record: found
          • Abstract: found
          • Article: not found

          Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

          To investigate whether funding of drug studies by the pharmaceutical industry is associated with outcomes that are favourable to the funder and whether the methods of trials funded by pharmaceutical companies differ from the methods in trials with other sources of support. Medline (January 1966 to December 2002) and Embase (January 1980 to December 2002) searches were supplemented with material identified in the references and in the authors' personal files. Data were independently abstracted by three of the authors and disagreements were resolved by consensus. 30 studies were included. Research funded by drug companies was less likely to be published than research funded by other sources. Studies sponsored by pharmaceutical companies were more likely to have outcomes favouring the sponsor than were studies with other sponsors (odds ratio 4.05; 95% confidence interval 2.98 to 5.51; 18 comparisons). None of the 13 studies that analysed methods reported that studies funded by industry was of poorer quality. Systematic bias favours products which are made by the company funding the research. Explanations include the selection of an inappropriate comparator to the product being investigated and publication bias.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            The hazards of scoring the quality of clinical trials for meta-analysis.

            Although it is widely recommended that clinical trials undergo some type of quality review, the number and variety of quality assessment scales that exist make it unclear how to achieve the best assessment. To determine whether the type of quality assessment scale used affects the conclusions of meta-analytic studies. Meta-analysis of 17 trials comparing low-molecular-weight heparin (LMWH) with standard heparin for prevention of postoperative thrombosis using 25 different scales to identify high-quality trials. The association between treatment effect and summary scores and the association with 3 key domains (concealment of treatment allocation, blinding of outcome assessment, and handling of withdrawals) were examined in regression models. Pooled relative risks of deep vein thrombosis with LMWH vs standard heparin in high-quality vs low-quality trials as determined by 25 quality scales. Pooled relative risks from high-quality trials ranged from 0.63 (95% confidence interval [CI], 0.44-0.90) to 0.90 (95% CI, 0.67-1.21) vs 0.52 (95% CI, 0.24-1.09) to 1.13 (95% CI, 0.70-1.82) for low-quality trials. For 6 scales, relative risks of high-quality trials were close to unity, indicating that LMWH was not significantly superior to standard heparin, whereas low-quality trials showed better protection with LMWH (P<.05). Seven scales showed the opposite: high quality trials showed an effect whereas low quality trials did not. For the remaining 12 scales, effect estimates were similar in the 2 quality strata. In regression analysis, summary quality scores were not significantly associated with treatment effects. There was no significant association of treatment effects with allocation concealment and handling of withdrawals. Open outcome assessment, however, influenced effect size with the effect of LMWH, on average, being exaggerated by 35% (95% CI, 1%-57%; P= .046). Our data indicate that the use of summary scores to identify trials of high quality is problematic. Relevant methodological aspects should be assessed individually and their influence on effect sizes explored.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)

              Background Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews. Methodology AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently. Conclusions The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were −0.19±1.38. This translates to a minimum detectable difference between reviews of 0.64 ‘AMSTAR points’. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.
                Bookmark

                Author and article information

                Contributors
                Role: senior methodologist, clinical investigator, and adjunct professor
                Role: professor
                Role: director and professor
                Role: research associate
                Role: senior clinical research associate
                Role: research student
                Role: senior scientist, associate professor, and university research chair
                Role: senior scientist and professor
                Role: clinical investigator and assistant professor
                Role: professor
                Role: professor and senior scientist
                Journal
                BMJ
                BMJ
                BMJ-UK
                bmj
                The BMJ
                BMJ Publishing Group Ltd.
                0959-8138
                1756-1833
                2017
                21 September 2017
                : 358
                : j4008
                Affiliations
                [1 ]Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada
                [2 ]Bruyère Research Institute, Ottawa, Canada
                [3 ]School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada
                [4 ]School of Clinical Sciences, University of Bristol, Bristol, UK
                [5 ]University of Ottawa Heart Institute, Ottawa, Canada
                [6 ]The Hospital for Sick Children, the Genetics and Genome Biology Program, Toronto, Canada
                [7 ]Department of Medicine, The Ottawa Hospital, Ottawa, Canada
                [8 ]Centre for Research in Educational and Community Services, School of Psychology, Faculty of Social Sciences, University of Ottawa, Canada
                [9 ]Centre for Research in Evidence-Based Practice, Bond University, Gold Coast, Australia;
                [10 ]Dalla Lana School of Public Health, University of Toronto, Toronto, Canada
                [11 ]Institute for Clinical Evaluative Sciences, Toronto, Canada
                Author notes
                Correspondence to: B J Shea bevshea@ 123456uottawa.ca
                Article
                sheb036104
                10.1136/bmj.j4008
                5833365
                28935701
                28a7148e-85d8-403d-b4c5-9e3ac2f7d41f
                Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions

                This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

                History
                : 04 August 2017
                Categories
                Research Methods & Reporting

                Medicine
                Medicine

                Comments

                Comment on this article