195
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: not found

      External Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)

      research-article

      Read this article at

      ScienceOpenPublisherPMC
      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Thousands of systematic reviews have been conducted in all areas of health care. However, the methodological quality of these reviews is variable and should routinely be appraised. AMSTAR is a measurement tool to assess systematic reviews.

          Methodology

          AMSTAR was used to appraise 42 reviews focusing on therapies to treat gastro-esophageal reflux disease, peptic ulcer disease, and other acid-related diseases. Two assessors applied the AMSTAR to each review. Two other assessors, plus a clinician and/or methodologist applied a global assessment to each review independently.

          Conclusions

          The sample of 42 reviews covered a wide range of methodological quality. The overall scores on AMSTAR ranged from 0 to 10 (out of a maximum of 11) with a mean of 4.6 (95% CI: 3.7 to 5.6) and median 4.0 (range 2.0 to 6.0). The inter-observer agreement of the individual items ranged from moderate to almost perfect agreement. Nine items scored a kappa of >0.75 (95% CI: 0.55 to 0.96). The reliability of the total AMSTAR score was excellent: kappa 0.84 (95% CI: 0.67 to 1.00) and Pearson's R 0.96 (95% CI: 0.92 to 0.98). The overall scores for the global assessment ranged from 2 to 7 (out of a maximum score of 7) with a mean of 4.43 (95% CI: 3.6 to 5.3) and median 4.0 (range 2.25 to 5.75). The agreement was lower with a kappa of 0.63 (95% CI: 0.40 to 0.88). Construct validity was shown by AMSTAR convergence with the results of the global assessment: Pearson's R 0.72 (95% CI: 0.53 to 0.84). For the AMSTAR total score, the limits of agreement were −0.19±1.38. This translates to a minimum detectable difference between reviews of 0.64 ‘AMSTAR points’. Further validation of AMSTAR is needed to assess its validity, reliability and perceived utility by appraisers and end users of reviews across a broader range of systematic reviews.

          Related collections

          Most cited references75

          • Record: found
          • Abstract: found
          • Article: not found

          Statistical methods for assessing agreement between two methods of clinical measurement.

          In clinical measurement comparison of a new measurement technique with an established one is often needed to see whether they agree sufficiently for the new to replace the old. Such investigations are often analysed inappropriately, notably by using correlation coefficients. The use of correlation is misleading. An alternative approach, based on graphical techniques and simple calculations, is described, together with the relation between this analysis and the assessment of repeatability.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Pharmaceutical industry sponsorship and research outcome and quality: systematic review.

            To investigate whether funding of drug studies by the pharmaceutical industry is associated with outcomes that are favourable to the funder and whether the methods of trials funded by pharmaceutical companies differ from the methods in trials with other sources of support. Medline (January 1966 to December 2002) and Embase (January 1980 to December 2002) searches were supplemented with material identified in the references and in the authors' personal files. Data were independently abstracted by three of the authors and disagreements were resolved by consensus. 30 studies were included. Research funded by drug companies was less likely to be published than research funded by other sources. Studies sponsored by pharmaceutical companies were more likely to have outcomes favouring the sponsor than were studies with other sponsors (odds ratio 4.05; 95% confidence interval 2.98 to 5.51; 18 comparisons). None of the 13 studies that analysed methods reported that studies funded by industry was of poorer quality. Systematic bias favours products which are made by the company funding the research. Explanations include the selection of an inappropriate comparator to the product being investigated and publication bias.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Meta-analyses of randomized controlled trials.

              A new type of research, termed meta-analysis, attempts to analyze and combine the results of previous reports. We found 86 meta-analyses of reports of randomized controlled trials in the English-language literature. We evaluated the quality of these meta-analyses, using a scoring method that considered 23 items in six major areas--study design, combinability, control of bias, statistical analysis, sensitivity analysis, and application of results. Only 24 meta-analyses (28 percent) addressed all six areas, 31 (36 percent) addressed five, 25 (29 percent) addressed four, 5 (6 percent) addressed three, and 1 (1 percent) addressed two. Of the 23 individual items, between 1 and 14 were addressed satisfactorily (mean +/- SD, 7.7 +/- 2.7). We conclude that an urgent need exists for improved methods in literature searching, quality evaluation of trials, and synthesizing of the results.
                Bookmark

                Author and article information

                Contributors
                Role: Academic Editor
                Journal
                PLoS ONE
                plos
                plosone
                PLoS ONE
                Public Library of Science (San Francisco, USA )
                1932-6203
                2007
                26 December 2007
                : 2
                : 12
                : e1350
                Affiliations
                [1 ]Community Information and Epidemiological Technologies (CIET), Ottawa, Ontario, Canada
                [2 ]Institute for Research in Extramural Medicine (EMGO Institute), Vrije Universiteit (VU) University Medical Center, Amsterdam, The Netherlands
                [3 ]Executive Board, Vrije Universiteit (VU) University Amsterdam, Amsterdam, The Netherlands
                [4 ]Clinical Epidemiology Program, Ottawa Health Research Institute, University of Ottawa, Ontario, Canada
                [5 ]Department of Clinical Epidemiology and Biostatistics, Vrije Universiteit (VU) University Medical Center, Amsterdam, The Netherlands
                [6 ]Centro de Investigación de Enfermedades Tropicales (CIET), Universidad Autónoma de Guerrero, Acapulco, Mexico
                [7 ]Epidemiological Research Institute, National Academy of Medicine, Buenos Aires, Argentina
                [8 ]Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa, Ontario, Canada
                University of Toronto, Canada
                Author notes
                * To whom correspondence should be addressed. E-mail: bshea@ 123456ciet.org

                Conceived and designed the experiments: JG MB BS NA LB. Performed the experiments: ZO JP VS AB. Analyzed the data: BS TR. Wrote the paper: ZO JG MB BS LB. Other: Designed the study: JS. Wrote the first draft of the paper: BS.

                Article
                07-PONE-RA-01096R1
                10.1371/journal.pone.0001350
                2131785
                18159233
                867e942e-a7b4-46b1-8921-d99c42c1b78b
                Shea et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
                History
                : 17 April 2007
                : 22 October 2007
                Page count
                Pages: 5
                Categories
                Research Article
                Evidence-Based Healthcare

                Uncategorized
                Uncategorized

                Comments

                Comment on this article