652
views
0
recommends
+1 Recommend
0 collections
    19
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Our objective was to develop an instrument to assess the methodological quality of systematic reviews, building upon previous tools, empirical evidence and expert consensus.

          Methods

          A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items recently judged to be of methodological importance. This tool was applied to 99 paper-based and 52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying components. The results were considered by methodological experts using a nominal group technique aimed at item reduction and design of an assessment tool with face and content validity.

          Results

          The factor analysis identified 11 components. From each component, one item was selected by the nominal group. The resulting instrument was judged to have face and content validity.

          Conclusion

          A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR) was developed. The tool consists of 11 items and has good face and content validity for measuring the methodological quality of systematic reviews. Additional studies are needed with a focus on the reproducibility and construct validity of AMSTAR, before strong recommendations can be made on its use.

          Related collections

          Most cited references30

          • Record: found
          • Abstract: found
          • Article: not found

          Language bias in randomised controlled trials published in English and German.

          Some randomised controlled trials (RCTs) done in German-speaking Europe are published in international English-language journals and others in national German-language journals. We assessed whether authors are more likely to report trials with statistically significant results in English than in German. We studied pairs of RCT reports, matched for first author and time of publication, with one report published in German and the other in English. Pairs were identified from reports round in a manual search of five leading German-language journals and from reports published by the same authors in English found on Medline. Quality of methods and reporting were assessed with two different scales by two investigators who were unaware of authors' identities, affiliations, and other characteristics of trial reports. Main study endpoints were selected by two investigators who were unaware of trial results. Our main outcome was the number of pairs of studies in which the levels of significance (shown by p values) were discordant. 62 eligible pairs of reports were identified but 19 (31%) were excluded because they were duplicate publications. A further three pairs (5%) were excluded because no p values were given. The remaining 40 pairs were analysed. Design characteristics and quality features were similar for reports in both languages. Only 35% of German-language articles, compared with 62% of English-language articles, reported significant (p < 0.05) differences in the main endpoint between study and control groups (p = 0.002 by McNemar's test). Logistic regression showed that the only characteristic that predicted publication in an English-language journal was a significant result. The odds ratio for publication of trials with significant results in English was 3.75 (95% CI 1.25-11.3). Authors were more likely to publish RCTs in an English-language journal if the results were statistically significant. English language bias may, therefore, be introduced in reviews and meta-analyses if they include only trials reported in English. The effort of the Cochrane Collaboration to identify as many controlled trials as possible, through the manual search of many medical journals published in different languages will help to reduce such bias.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Meta-analyses of randomized controlled trials.

            A new type of research, termed meta-analysis, attempts to analyze and combine the results of previous reports. We found 86 meta-analyses of reports of randomized controlled trials in the English-language literature. We evaluated the quality of these meta-analyses, using a scoring method that considered 23 items in six major areas--study design, combinability, control of bias, statistical analysis, sensitivity analysis, and application of results. Only 24 meta-analyses (28 percent) addressed all six areas, 31 (36 percent) addressed five, 25 (29 percent) addressed four, 5 (6 percent) addressed three, and 1 (1 percent) addressed two. Of the 23 individual items, between 1 and 14 were addressed satisfactorily (mean +/- SD, 7.7 +/- 2.7). We conclude that an urgent need exists for improved methods in literature searching, quality evaluation of trials, and synthesizing of the results.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              The existence of publication bias and risk factors for its occurrence.

              Publication bias is the tendency on the parts of investigators, reviewers, and editors to submit or accept manuscripts for publication based on the direction or strength of the study findings. Much of what has been learned about publication bias comes from the social sciences, less from the field of medicine. In medicine, three studies have provided direct evidence for this bias. Prevention of publication bias is important both from the scientific perspective (complete dissemination of knowledge) and from the perspective of those who combine results from a number of similar studies (meta-analysis). If treatment decisions are based on the published literature, then the literature must include all available data that is of acceptable quality. Currently, obtaining information regarding all studies undertaken in a given field is difficult, even impossible. Registration of clinical trials, and perhaps other types of studies, is the direction in which the scientific community should move.
                Bookmark

                Author and article information

                Journal
                BMC Med Res Methodol
                BMC Medical Research Methodology
                BioMed Central (London )
                1471-2288
                2007
                15 February 2007
                : 7
                : 10
                Affiliations
                [1 ]EMGO Institute, VU University Medical Center, Amsterdam, the Netherlands
                [2 ]Institute of Population Health, Ottawa, Ontario, Canada
                [3 ]Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada
                [4 ]Department of Clinical Epidemiology and Biostatistics. VU University Medical Center, Amsterdam, the Netherlands
                [5 ]Community Information and Epidemiological Technologies (CIETcanada), Ottawa, Ontario, Canada
                [6 ]University of Ottawa, Ottawa, Ontario, Canada
                Article
                1471-2288-7-10
                10.1186/1471-2288-7-10
                1810543
                17302989
                1a808926-d8ed-4ae1-9669-0a264a87aa20
                Copyright © 2007 Shea et al; licensee BioMed Central Ltd.

                This is an Open Access article distributed under the terms of the Creative Commons Attribution License ( http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

                History
                : 10 August 2006
                : 15 February 2007
                Categories
                Research Article

                Medicine
                Medicine

                Comments

                Comment on this article