Methodological Quality of Systematic Reviews in Subfertility: A Comparison of Two Different Approaches

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Systematic reviews are used widely to guide health care decisions. Several tools have been created to assess systematic review quality. The measurement tool for assessing the methodological quality of systematic reviews known as the AMSTAR tool applies a yes/no score to eleven relevant domains of review methodology. This tool has been reworked so that each domain is scored based on a four point scale, producing R-AMSTAR.

Methods and Findings

We aimed to compare the AMSTAR and R-AMSTAR tools in assessing systematic reviews in the field of assisted reproduction for subfertility. All published systematic reviews on assisted reproductive technology, with the latest search for studies taking place from 2007–2011, were considered. Reviews that contained no included studies or considered diagnostic outcomes were excluded. Thirty each of Cochrane and non-Cochrane reviews were randomly selected from a search of relevant databases. Both tools were then applied to all sixty reviews. The results were converted to percentage scores and all reviews graded and ranked based on this. AMSTAR produced a much wider variation in percentage scores and achieved higher inter-rater reliability than R-AMSTAR according to kappa statistics. The average rating for Cochrane reviews was consistent between the two tools (88.3% for R-AMSTAR versus 83.6% for AMSTAR) but inconsistent for non-Cochrane reviews (63.9% R-AMSTAR vs. 38.5% AMSTAR). In comparing the rankings generated between the two tools Cochrane reviews changed an average of 4.2 places, compared to 2.9 for non-Cochrane.

Conclusion

R-AMSTAR provided greater guidance in the assessment of domains and produced quantitative results. However, there were many problems with the construction of its criteria and AMSTAR was much easier to apply consistently. We recommend that AMSTAR incorporates the findings of this study and produces additional guidance for its application in order to improve its reliability and usefulness.

Related collections

Most cited references 1

Record: found
Abstract: found
Article: found

Is Open Access

From Systematic Reviews to Clinical Recommendations for Evidence-Based Health Care: Validation of Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) for Grading of Clinical Relevance

Jason Kung, Francesco Chiappelli, Olivia O Cajulis … (2010)

Research synthesis seeks to gather, examine and evaluate systematically research reports that converge toward answering a carefully crafted research question, which states the problem patient population, the intervention under consideration, and the clinical outcome of interest. The product of the process of systematically reviewing the research literature pertinent to the research question thusly stated is the “systematic review”. The objective and transparent approach of the systematic review aims to minimize bias. Most systematic reviews yield quantitative analyses of measurable data (e.g., acceptable sampling analysis, meta-analysis). Systematic reviews may also be qualitative, while adhering to accepted standards for gathering, evaluating, and reporting evidence. Systematic reviews provide highly rated recommendations for evidence-based health care; but, systematic reviews are not equally reliable and successful in minimizing bias. Several instruments are available to evaluate the quality of systematic reviews. The 'assessment of multiple systematic reviews' (AMSTAR) was derived from factor analysis of the most relevant items among them. AMSTAR consists of eleven items with good face and content validity for measuring the methodological quality of systematic reviews, has been widely accepted and utilized, and has gained in reliability, reproducibility. AMSTAR does not produce quantifiable assessments of systematic review quality and clinical relevance. In this study, we have revised the AMSTAR instrument, detracting nothing from its content and construct validity, and utilizing the very criteria employed in the development of the original tool, with the aim of yielding an instrument that can quantify the quality of systematic reviews. We present validation data of the revised AMSTAR (R-AMSTAR), and discuss its implications and application in evidence-based health care.

0 comments Cited 79 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Agustin Ciapponi: Role: Editor

Journal

Journal ID (nlm-ta): PLoS One

Journal ID (iso-abbrev): PLoS ONE

Journal ID (publisher-id): plos

Journal ID (pmc): plosone

Title: PLoS ONE

Publisher: Public Library of Science (San Francisco, USA )

ISSN (Electronic): 1932-6203

Publication date Collection: 2012

Publication date (Electronic): 28 December 2012

Volume: 7

Issue: 12

Electronic Location Identifier: e50403

Affiliations

[1 ]Faculty of Medicine and Health Sciences, University of Auckland, Auckland, New Zealand

[2 ]Department of Obstetrics and Gynaecology, University of Auckland, Auckland, New Zealand

[3 ]Cochrane Menstrual Disorders and Subfertility Group, Department of Obstetrics and Gynaecology, University of Auckland, Auckland, New Zealand

[4 ]Community Information Epidemiological Communities, University of Ottawa, Ottawa, Canada

Instituto de Efectividad Clínica y Sanitaria – IECS, Argentina

Author notes

* E-mail: c.farquhar@ 123456auckland.ac.nz

Competing Interests: The authors have read the journal's policy and have the following conflicts: Bev Shea was author of the original AMSTAR paper.

Conceived and designed the experiments: CF. Performed the experiments: IP BW. Analyzed the data: IP BW. Contributed reagents/materials/analysis tools: MS VJ BS. Wrote the paper: IP. Provided input on manuscript: BW MS VJ BS CF.

Article

Publisher ID: PONE-D-12-22035

DOI: 10.1371/journal.pone.0050403

PMC ID: 3532502

PubMed ID: 23300526

SO-VID: 2b764085-6755-4745-b631-c8fe76347e69

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

History

Date received : 24 July 2012

Date accepted : 19 October 2012

Page count

Pages: 9

Funding

Ivor Popovich and Bethany Windsor were funded to take part by the University of Auckland Summer Studentship Programme. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Methodological Quality of Systematic Reviews in Subfertility: A Comparison of Two Different Approaches

Read this article at

Abstract

Background

Methods and Findings

Conclusion

Related collections

PLOS Climate

Most cited references 1

From Systematic Reviews to Clinical Recommendations for Evidence-Based Health Care: Validation of Revised Assessment of Multiple Systematic Reviews (R-AMSTAR) for Grading of Clinical Relevance

Author and article information

Contributors

Journal

Affiliations

Author notes

Article

History

Page count

Funding

Categories

Comments

Comment on this article

Similar content 115

Cited by 17

Most referenced authors 424