73
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Evaluating the Quality of Evidence from a Network Meta-Analysis

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Systematic reviews that collate data about the relative effects of multiple interventions via network meta-analysis are highly informative for decision-making purposes. A network meta-analysis provides two types of findings for a specific outcome: the relative treatment effect for all pairwise comparisons, and a ranking of the treatments. It is important to consider the confidence with which these two types of results can enable clinicians, policy makers and patients to make informed decisions. We propose an approach to determining confidence in the output of a network meta-analysis. Our proposed approach is based on methodology developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group for pairwise meta-analyses. The suggested framework for evaluating a network meta-analysis acknowledges (i) the key role of indirect comparisons (ii) the contributions of each piece of direct evidence to the network meta-analysis estimates of effect size; (iii) the importance of the transitivity assumption to the validity of network meta-analysis; and (iv) the possibility of disagreement between direct evidence and indirect evidence. We apply our proposed strategy to a systematic review comparing topical antibiotics without steroids for chronically discharging ears with underlying eardrum perforations. The proposed framework can be used to determine confidence in the results from a network meta-analysis. Judgements about evidence from a network meta-analysis can be different from those made about evidence from pairwise meta-analyses.

          Related collections

          Most cited references6

          • Record: found
          • Abstract: found
          • Article: found
          Is Open Access

          Evidence Synthesis for Decision Making 4

          Inconsistency can be thought of as a conflict between “direct” evidence on a comparison between treatments B and C and “indirect” evidence gained from AC and AB trials. Like heterogeneity, inconsistency is caused by effect modifiers and specifically by an imbalance in the distribution of effect modifiers in the direct and indirect evidence. Defining inconsistency as a property of loops of evidence, the relation between inconsistency and heterogeneity and the difficulties created by multiarm trials are described. We set out an approach to assessing consistency in 3-treatment triangular networks and in larger circuit structures, its extension to certain special structures in which independent tests for inconsistencies can be created, and describe methods suitable for more complex networks. Sample WinBUGS code is given in an appendix. Steps that can be taken to minimize the risk of drawing incorrect conclusions from indirect comparisons and network meta-analysis are the same steps that will minimize heterogeneity in pairwise meta-analysis. Empirical indicators that can provide reassurance and the question of how to respond to inconsistency are also discussed.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers

            Background In the last decade, network meta-analysis of randomized controlled trials has been introduced as an extension of pairwise meta-analysis. The advantage of network meta-analysis over standard pairwise meta-analysis is that it facilitates indirect comparisons of multiple interventions that have not been studied in a head-to-head fashion. Although assumptions underlying pairwise meta-analyses are well understood, those concerning network meta-analyses are perceived to be more complex and prone to misinterpretation. Discussion In this paper, we aim to provide a basic explanation when network meta-analysis is as valid as pairwise meta-analysis. We focus on the primary role of effect modifiers, which are study and patient characteristics associated with treatment effects. Because network meta-analysis includes different trials comparing different interventions, the distribution of effect modifiers cannot only vary across studies for a particular comparison (as with standard pairwise meta-analysis, causing heterogeneity), but also between comparisons (causing inconsistency). If there is an imbalance in the distribution of effect modifiers between different types of direct comparisons, the related indirect comparisons will be biased. If it can be assumed that this is not the case, network meta-analysis is as valid as pairwise meta-analysis. Summary The validity of network meta-analysis is based on the underlying assumption that there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons, regardless of the structure of the evidence network.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Adjusted indirect comparison may be less biased than direct comparison for evaluating new pharmaceutical interventions.

              To investigate discrepancies between direct comparison and adjusted indirect comparison in meta-analyses of new versus conventional pharmaceutical interventions. Results of direct comparison were compared with results of adjusted indirect comparison in three meta-analyses of new versus conventional drugs. The three case studies are (1) bupropion versus nicotine replacement therapy for smoking cessation, (2) risperidone versus haloperidol for schizophrenia, and (3) fluoxetine versus imipramine for depressive disorders. In all the three cases, effects of new drugs estimated by head-to-head trials tend to be greater than that by adjusted indirect comparisons. The observed discrepancies could not be satisfactorily explained by the play of chance or by bias and heterogeneity in adjusted indirect comparison. This observation, along with analysis of possible systematic bias in the direct comparisons, suggested that the indirect method might have produced less biased results. Simulations found that adjusted indirect comparison may counterbalance bias under certain circumstances. Adjusted indirect comparison could be used to cross-examine the validity and applicability of results from head-to-head randomized trials. The hypothesis that adjusted indirect comparison may provide less biased results than head-to-head randomized trials needs to be investigated by further research.
                Bookmark

                Author and article information

                Contributors
                Role: Editor
                Journal
                PLoS One
                PLoS ONE
                plos
                plosone
                PLoS ONE
                Public Library of Science (San Francisco, USA )
                1932-6203
                2014
                3 July 2014
                : 9
                : 7
                : e99682
                Affiliations
                [1 ]Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece
                [2 ]Statistics Unit, Department of Clinical and Diagnostic Medicine and Public Health, University of Modena and Reggio Emilia, Modena, Italy
                [3 ]School of Social and Community Medicine, University of Bristol, Bristol, United Kingdom
                [4 ]Centre for Reviews and Dissemination, University of York, York, United Kingdom
                National Taiwan University, Taiwan
                Author notes

                Competing Interests: The authors have declared that no competing interests exist.

                Conceived and designed the experiments: GS CDG AC DMC JPTH. Performed the experiments: GS CDG AC DMC JPTH. Analyzed the data: CDG AC. Contributed reagents/materials/analysis tools: GS CDG AC. Wrote the paper: GS CDG AC DMC JPTH.

                Article
                PONE-D-14-01780
                10.1371/journal.pone.0099682
                4084629
                24992266
                33b3ed59-b406-4687-88b8-7a634eb87d53
                Copyright @ 2014

                This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                : 13 January 2014
                : 18 May 2014
                Page count
                Pages: 14
                Funding
                GS and AC received funding from the European Research Council (IMMA, grant no. 260559). DC was supported by a Medical Research Council Population Health Scientist fellowship award (grant no. G0902118). This research was also supported in part by The Cochrane Collaboration’s Methods Innovation Funding programme. The views expressed in this article are those of the authors and not necessarily those of The Cochrane Collaboration or its registered entities, committees, or working groups. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
                Categories
                Research Article
                Medicine and Health Sciences
                Epidemiology
                Epidemiological Methods and Statistics
                Research and Analysis Methods
                Research Assessment
                Research Validity
                Science Policy

                Uncategorized
                Uncategorized

                Comments

                Comment on this article