10
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Objective

          To examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials of cancer drugs approved by the European Medicines Agency (EMA).

          Design

          Cross sectional analysis.

          Setting

          European regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices.

          Eligibility criteria

          Pivotal randomised controlled trials of new cancer drugs approved by the EMA between 2014 and 2016.

          Main outcome measures

          Study design characteristics (randomisation, comparators, and endpoints); risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result); and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents).

          Results

          Between 2014 and 2016, the EMA approved 32 new cancer drugs on the basis of 54 pivotal studies. Of these, 41 (76%) were randomised controlled trials and 13 (24%) were either non-randomised studies or single arm studies. 39/41 randomised controlled trials had available publications and were included in our study. Only 10 randomised controlled trials (26%) measured overall survival as either a primary or coprimary endpoint, with the remaining trials evaluating surrogate measures such as progression free survival and response rates. Overall, 19 randomised controlled trials (49%) were judged to be at high risk of bias for their primary outcome. Concerns about missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer randomised controlled trials that evaluated overall survival as the primary endpoint were at high risk of bias than those that evaluated surrogate efficacy endpoints (2/10 (20%) v 16/29 (55%), respectively). When information available in regulatory documents and the scientific literature was considered separately, overall risk of bias judgments differed for eight randomised controlled trials (21%), which reflects reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These deficits included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications.

          Conclusions

          Most pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were randomised controlled trials. However, almost half of these were judged to be at high risk of bias based on their design, conduct, or analysis, some of which might be unavoidable because of the complexity of cancer trials. Regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.

          Related collections

          Most cited references71

          • Record: found
          • Abstract: found
          • Article: not found

          Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.

          To determine if inadequate approaches to randomized controlled trial design and execution are associated with evidence of bias in estimating treatment effects. An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated treatment effects. Meta-analyses from the Cochrane Pregnancy and Childbirth Database. The associations between estimates of treatment effects and inadequate allocation concealment, exclusions after randomization, and lack of double-blinding. Compared with trials in which authors reported adequately concealed treatment allocation, trials in which concealment was either inadequate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P < .001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of quality). Trials in which participants had been excluded after randomization did not yield larger estimates of effects, but that lack of association may be due to incomplete reporting. Trials that were not double-blind also yielded larger estimates of effects (P = .01), with odds ratios being exaggerated by 17%. This study provides empirical evidence that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias. Readers of trial reports should be wary of these pitfalls, and investigators must improve their design, execution, and reporting of trials.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13

            Objective To determine the availability of data on overall survival and quality of life benefits of cancer drugs approved in Europe. Design Retrospective cohort study. Setting Publicly accessible regulatory and scientific reports on cancer approvals by the European Medicines Agency (EMA) from 2009 to 2013. Main outcome measures Pivotal and postmarketing trials of cancer drugs according to their design features (randomisation, crossover, blinding), comparators, and endpoints. Availability and magnitude of benefit on overall survival or quality of life determined at time of approval and after market entry. Validated European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) used to assess the clinical value of the reported gains in published studies of cancer drugs. Results From 2009 to 2013, the EMA approved the use of 48 cancer drugs for 68 indications. Of these, eight indications (12%) were approved on the basis of a single arm study. At the time of market approval, there was significant prolongation of survival in 24 of the 68 (35%). The magnitude of the benefit on overall survival ranged from 1.0 to 5.8 months (median 2.7 months). At the time of market approval, there was an improvement in quality of life in seven of 68 indications (10%). Out of 44 indications for which there was no evidence of a survival gain at the time of market authorisation, in the subsequent postmarketing period there was evidence for extension of life in three (7%) and reported benefit on quality of life in five (11%). Of the 68 cancer indications with EMA approval, and with a median of 5.4 years’ follow-up (minimum 3.3 years, maximum 8.1 years), only 35 (51%) had shown a significant improvement in survival or quality of life, while 33 (49%) remained uncertain. Of 23 indications associated with a survival benefit that could be scored with the ESMO-MCBS tool, the benefit was judged to be clinically meaningful in less than half (11/23, 48%). Conclusions This systematic evaluation of oncology approvals by the EMA in 2009-13 shows that most drugs entered the market without evidence of benefit on survival or quality of life. At a minimum of 3.3 years after market entry, there was still no conclusive evidence that these drugs either extended or improved life for most cancer indications. When there were survival gains over existing treatment options or placebo, they were often marginal.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              The Strength of Association Between Surrogate End Points and Survival in Oncology: A Systematic Review of Trial-Level Meta-analyses.

              The strength of association between surrogate end points and survival in oncology is important to understand because surrogate end points are frequently used in oncology clinical trials, supporting US Food and Drug Administration approvals and National Comprehensive Cancer Network guideline recommendations.
                Bookmark

                Author and article information

                Contributors
                Role: assistant professor of health policy and Harkness fellow
                Role: reader
                Role: senior research fellow in evidence synthesis
                Role: professor of evidence synthesis
                Role: professor of medical statistics and epidemiology
                Role: fellow and assistant professor of public health sciences
                Role: research assistant
                Role: research assistant
                Role: professor of oncology
                Journal
                BMJ
                BMJ
                BMJ-UK
                bmj
                The BMJ
                BMJ Publishing Group Ltd.
                0959-8138
                1756-1833
                2019
                18 September 2019
                : 366
                : l5221
                Affiliations
                [1 ]Department of Health Policy, London School of Economics and Political Science, London WC2A 2AE, UK
                [2 ]Program on Regulation, Therapeutics, and Law (PORTAL), Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, USA
                [3 ]Department of Global Health and Social Medicine, King’s College London, London, UK
                [4 ]Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
                [5 ]The National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care West (NIHR CLAHRC West) at University Hospitals Bristol NHS Foundation Trust, Bristol, UK
                [6 ]National Institute for Health Research Bristol Biomedical Research Centre, Bristol, UK
                [7 ]Cancer Research Institute, Queen’s University at Kingston, Kingston, Ontario, Canada
                Author notes
                Correspondence to: H Naci h.naci@ 123456lse.ac.uk (or @huseyinnaci2 on Twitter)
                Author information
                http://orcid.org/0000-0002-7192-5751
                http://orcid.org/0000-0002-8323-2514
                http://orcid.org/0000-0001-5798-4512
                Article
                nach049576
                10.1136/bmj.l5221
                6749182
                31533922
                a22e0b08-67ae-4380-928d-43d0a76f8401
                Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions

                This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

                History
                : 17 July 2019
                Categories
                Research

                Medicine
                Medicine

                Comments

                Comment on this article