270
views
0
recommends
+1 Recommend
1 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research

      research-article

      Read this article at

      ScienceOpenPublisherPMC
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          In this article, the third in the PROGRESS series on prognostic factor research, Sara Schroter and colleagues review how prognostic models are developed and validated, and then address how prognostic models are assessed for their impact on practice and patient outcomes, illustrating these ideas with examples.

          Related collections

          Most cited references76

          • Record: found
          • Abstract: found
          • Article: not found

          Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.

          Multivariable regression models are powerful tools that are used frequently in studies of clinical outcomes. These models can use a mixture of categorical and continuous variables and can handle partially observed (censored) responses. However, uncritical application of modelling techniques can result in models that poorly fit the dataset at hand, or, even more likely, inaccurately predict outcomes on new subjects. One must know how to measure qualities of a model's fit in order to avoid poorly fitted or overfitted models. Measurement of predictive accuracy can be difficult for survival time data in the presence of censoring. We discuss an easily interpretable index of predictive discrimination as well as methods for assessing calibration of predicted survival probabilities. Both types of predictive accuracy should be unbiasedly validated using bootstrapping or cross-validation, before using predictions in a new data series. We discuss some of the hazards of poorly fitted and overfitted regression models and present one modelling strategy that avoids many of the problems discussed. The methods described are applicable to all regression models, but are particularly needed for binary, ordinal, and time-to-event outcomes. Methods are illustrated with a survival analysis in prostate cancer using Cox regression.
            • Record: found
            • Abstract: found
            • Article: not found
            Is Open Access

            Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls

            Most studies have some missing data. Jonathan Sterne and colleagues describe the appropriate use and reporting of the multiple imputation approach to dealing with them
              • Record: found
              • Abstract: found
              • Article: not found

              Assessing the performance of prediction models: a framework for traditional and novel measures.

              The performance of prediction models can be assessed using a variety of methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic [ROC] curve), and goodness-of-fit statistics for calibration.Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision-analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions.We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration, we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n = 544 for model development, n = 273 for external validation).We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.

                Author and article information

                Journal
                PLoS Med
                PLoS Med
                PLoS
                plosmed
                PLoS Medicine
                Public Library of Science (San Francisco, USA )
                1549-1277
                1549-1676
                February 2013
                February 2013
                5 February 2013
                : 10
                : 2
                : e1001381
                Affiliations
                [1 ]Department of Public Health, Erasmus MC, Rotterdam, Netherlands
                [2 ]Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht, Netherlands
                [3 ]Arthritis Research UK Primary Care Centre, Keele University, Keele, United Kingdom
                [4 ]Department of Community Health and Epidemiology, Dalhousie University, Halifax, Nova Scotia, Canada
                [5 ]London School of Hygiene & Tropical Medicine, London, United Kingdom
                [6 ]BMJ, BMA House, Tavistock Square, London, United Kingdom
                [7 ]School of Health and Population Sciences, University of Birmingham, Birmingham, United Kingdom
                [8 ]Department of Epidemiology and Public Health, University College London, London, United Kingdom
                [9 ]Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom
                Author notes

                SS is a full time employee of the BMJ Group but is not involved in the decision making on manuscripts. The authors declare no other competing interests.

                Wrote the first draft of the manuscript: EWS KGMM. Contributed to the writing of the manuscript: EWS KGMM DAvdW JAH PP SS RDR HH DGA. Initiated the PROGRESS Group, organised the three workshops, coordinated the writing groups, and were the scientific writing editors for all the papers in the PROGRESS series: HH RDR SS DGA. Guarantors for this paper: HH RDR DGA. Contributed through workshops and discussions to the development of the article series: PROGRESS Group. ICMJE criteria for authorship read and met: EWS KGMM DAvdW JAH PP SS RDR HH DGA. Agree with manuscript results and conclusions: EWS KGMM DAvdW JAH PP SS RDR HH DGA.

                The Guidelines and Guidance section contains advice on conducting and reporting medical research.

                ¶ These authors contributed equally and are joint first authors on this work.

                Article
                PMEDICINE-D-12-02102
                10.1371/journal.pmed.1001381
                3564751
                23393430
                b0716021-1218-470f-9009-716dc8b9cead
                Copyright @ 2013

                This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

                History
                Page count
                Pages: 9
                Funding
                This series had no explicit funding, but some of the authors were supported by research grants: PROGRESS is supported by a Partnership grant from the Medical Research Council (G0902393), involving University College London (HH, AH), University of Oxford (DGA), Birmingham University (RDR), London School of Hygiene and Tropical Medicine (IR, PP), Keele University (PC, DAvdW), and Queen Mary University London (ADT). DGA is supported by a programme grant from Cancer Research UK (C5529). HH is supported by grants from the UK National Institute for Health Research (RP-PG-0407-10314) and the Wellcome Trust (086091/Z/08/Z). JAH is supported by a New Investigator Award from the Canadian Institutes of Health Research and a grant from the Nova Scotia Health Research Foundation, and holds a Dalhousie University/CCRF research professorship. KGMM is supported by The Netherlands Organization for Scientific Research (ZON-MW 918.10.615 and 9120.8004). EWS was supported by The Netherlands Organization for Scientific Research (grant 9120.8004) and the NIH (grant NS-042691). RDR is supported by the MRC Midlands Hub for Trials Methodology Research (Medical Research Council Grant ID G0800808). DAvdW is supported by the Arthritis Research UK Centre of Excellence in Primary Care. The work of HH, AH and ADT is supported by the Health eResearch Centre Network (HERC-UK), funded by The Medical Research Council, in partnership with Arthritis Research UK, the British Heart Foundation, Cancer Research UK, the Economic and Social Research Council, the Engineering and Physical Sciences Research Council, the National Institute of Health Research, the National Institute for Social Care and Health Research (Welsh Assembly Government), the Chief Scientist Office (Scottish Government Health Directorates) and the Wellcome Trust. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The views expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
                Categories
                Guidelines and Guidance
                Medicine

                Medicine
                Medicine

                Comments

                Comment on this article

                Related Documents Log