Blog
About

16
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Feasibility and evaluation of a large-scale external validation approach for patient-level prediction in an international data network: validation of models predicting stroke in female patients newly diagnosed with atrial fibrillation

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          To demonstrate how the Observational Healthcare Data Science and Informatics (OHDSI) collaborative network and standardization can be utilized to scale-up external validation of patient-level prediction models by enabling validation across a large number of heterogeneous observational healthcare datasets.

          Methods

          Five previously published prognostic models (ATRIA, CHADS 2, CHADS 2VASC, Q-Stroke and Framingham) that predict future risk of stroke in patients with atrial fibrillation were replicated using the OHDSI frameworks. A network study was run that enabled the five models to be externally validated across nine observational healthcare datasets spanning three countries and five independent sites.

          Results

          The five existing models were able to be integrated into the OHDSI framework for patient-level prediction and they obtained mean c-statistics ranging between 0.57–0.63 across the 6 databases with sufficient data to predict stroke within 1 year of initial atrial fibrillation diagnosis for females with atrial fibrillation. This was comparable with existing validation studies. The validation network study was run across nine datasets within 60 days once the models were replicated. An R package for the study was published at https://github.com/OHDSI/StudyProtocolSandbox/tree/master/ExistingStrokeRiskExternalValidation.

          Conclusion

          This study demonstrates the ability to scale up external validation of patient-level prediction models using a collaboration of researchers and a data standardization that enable models to be readily shared across data sites. External validation is necessary to understand the transportability or reproducibility of a prediction model, but without collaborative approaches it can take three or more years for a model to be validated by one independent researcher. In this paper we show it is possible to both scale-up and speed-up external validation by showing how validation can be done across multiple databases in less than 2 months. We recommend that researchers developing new prediction models use the OHDSI network to externally validate their models.

          Related collections

          Most cited references 8

          • Record: found
          • Abstract: found
          • Article: not found

          A risk score for predicting stroke or death in individuals with new-onset atrial fibrillation in the community: the Framingham Heart Study.

          Prior risk stratification schemes for atrial fibrillation (AF) have been based on randomized trial cohorts or Medicare administrative databases, have included patients with established AF, and have focused on stroke as the principal outcome. To derive risk scores for stroke alone and stroke or death in community-based individuals with new-onset AF. Prospective, community-based, observational cohort in Framingham, Mass. We identified 868 participants with new-onset AF, 705 of whom were not treated with warfarin at baseline. Risk scores for stroke (ischemic or hemorrhagic) and stroke or death were developed with censoring when warfarin initiation occurred during follow-up. Event rates were examined in low-risk individuals, as defined by the risk score and 4 previously published risk schemes. Stroke and the combination of stroke or death. During a mean follow-up of 4.0 years free of warfarin use, stroke alone occurred in 83 participants and stroke or death occurred in 382 participants. A risk score for stroke was derived that included the following risk predictors: advancing age, female sex, increasing systolic blood pressure, prior stroke or transient ischemic attack, and diabetes. With the risk score, 14.3% of the cohort had a predicted 5-year stroke rate < or =7.5% (average annual rate < or =1.5%), and 30.6% of the cohort had a predicted 5-year stroke rate < or =10% (average annual rate < or =2%). Actual stroke rates in these low-risk groups were 1.1 and 1.5 per 100 person-years, respectively. Previous risk schemes classified 6.4% to 17.3% of subjects as low risk, with actual stroke rates of 0.9 to 2.3 per 100 person-years. A risk score for stroke or death is also presented. These risk scores can be used to estimate the absolute risk of an adverse event in individuals with AF, which may be helpful in counseling patients and making treatment decisions.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: found
            Is Open Access

            Primary prevention of cardiovascular disease: A review of contemporary guidance and literature

            Cardiovascular disease is a significant and ever-growing problem in the United Kingdom, accounting for nearly one-third of all deaths and leading to significant morbidity. It is also of particular and pressing interest as developing countries experience a change in lifestyle which introduces novel risk factors for cardiovascular disease, leading to a boom in cardiovascular disease risk throughout the developing world. The burden of cardiovascular disease can be ameliorated by careful risk reduction and, as such, primary prevention is an important priority for all developers of health policy. Strong consensus exists between international guidelines regarding the necessity of smoking cessation, weight optimisation and the importance of exercise, whilst guidelines vary slightly in their approach to hypertension and considerably regarding their approach to optimal lipid profile which remains a contentious issue. Previously fashionable ideas such as the polypill appear devoid of in-vivo efficacy, but there remain areas of future interest such as the benefit of serum urate reduction and utility of reduction of homocysteine levels.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination.

              To evaluate how often newly developed risk prediction models undergo external validation and how well they perform in such validations.
                Bookmark

                Author and article information

                Contributors
                jreps@its.jnj.com
                Journal
                BMC Med Res Methodol
                BMC Med Res Methodol
                BMC Medical Research Methodology
                BioMed Central (London )
                1471-2288
                6 May 2020
                6 May 2020
                2020
                : 20
                Affiliations
                [1 ]GRID grid.497530.c, ISNI 0000 0004 0389 4927, Janssen Research and Development, ; 1125 Trenton Harbourton Rd, Titusville, NJ 08560 USA
                [2 ]GRID grid.5645.2, ISNI 000000040459992X, Department of Medical Informatics, , Erasmus University Medical Center, ; Rotterdam, The Netherlands
                [3 ]GRID grid.251916.8, ISNI 0000 0004 0532 3933, Department of Biomedical Informatics, , Ajou University School of Medicine, ; Suwon, Republic of Korea
                [4 ]GRID grid.239585.0, ISNI 0000 0001 2285 2675, Department of Biomedical Informatics, , Columbia University Medical Center, ; New York, USA
                [5 ]GRID grid.22072.35, ISNI 0000 0004 1936 7697, O’Brien Institute for Public Health, Faculty of Medicine, , University of Calgary, ; Calgary, Alberta Canada
                [6 ]GRID grid.168010.e, ISNI 0000000419368956, Center for Biomedical Informatics Research, School of Medicine, , Stanford University, ; Stanford, CA USA
                [7 ]GRID grid.251916.8, ISNI 0000 0004 0532 3933, Department of Biomedical Sciences, , Ajou University Graduate School of Medicine, ; Suwon, Republic of Korea
                [8 ]GRID grid.411261.1, ISNI 0000 0004 0648 1036, Department of Cardiology, , Ajou University Medical Centre, ; Suwon, Republic of Korea
                Article
                991
                10.1186/s12874-020-00991-3
                7201646
                © The Author(s) 2020

                Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

                Funding
                Funded by: Health Promotion Administration, Ministry of Health and Welfare (TW)
                Award ID: HI16C0992
                Award Recipient :
                Funded by: Innovative Medicines Initiative Joint Undertaking
                Award ID: 806968
                Award Recipient :
                Categories
                Research Article
                Custom metadata
                © The Author(s) 2020

                Comments

                Comment on this article