Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease <sup> </sup>

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

The medical field has seen a rapid increase in the development of artificial intelligence (AI)-based prediction models. With the introduction of such AI-based prediction model tools and software in cardiovascular patient care, the cardiovascular researcher and healthcare professional are challenged to understand the opportunities as well as the limitations of the AI-based predictions. In this article, we present 12 critical questions for cardiovascular health professionals to ask when confronted with an AI-based prediction model. We aim to support medical professionals to distinguish the AI-based prediction models that can add value to patient care from the AI that does not.

Graphical Abstract

Twelve critical questions to be asked by readers and reviewers when confronted with prediction models that are based on AI.

Related collections

Most cited references 82

Record: found
Abstract: found
Article: not found

General cardiovascular risk profile for use in primary care: the Framingham Heart Study.

Ralph B. D’Agostino, Ramachandran S. Vasan, Michael J. Pencina … (2008)

Separate multivariable risk algorithms are commonly used to assess risk of specific atherosclerotic cardiovascular disease (CVD) events, ie, coronary heart disease, cerebrovascular disease, peripheral vascular disease, and heart failure. The present report presents a single multivariable risk function that predicts risk of developing all CVD and of its constituents. We used Cox proportional-hazards regression to evaluate the risk of developing a first CVD event in 8491 Framingham study participants (mean age, 49 years; 4522 women) who attended a routine examination between 30 and 74 years of age and were free of CVD. Sex-specific multivariable risk functions ("general CVD" algorithms) were derived that incorporated age, total and high-density lipoprotein cholesterol, systolic blood pressure, treatment for hypertension, smoking, and diabetes status. We assessed the performance of the general CVD algorithms for predicting individual CVD events (coronary heart disease, stroke, peripheral artery disease, or heart failure). Over 12 years of follow-up, 1174 participants (456 women) developed a first CVD event. All traditional risk factors evaluated predicted CVD risk (multivariable-adjusted P<0.0001). The general CVD algorithm demonstrated good discrimination (C statistic, 0.763 [men] and 0.793 [women]) and calibration. Simple adjustments to the general CVD risk algorithms allowed estimation of the risks of each CVD component. Two simple risk scores are presented, 1 based on all traditional risk factors and the other based on non-laboratory-based predictors. A sex-specific multivariable risk factor algorithm can be conveniently used to assess general CVD risk and risk of individual CVD events (coronary, cerebrovascular, and peripheral arterial disease and heart failure). The estimated absolute CVD event rates can be used to quantify risk and to guide preventive care.

0 comments Cited 2195 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal

Laure Wynants, Ben Van Calster, Marc M. J. Bonten … (2020)

Abstract Objective To review and critically appraise published and preprint reports of prediction models for diagnosing coronavirus disease 2019 (covid-19) in patients with suspected infection, for prognosis of patients with covid-19, and for detecting people in the general population at risk of being admitted to hospital for covid-19 pneumonia. Design Rapid systematic review and critical appraisal. Data sources PubMed and Embase through Ovid, Arxiv, medRxiv, and bioRxiv up to 24 March 2020. Study selection Studies that developed or validated a multivariable covid-19 related prediction model. Data extraction At least two authors independently extracted data using the CHARMS (critical appraisal and data extraction for systematic reviews of prediction modelling studies) checklist; risk of bias was assessed using PROBAST (prediction model risk of bias assessment tool). Results 2696 titles were screened, and 27 studies describing 31 prediction models were included. Three models were identified for predicting hospital admission from pneumonia and other events (as proxy outcomes for covid-19 pneumonia) in the general population; 18 diagnostic models for detecting covid-19 infection (13 were machine learning based on computed tomography scans); and 10 prognostic models for predicting mortality risk, progression to severe disease, or length of hospital stay. Only one study used patient data from outside of China. The most reported predictors of presence of covid-19 in patients with suspected disease included age, body temperature, and signs and symptoms. The most reported predictors of severe prognosis in patients with covid-19 included age, sex, features derived from computed tomography scans, C reactive protein, lactic dehydrogenase, and lymphocyte count. C index estimates ranged from 0.73 to 0.81 in prediction models for the general population (reported for all three models), from 0.81 to more than 0.99 in diagnostic models (reported for 13 of the 18 models), and from 0.85 to 0.98 in prognostic models (reported for six of the 10 models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and high risk of model overfitting. Reporting quality varied substantially between studies. Most reports did not include a description of the study population or intended use of the models, and calibration of predictions was rarely assessed. Conclusion Prediction models for covid-19 are quickly entering the academic literature to support medical decision making at a time when they are urgently needed. This review indicates that proposed models are poorly reported, at high risk of bias, and their reported performance is probably optimistic. Immediate sharing of well documented individual participant data from covid-19 studies is needed for collaborative efforts to develop more rigorous prediction models and validate existing ones. The predictors identified in included studies could be considered as candidate predictors for new models. Methodological guidance should be followed because unreliable predictions could cause more harm than benefit in guiding clinical decisions. Finally, studies should adhere to the TRIPOD (transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) reporting guideline. Systematic review registration Protocol https://osf.io/ehc47/, registration https://osf.io/wy245.

0 comments Cited 1251 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration

Karel Moons, Douglas G. Altman, Johannes Reitsma … (2015)

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

0 comments Cited 1193 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Maarten van Smeden:

ORCID: https://orcid.org/0000-0002-5529-1541

Georg Heinze:

ORCID: https://orcid.org/0000-0003-1147-8491

Ben Van Calster:

ORCID: https://orcid.org/0000-0003-1613-7450

Folkert W Asselbergs

Panos E Vardas:

ORCID: https://orcid.org/0000-0002-4499-0806

Nico Bruining:

ORCID: https://orcid.org/0000-0002-0272-5617

Peter de Jaegere

Jason H Moore

Spiros Denaxas:

ORCID: https://orcid.org/0000-0001-9612-7791

Anne Laure Boulesteix

Karel G M Moons

Journal

Journal ID (nlm-ta): Eur Heart J

Journal ID (iso-abbrev): Eur Heart J

Journal ID (publisher-id): eurheartj

Title: European Heart Journal

Publisher: Oxford University Press

ISSN (Print): 0195-668X

ISSN (Electronic): 1522-9645

Publication date Collection: 14 August 2022

Publication date (Electronic): 26 May 2022

Publication date PMC-release: 26 May 2022

Volume: 43

Issue: 31 , Focus Issue on Heart Failure and Cardiomyopathies

Pages: 2921-2930

Affiliations

Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University , Universiteitsweg 100, 3584 CG Utrecht, The Netherlands

Section for Clinical Biometrics, Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna , Vienna, Austria

Department of Development and Regeneration , KU Leuven, Leuven, Belgium

EPI Centre, KU Leuven , Leuven, Belgium

Department of Biomedical Data Sciences, Leiden University Medical Centre , Leiden, The Netherlands

Department of Cardiology, Division Heart and Lungs, University Medical Center Utrecht, Utrecht University , Utrecht, The Netherlands

Institute of Cardiovascular Science, Faculty of Population Health Sciences, University College London , London, UK

Health Data Research UK and Institute of Health Informatics, University College London , London, UK

Department of Cardiology, Heraklion University Hospital , Heraklion, Greece

Heart Sector, Hygeia Hospitals Group , Athens, Greece

Department of Cardiology , Erasmus MC , Thorax Center, Rotterdam, The Netherlands

Department of Cardiology , Erasmus MC, Thorax Center, Rotterdam, The Netherlands

Department of Computational Biomedicine, Cedars-Sinai Medical Center , Los Angeles, CA, USA

Health Data Research UK and Institute of Health Informatics, University College London , London, UK

The Alan Turing Institute , London, UK

Institute for Medical Information Processing, Biometry and Epidemiology , LMU Munich, Germany

Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University , Universiteitsweg 100, 3584 CG Utrecht, The Netherlands

Author notes

Corresponding author. Tel: +31 648 931 109, Email: M.vanSmeden@ 123456umcutrecht.nl

Conflict of interest: The authors declare no conflict of interest.

Author information

Maarten van Smeden https://orcid.org/0000-0002-5529-1541

Georg Heinze https://orcid.org/0000-0003-1147-8491

Ben Van Calster https://orcid.org/0000-0003-1613-7450

Panos E Vardas https://orcid.org/0000-0002-4499-0806

Nico Bruining https://orcid.org/0000-0002-0272-5617

Spiros Denaxas https://orcid.org/0000-0001-9612-7791

Article

Publisher ID: ehac238

DOI: 10.1093/eurheartj/ehac238

PMC ID: 9443991

PubMed ID: 35639667

SO-VID: 6e36bbf6-ebc5-4a7c-8f53-e103013db22b

License:

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License ( https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com

History

Date received : 09 November 2021

Date revision received : 29 March 2022

Date accepted : 26 April 2022

Page count

Pages: 10

Funding

Funded by: UCL Hospitals, NIHR Biomedical Research Centre;

Funded by: Innovative Medicines Initiative-2 joint undertaking under grant agreement;

Award ID: 116074

Funded by: National Institutes of Health, DOI 10.13039/100000002;

Award ID: LM010098

Funded by: German Research Foundation, DOI 10.13039/501100001659;

Award ID: BO3139/4-3

Funded by: Federal Ministry of Education and Research, DOI 10.13039/501100002347;

Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease

Read this article at

Abstract

Graphical Abstract

Related collections

UCL: UN SDG 03 Good Health and Well-Being

Most cited references 82

General cardiovascular risk profile for use in primary care: the Framingham Heart Study.

Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal

Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration

Author and article information

Contributors

Journal

Affiliations

Author notes

Author information

Article

History

Page count

Funding

Categories

Comments

Comment on this article

Similar content 237

Cited by 19

Most referenced authors 1,519