3
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Prospective Validation and Comparative Analysis of Coronary Risk Stratification Strategies Among Emergency Department Patients With Chest Pain

      research-article
      , MD 1 , 2 , 3 , , , PhD 3 , , MD, MPH 4 , , MD, MPH 1 , 3 , , MD 5 , , MD 6 , , MD 7 , , MD, MPH 8 , , MD 9 , , MD 10 , , BA 11 , , BS 3 , , MPH 3 , , MD 3 , 9 , , MD, MBE 3 , 12 , , DrPH 3 , for the Kaiser Permanente CREST Network Investigators
      Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease
      John Wiley and Sons Inc.
      acute coronary syndrome, emergency department, risk score, Diagnostic Testing, Acute Coronary Syndromes, Prognosis

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Background

          Coronary risk stratification is recommended for emergency department patients with chest pain. Many protocols are designed as “rule‐out” binary classification strategies, while others use graded‐risk stratification. The comparative performance of competing approaches at varying levels of risk tolerance has not been widely reported.

          Methods and Results

          This is a prospective cohort study of adult patients with chest pain presenting between January 2018 and December 2019 to 13 medical center emergency departments within an integrated healthcare delivery system. Using an electronic clinical decision support interface, we externally validated and assessed the net benefit (at varying risk thresholds) of several coronary risk scores (History, ECG, Age, Risk Factors, and Troponin [HEART] score, HEART pathway, Emergency Department Assessment of Chest Pain Score Accelerated Diagnostic Protocol), troponin‐only strategies (fourth‐generation assay), unstructured physician gestalt, and a novel risk algorithm (RISTRA‐ACS). The primary outcome was 60‐day major adverse cardiac event defined as myocardial infarction, cardiac arrest, cardiogenic shock, coronary revascularization, or all‐cause mortality. There were 13 192 patient encounters included with a 60‐day major adverse cardiac event incidence of 3.7%. RISTRA‐ACS and HEART pathway had the lowest negative likelihood ratios (0.06, 95% CI, 0.03–0.10 and 0.07, 95% CI, 0.04–0.11, respectively) and the greatest net benefit across a range of low‐risk thresholds. RISTRA‐ACS demonstrated the highest discrimination for 60‐day major adverse cardiac event (area under the receiver operating characteristic curve 0.92, 95% CI, 0.91–0.94, P<0.0001).

          Conclusions

          RISTRA‐ACS and HEART pathway were the optimal rule‐out approaches, while RISTRA‐ACS was the best‐performing graded‐risk approach. RISTRA‐ACS offers promise as a versatile single approach to emergency department coronary risk stratification.

          Registration

          URL: https://www.clinicaltrials.gov; Unique identifier: NCT03286179.

          Related collections

          Most cited references70

          • Record: found
          • Abstract: found
          • Article: not found

          Assessing the performance of prediction models: a framework for traditional and novel measures.

          The performance of prediction models can be assessed using a variety of methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic [ROC] curve), and goodness-of-fit statistics for calibration.Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision-analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions.We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration, we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n = 544 for model development, n = 273 for external validation).We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Fourth Universal Definition of Myocardial Infarction (2018).

              Bookmark
              • Record: found
              • Abstract: found
              • Article: found
              Is Open Access

              Calibration: the Achilles heel of predictive analytics

              Background The assessment of calibration performance of risk prediction models based on regression or more flexible machine learning algorithms receives little attention. Main text Herein, we argue that this needs to change immediately because poorly calibrated algorithms can be misleading and potentially harmful for clinical decision-making. We summarize how to avoid poor calibration at algorithm development and how to assess calibration at algorithm validation, emphasizing balance between model complexity and the available sample size. At external validation, calibration curves require sufficiently large samples. Algorithm updating should be considered for appropriate support of clinical practice. Conclusion Efforts are required to avoid poor calibration when developing prediction models, to evaluate calibration when validating models, and to update models when indicated. The ultimate aim is to optimize the utility of predictive analytics for shared decision-making and patient counseling.
                Bookmark

                Author and article information

                Contributors
                dustin.g.mark@kp.org
                Journal
                J Am Heart Assoc
                J Am Heart Assoc
                10.1002/(ISSN)2047-9980
                JAH3
                ahaoa
                Journal of the American Heart Association: Cardiovascular and Cerebrovascular Disease
                John Wiley and Sons Inc. (Hoboken )
                2047-9980
                31 March 2021
                06 April 2021
                : 10
                : 7 ( doiID: 10.1002/jah3.v10.7 )
                : e020082
                Affiliations
                [ 1 ] Department of Emergency Medicine Kaiser Permanente Oakland Medical Center Oakland CA
                [ 2 ] Department of Critical Care Medicine Kaiser Permanente Oakland Medical Center Oakland CA
                [ 3 ] Division of Research Kaiser Permanente Northern California Oakland CA
                [ 4 ] Department of Emergency Medicine Kaiser Permanente San Leandro Medical Center San Leandro CA
                [ 5 ] Department of Emergency Medicine Kaiser Permanente South Sacramento Medical Center Sacramento CA
                [ 6 ] Department of Emergency Medicine Kaiser Permanente Santa Clara Medical Center Santa Clara CA
                [ 7 ] Department of Emergency Medicine Kaiser Permanente Walnut Creek Medical Center Walnut Creek CA
                [ 8 ] Department of Emergency Medicine Kaiser Permanente South San Francisco Medical Center South San Francisco CA
                [ 9 ] Department of Emergency Medicine Kaiser Permanente Roseville Medical Center Roseville CA
                [ 10 ] Department of Emergency Medicine Kaiser Permanente San Francisco Medical Center San Francisco CA
                [ 11 ] University of California San Diego School of Medicine San Diego CA
                [ 12 ] Department of Emergency Medicine Kaiser Permanente San Rafael Medical Center San Rafael CA
                Author notes
                [*] [* ] Correspondence to: Dustin G. Mark, MD, Department of Emergency Medicine, Kaiser Permanente Medical Center, 3600 Broadway, Oakland, CA 94611. E‐mail: dustin.g.mark@ 123456kp.org

                Author information
                https://orcid.org/0000-0002-5001-7228
                https://orcid.org/0000-0001-7493-8438
                https://orcid.org/0000-0002-9061-7746
                https://orcid.org/0000-0002-4146-4202
                https://orcid.org/0000-0002-6329-6836
                https://orcid.org/0000-0001-6559-1858
                https://orcid.org/0000-0002-5801-8794
                https://orcid.org/0000-0002-8203-1016
                Article
                JAH36124
                10.1161/JAHA.120.020082
                8174350
                33787290
                2f8e5d89-1d64-4e27-ad80-6810df30a801
                © 2021 The Authors. Published on behalf of the American Heart Association, Inc., by Wiley.

                This is an open access article under the terms of the http://creativecommons.org/licenses/by-nc/4.0/ License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

                History
                : 06 November 2020
                : 22 February 2021
                Page count
                Figures: 8, Tables: 5, Pages: 18, Words: 11177
                Funding
                Funded by: The Permanente Medical Group (TPMG) Delivery Science Research Program
                Categories
                Original Research
                Original Research
                Coronary Heart Disease
                Custom metadata
                2.0
                April 6, 2021
                Converter:WILEY_ML3GV2_TO_JATSPMC version:6.0.1 mode:remove_FC converted:06.04.2021

                Cardiovascular Medicine
                acute coronary syndrome,emergency department,risk score,diagnostic testing,acute coronary syndromes,prognosis

                Comments

                Comment on this article