Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Background

Acute respiratory failure occurs frequently in hospitalized patients and often starts before ICU admission. A risk stratification tool to predict mortality and risk for mechanical ventilation (MV) may allow for earlier evaluation and intervention. We developed and validated an automated electronic health record (EHR)-based model—Accurate Prediction of Prolonged Ventilation (APPROVE)—to identify patients at risk of death or respiratory failure requiring >= 48 h of MV.

Methods

This was an observational study of adults admitted to four hospitals in 2013 or a fifth hospital in 2017. Clinical data were extracted from the EHRs. The 2013 patients were randomly split 50:50 into a derivation/validation cohort. The qualifying event was death or intubation leading to MV >= 48 h. Random forest method was used in model derivation. APPROVE was calculated retrospectively whenever data were available in 2013, and prospectively every 4 h after hospital admission in 2017. The Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS) were calculated at the same times as APPROVE. Clinicians were not alerted except for APPROVE in 2017cohort.

Results

There were 68,775 admissions in 2013 and 2258 in 2017. APPROVE had an area under the receiver operator curve of 0.87 (95% CI 0.85–0.88) in 2013 and 0.90 (95% CI 0.84–0.95) in 2017, which is significantly better than the MEWS and NEWS in 2013 but similar to the MEWS and NEWS in 2017. At a threshold of > 0.25, APPROVE had similar sensitivity and positive predictive value (PPV) (sensitivity 63% and PPV 21% in 2013 vs 64% and 16%, respectively, in 2017). Compared to APPROVE in 2013, at a threshold to achieve comparable PPV (19% at MEWS > 4 and 22% at NEWS > 6), the MEWS and NEWS had lower sensitivity (16% for MEWS and NEWS). Similarly in 2017, at a comparable sensitivity threshold (64% for APPROVE > 0.25 and 67% for MEWS and NEWS > 4), more patients who triggered an alert developed the event with APPROVE (PPV 16%) while achieving a lower false positive rate (FPR 5%) compared to the MEWS (PPV 7%, FPR 14%) and NEWS (PPV 4%, FPR 25%).

Conclusions

An automated EHR model to identify patients at high risk of MV or death was validated retrospectively and prospectively, and was determined to be feasible for real-time risk identification.

Trial registration

ClinicalTrials.gov, NCT02488174. Registered on 18 March 2015.

Electronic supplementary material

The online version of this article (10.1186/s13054-018-2194-7) contains supplementary material, which is available to authorized users.

Related collections

Most cited references 24

Record: found
Abstract: found
Article: not found

A protocol of no sedation for critically ill patients receiving mechanical ventilation: a randomised trial.

Thomas Strøm, Torben Martinussen, Palle Toft (2010)

Standard treatment of critically ill patients undergoing mechanical ventilation is continuous sedation. Daily interruption of sedation has a beneficial effect, and in the general intesive care unit of Odense University Hospital, Denmark, standard practice is a protocol of no sedation. We aimed to establish whether duration of mechanical ventilation could be reduced with a protocol of no sedation versus daily interruption of sedation. Of 428 patients assessed for eligibility, we enrolled 140 critically ill adult patients who were undergoing mechanical ventilation and were expected to need ventilation for more than 24 h. Patients were randomly assigned in a 1:1 ratio (unblinded) to receive: no sedation (n=70 patients); or sedation (20 mg/mL propofol for 48 h, 1 mg/mL midazolam thereafter) with daily interruption until awake (n=70, control group). Both groups were treated with bolus doses of morphine (2.5 or 5 mg). The primary outcome was the number of days without mechanical ventilation in a 28-day period, and we also recorded the length of stay in the intensive care unit (from admission to 28 days) and in hospital (from admission to 90 days). Analysis was by intention to treat. This study is registered with ClinicalTrials.gov, number NCT00466492. 27 patients died or were successfully extubated within 48 h, and, as per our study design, were excluded from the study and statistical analysis. Patients receiving no sedation had significantly more days without ventilation (n=55; mean 13.8 days, SD 11.0) than did those receiving interrupted sedation (n=58; mean 9.6 days, SD 10.0; mean difference 4.2 days, 95% CI 0.3-8.1; p=0.0191). No sedation was also associated with a shorter stay in the intensive care unit (HR 1.86, 95% CI 1.05-3.23; p=0.0316), and, for the first 30 days studied, in hospital (3.57, 1.52-9.09; p=0.0039), than was interrupted sedation. No difference was recorded in the occurrences of accidental extubations, the need for CT or MRI brain scans, or ventilator-associated pneumonia. Agitated delirium was more frequent in the intervention group than in the control group (n=11, 20%vs n=4, 7%; p=0.0400). No sedation of critically ill patients receiving mechanical ventilation is associated with an increase in days without ventilation. A multicentre study is needed to establish whether this effect can be reproduced in other facilities. Danish Society of Anesthesiology and Intensive Care Medicine, the Fund of Danielsen, the Fund of Kirsten Jensa la Cour, and the Fund of Holger og Ruth Hess. Copyright 2010 Elsevier Ltd. All rights reserved.

0 comments Cited 203 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Clinical antecedents to in-hospital cardiopulmonary arrest.

Roland Schein, Nelson Hazday, María Fernanda Peña … (1990)

While the outcome of in-hospital cardiopulmonary arrest has been studied extensively, the clinical antecedents of arrest are less well defined. We studied a group of consecutive general hospital ward patients developing cardiopulmonary arrest. Prospectively determined definitions of underlying pathophysiology, severity of underlying disease, patient complaints, and clinical observations were used to determine common clinical features. Sixty-four patients arrested 161 +/- 26 hours following hospital admission. Pathophysiologic alterations preceding arrest were classified as respiratory in 24 patients (38 percent), metabolic in 7 (11 percent), cardiac in 6 (9 percent), neurologic in 4 (6 percent), multiple in 17 (27 percent), and unclassified in 6 (9 percent). Patients with multiple disturbances had mainly respiratory (39 percent) and metabolic (44 percent) disorders. Fifty-four patients (84 percent) had documented observations of clinical deterioration or new complaints within eight hours of arrest. Seventy percent of all patients had either deterioration of respiratory or mental function observed during this time. Routine laboratory tests obtained before arrest showed no consistent abnormalities, but vital signs showed a mean respiratory rate of 29 +/- 1 breaths per minute. The prognoses of patients' underlying diseases were classified as ultimately fatal in 26 (41 percent), nonfatal in 23 (36 percent), and rapidly fatal in 15 (23 percent). Five patients (8 percent) survived to hospital discharge. Patients developing arrest on the general hospital ward services have predominantly respiratory and metabolic derangements immediately preceding their arrests. Their underlying diseases are generally not rapidly fatal. Arrest is frequently preceded by a clinical deterioration involving either respiratory or mental function. These features and the high mortality associated with arrest suggest that efforts to predict and prevent arrest might prove beneficial.

0 comments Cited 169 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study

Anoop D Shah, Jonathan Bartlett, James Carpenter … (2014)

Multivariate imputation by chained equations (MICE) is commonly used for imputing missing data in epidemiologic research. The “true” imputation model may contain nonlinearities which are not included in default imputation models. Random forest imputation is a machine learning technique which can accommodate nonlinearities and interactions and does not require a particular regression model to be specified. We compared parametric MICE with a random forest-based MICE algorithm in 2 simulation studies. The first study used 1,000 random samples of 2,000 persons drawn from the 10,128 stable angina patients in the CALIBER database (Cardiovascular Disease Research using Linked Bespoke Studies and Electronic Records; 2001–2010) with complete data on all covariates. Variables were artificially made “missing at random,” and the bias and efficiency of parameter estimates obtained using different imputation methods were compared. Both MICE methods produced unbiased estimates of (log) hazard ratios, but random forest was more efficient and produced narrower confidence intervals. The second study used simulated data in which the partially observed variable depended on the fully observed variables in a nonlinear way. Parameter estimates were less biased using random forest MICE, and confidence interval coverage was better. This suggests that random forest imputation may be useful for imputing complex epidemiologic data sets in which some patients have missing data.

0 comments Cited 162 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Michelle Ng Gong:

ORCID: http://orcid.org/0000-0001-7952-5384

+1 718 920 2956 , mgong@montefiore.org

Journal

Journal ID (nlm-ta): Crit Care

Title: Critical Care

Publisher: BioMed Central (London )

ISSN (Print): 1364-8535

ISSN (Electronic): 1466-609X

Publication date (Electronic): 30 October 2018

Publication date PMC-release: 30 October 2018

Publication date Collection: 2018

Volume: 22

Electronic Location Identifier: 286

Affiliations

[1 ]ISNI 0000 0004 0459 167X, GRID grid.66875.3a, Department of Anesthesiology and Perioperative Medicine, , Mayo Clinic, ; Rochester, MN USA

[2 ]ISNI 0000 0001 2163 3825, GRID grid.413852.9, Department of Anesthesiology, , HCL CHU Croix-Rousse, ; Lyon, France

[3 ]ISNI 0000 0004 0459 167X, GRID grid.66875.3a, Department of Health Sciences Research, , Mayo Clinic, ; Rochester, MN USA

[4 ]ISNI 0000 0004 0459 167X, GRID grid.66875.3a, Department of Pulmonary and Critical Care Medicine, , Mayo Clinic, ; Rochester, MN USA

[5 ]ISNI 0000000121791997, GRID grid.251993.5, Department of Systems & Computational Biology, Montefiore Health System, , Albert Einstein College of Medicine, ; Bronx, NY USA

[6 ]ISNI 0000000121791997, GRID grid.251993.5, Division of Critical Care Medicine, Department of Medicine, Montefiore Health System, , Albert Einstein College of Medicine, ; Main Floor, Gold Zone, 111 East 210th Street, Bronx, NY 10467 USA

Author information

Michelle Ng Gong http://orcid.org/0000-0001-7952-5384

Article

Publisher ID: 2194

DOI: 10.1186/s13054-018-2194-7

PMC ID: 6206729

PubMed ID: 30373653

SO-VID: 0a70beb4-b680-4fcd-8282-8acad323efb1

License:

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

History

Date received : 27 November 2017

Date accepted : 14 September 2018

Funding

Funded by: FundRef http://dx.doi.org/10.13039/100000050, National Heart, Lung, and Blood Institute;

Award ID: UH2 HL125119

Award ID: UH3 HL125119

Custom metadata

ScienceOpen disciplines: Emergency medicine & Trauma

Keywords: acute respiratory failure,prediction,electronic health records,early warning scores,random forest

Data availability:

ScienceOpen disciplines: Emergency medicine & Trauma

Keywords: acute respiratory failure, prediction, electronic health records, early warning scores, random forest

Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital

Read this article at

Abstract

Background

Methods

Results

Conclusions

Trial registration

Electronic supplementary material

Related collections

Irish Journal of Paramedicine

Most cited references 24

A protocol of no sedation for critically ill patients receiving mechanical ventilation: a randomised trial.

Clinical antecedents to in-hospital cardiopulmonary arrest.

Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study

Author and article information

Contributors

Journal

Affiliations

Author information

Article

History

Funding

Categories

Custom metadata

Comments

Comment on this article

Similar content 65

Cited by 27

Most referenced authors 436