<div class="section">
<a class="named-anchor" id="S1">
<!--
named anchor
-->
</a>
<h5 class="section-title" id="d2741506e207">Background</h5>
<p id="P1">Patients in general medical-surgical wards who experience unplanned transfer
to the
intensive care unit (ICU) show evidence of physiologic derangement 6–24 h prior to
their deterioration. With increasing availability of electronic medical records (EMRs),
automated early warning scores (EWSs) are becoming feasible.
</p>
</div><div class="section">
<a class="named-anchor" id="S2">
<!--
named anchor
-->
</a>
<h5 class="section-title" id="d2741506e212">Objective</h5>
<p id="P2">To describe the development and performance of an automated EWS based on
EMR data.</p>
</div><div class="section">
<a class="named-anchor" id="S3">
<!--
named anchor
-->
</a>
<h5 class="section-title" id="d2741506e217">Materials and methods</h5>
<p id="P3">We used a discrete-time logistic regression model to obtain an hourly risk
score to
predict unplanned transfer to the ICU within the next 12 h. The model was based on
hospitalization episodes from all adult patients (18 years) admitted to 21 Kaiser
Permanente Northern California (KPNC) hospitals from 1/1/2010 to 12/31/2013. Eligible
patients met these entry criteria: initial hospitalization occurred at a KPNC hospital;
the hospitalization was not for childbirth; and the EMR had been operational at the
hospital for at least 3 months. We evaluated the performance of this risk score, called
Advanced Alert Monitor (AAM) and compared it against two other EWSs (eCART and NEWS)
in terms of their sensitivity, specificity, negative predictive value, positive predictive
value, and area under the receiver operator characteristic curve (c statistic).
</p>
</div><div class="section">
<a class="named-anchor" id="S4">
<!--
named anchor
-->
</a>
<h5 class="section-title" id="d2741506e222">Results</h5>
<p id="P4">A total of 649,418 hospitalization episodes involving 374,838 patients
met inclusion
criteria, with 19,153 of the episodes experiencing at least one outcome. The analysis
data set had 48,723,248 hourly observations. Predictors included physiologic data
(laboratory tests and vital signs); neurological status; severity of illness and longitudinal
comorbidity indices; care directives; and health services indicators (e.g. elapsed
length of stay). AAM showed better performance compared to NEWS and eCART in all the
metrics and prediction intervals. The AAM AUC was 0.82 compared to 0.79 and 0.76 for
eCART and NEWS, respectively. Using a threshold that generated 1 alert per day in
a unit with a patient census of 35, the sensitivity of AAM was 49% (95% CI: 47.6–50.3%)
compared to the sensitivities of eCART and NEWS scores of 44% (42.3–45.1) and 40%
(38.2–40.9), respectively. For all three scores, about half of alerts occurred within
12 h of the event, and almost two thirds within 24 h of the event.
</p>
</div><div class="section">
<a class="named-anchor" id="S5">
<!--
named anchor
-->
</a>
<h5 class="section-title" id="d2741506e227">Conclusion</h5>
<p id="P5">The AAM score is an example of a score that takes advantage of multiple
data streams
now available in modern EMRs. It highlights the ability to harness complex algorithms
to maximize signal extraction. The main challenge in the future is to develop detection
approaches for patients in whom data are sparser because their baseline risk is lower.
</p>
</div>