Vital signs and their cross-correlation in sepsis and NEC: a study of 1,065 very-low-birth-weight infants in two NICUs

The net reclassification improvement (NRI) is an increasingly popular measure for evaluating improvements in risk predictions. This article details a review of 67 publications in high-impact general clinical journals that considered the NRI. Incomplete reporting of NRI methods, incorrect calculation, and common misinterpretations were found. To aid improved applications of the NRI, the article elaborates on several aspects of the computation and interpretation in various settings. Limitations and controversies are discussed, including the effect of miscalibration of prediction models, the use of the continuous NRI and “clinical NRI,” and the relation with decision analytic measures. A systematic approach toward presenting NRI analysis is proposed: Detail and motivate the methods used for computation of the NRI, use clinically meaningful risk cutoffs for the category-based NRI, report both NRI components, address issues of calibration, and do not interpret the overall NRI as a percentage of the study population reclassified. Promising NRI findings need to be followed with decision analytic or formal cost-effectiveness evaluations.

There are two popular statistical approaches to biomarker evaluation. One models the risk of disease (or disease outcome) with, for example, logistic regression. A marker is considered useful if it has a strong effect on risk. The second evaluates classification performance by use of measures such as sensitivity, specificity, predictive values, and receiver operating characteristic curves. There is controversy about which approach is more appropriate. Moreover, the two approaches can give contradictory results on the same data. The authors present a new graphic, the predictiveness curve, which complements the risk modeling approach. It assesses the usefulness of a risk model when applied to the population. Although the predictiveness curve relates to classification performance measures, it also displays essential information about risk that is not displayed by the receiver operating characteristic curve. The authors propose that the predictiveness and classification performance of a marker, displayed together in an integrated plot, provide a comprehensive and cohesive assessment of a risk marker or model. The methods are demonstrated with data on prostate-specific antigen and risk factors from the Prostate Cancer Prevention Trial, 1993-2003.

Late-onset neonatal sepsis is a significant cause of morbidity and mortality, and early detection could prove beneficial. Previously, we found that abnormal heart rate characteristics (HRC) of reduced variability and transient decelerations occurred early in the course of neonatal sepsis and sepsis-like illness in infants in a single neonatal intensive care unit (NICU). We hypothesized that this finding can be generalized to other NICUs. We prospectively collected clinical data and continuously measured RR intervals in all infants in two NICUs who stayed for >7 d. We defined episodes of sepsis and sepsis-like illness as acute clinical deteriorations that prompted physicians to obtain blood cultures and start antibiotics. A predictive statistical model yielding an HRC index was developed on a derivation cohort of 316 neonates in the University of Virginia NICU and then applied to the validation cohort of 317 neonates in the Wake Forest University NICU. In the derivation cohort, there were 155 episodes of sepsis and sepsis-like illness in 101 infants, and in the validation cohort, there were 118 episodes in 93 infants. In the validation cohort, the HRC index 1) showed highly significant association with impending sepsis and sepsis-like illness (receiver operator characteristic area 0.75, p < 0.001) and 2) added significantly to the demographic information of birth weight, gestational age, and days of postnatal age in predicting sepsis and sepsis-like illness (p < 0.001). Continuous HRC monitoring is a generally valid and potentially useful noninvasive tool in the early diagnosis of neonatal sepsis and sepsis-like illness.