External Validation of a Smartphone App Model to Predict the Need for Massive Transfusion Using Five Different Definitions

Abstract

Introduction

Previously, a model to predict massive transfusion protocol (activation) was derived using a single-institution dataset. The PROMMTT database was used to externally validate this model’s ability to predict both massive transfusion protocol (MTP) activation and massive transfusion (MT) administration using multiple MT definitions.

Methods

The app model was used to calculate the predicted probability of massive transfusion protocol activation or massive transfusion delivery. The five definitions of MT used were: 1) 10 units packed red blood cells (PRBCs) in 24 hours; 2) Resuscitation Intensity score ≥ 4; 3) Critical Administration Threshold; 4) 4 units PRBCs in 4 hours; and 5) 6 units PRBCs in 6 hours. Receiver operating curves were plotted to compare the predicted probability of MT with observed outcomes.

Results

Of 1245 patients in the dataset, 297 (24%) met definition 1, 570 (47%) met definition 2, 364 (33%) met definition 3, 599 met definition 4 (49.1%), and 395 met definition 5 (32.4%). Regardless of the outcome (MTP activation or MT administration), the predictive ability of the app model was consistent: when predicting activation of the MTP, the area under the curve (AUC) for the model was 0.694 and when predicting MT administration the AUC ranged from 0.695 – 0.711.

Conclusion

Regardless of the definition of massive transfusion used, the app model demonstrates moderate ability to predict the need for massive transfusion in an external, homogenous population. Importantly, the app allows the model to be iteratively re-calibrated (“machine learning”) and thus could improve its predictive capability as additional data are accrued.

Level of Evidence

III

Study Type

Diagnostic test study