An Editorial Note on Early Identification of Sepsis Risk

EDITORIAL

Sepsis is a leading cause of morbidity and mortality worldwide. Early identification of sepsis is important as it allows timely administration of potentially life-saving resuscitation and antimicrobial therapy. We present COMPOSER (Conformal Multidimensional Prediction of Sepsis Risk), a deep learning model for the early prediction of sepsis, specifically designed to reduce false alarms by detecting unfamiliar patients/situations arising from erroneous data, messiness, distributional shift and data drifts. COMPOSER flags these unfamiliar cases as indeterminate rather than making spurious predictions. Six patient cohorts (515,720 patients) curated from two healthcare systems in the United States across intensive care units (ICU) and Emergency Departments (ED) were used to train and externally and temporally validate this model.

In a sequential prediction setting, COMPOSER achieved a consistently high area under the curve (AUC) (ICU: 0.925–0.953; ED: 0.938–0.945). Out of over 6 million prediction windows roughly 20% and 8% were identified as indeterminate amongst non-septic and septic patients, respectively. COMPOSER provided early warning within a clinically actionable timeframe (ICU: 12.2 [3.2 22.8] and ED: 2.1 [0.8 4.5] hours prior to first antibiotics order) across all six cohorts, thus allowing for identification and prioritization of patients at high risk for sepsis. Sepsis is a deregulated host response to infection causing life-threatening organ dysfunction. Approximately one in three hospital deaths are attributable to sepsis.

While effective protocols exist for treating sepsis, challenges remain in early and reliable detection of this condition. In recent years, the increased adoption of Electronic Medical Records (EHRs) in hospitals has motivated the development of machine learning-based surveillance tools for detection and prediction of sepsis. However, most existing published sepsis prediction models are either based on data from a single hospital or multiple hospitals from the same healthcare system where the care processes are mostly standardized.

Three major barriers to the regulatory approval and widespread adoption of these systems are lack of generalizability across institutions, high false alarm rates, and risk of automation bias, wherein users tend to over-rely on the system output instead of active information seeking and risk assessment. One of the main factors contributing to an algorithm’s performance degradation (including increased false alarm and misseddetection rate) across sites is the data distribution shift (encountering unfamiliar patients) and variations in levels of data messiness caused by differences in hospital workflow and practices. Moreover, a recent study demonstrated that detecting outlier cases and showing users an outlier focused message better enabled them to detect and correct for potential spurious predictions by an AI model.

However, while recent literature has emphasized the importance of including clear ‘indication of use’ labels with machine learning algorithms, none of the existing sepsis prediction algorithms include a built-in mechanism for detecting outliers and for establishing the ‘condition for use’ of the model across geographical and temporal domains. In this work, we propose COMPOSER (Conformal Multidimensional Prediction of Sepsis Risk), a deep learning model designed to predict onset of sepsis 4–48h prior to time of clinical suspicion. COMPOSER achieves improved generalizability and low false alarm rates through a prediction scheme that statistically determines conformity with a predefined collection of representations (aka conformal set), as a means to establish the ‘conditions for use’ of the algorithm under unseen prediction scenarios including new patient populations and different levels of data quality and messiness. The proposed COMPOSER model consists of three modules.

The first module makes use of clinical variables and timing information about measurements to generate lower dimensional representations that are robust to patterns of data messiness and institution-specific workflow practices. The second module includes a conformal prediction network, which provides a statistical framework for detecting out-ofdistribution (i.e., indeterminate) samples during the risk assessment phase in a deployment environment. Two bags of data representations (aka, conformal sets) are used to quantify explicitly the conformity of new patient-level feature vectors to the previously seen examples of septic and non-septic feature vectors within the development cohort.

The conformal prediction allows the model to detect outlier inputs that do not satisfy the conditions for use of the algorithm, which are subsequently assigned to an indeterminate predicted label class. Supplementary provides an illustration of scenarios under which a test sample is accepted or rejected by the conformal prediction module. The third module includes a sepsis predictor that is a feed forward neural network followed by a logistic regression.