565 - Early detection of Late onset Sepsis and Necrotizing Enterocolitis using bedside Artificial Intelligence based clinical decision support technology.

Poster Number: 565
Publication Number: 565.427

David Van Laere, University Hospital Antwerp, Edegem, Antwerpen, Belgium; Antonius Mulder, University Hospital Antwerp, Edegem, Antwerpen, Belgium; Ludo Mahieu, Antwerp University Hospital, Antwerp, Antwerpen, Belgium; Marisse Meeus, Antwerp university hospital, Edegem, Antwerpen, Belgium

Background: Late onset sepsis and surgical NEC (LOS/NEC) are complications affecting preterm infants. Although many NICU's collect high frequency patient data, timely recognition of disease onset remains challenging.

Objective: To develop a Machine Learning (ML) Model that timely detects LOS/NEC, which is integrated and tested in a newly developed bedside Artificial Intelligence-based Clinical Decision Support system.

Design/Methods: During a retrospective single center cohort study at the Antwerp University Hospital, data from infants born < 32 weeks of gestation between 2012 and 2019 (n=953) were used to train and test an algorithm using a supervised ML technique. Episodes of LOS/NEC (n=172) were labelled (NeoKISS benchmark definitions/Bell's criteria for NEC), using the time point of blood culture sampling (LOS) and/or X-ray diagnosis (NEC) as reference points for clinical diagnosis. Severe sepsis was defined as any LOS with organ failure and/or shock. Features were engineered on time series data from pulse oximeter- and ECG sensors (sampled at a rate of 1/minute) as well as clinical observations in 36 hour time-windows prior to the reference point. These features were compared between LOS/NEC- and control time-windows. ML model performance was evaluated using precision (true positive rate) and recall (sensitivity) on hourly predictions both on the retrospective dataset and on prospective data gathered during a 5-month observational, blinded pilot study (n=48) using the bedside technology. True positive spikes were defined as any increase in ML model probability output in a 24 hour time period prior to the reference point. Any other spike outside this time period was considered false positive. Time gain was defined as the time range between the maximum ML model probability output and the reference point.

Results: The ML model reached a recall of 73% (n=123/172) of LOS/NEC and 78% of severe LOS/NEC (n=67/85) during cross-validation (AUC=0,97) on retrospective data (figure 1). Overall there was a median precision of  < 1 patient alarm per week (figure 2). During the pilot study, the technology produced 78.696 hourly predictions. Recall was 64% (n=9/15) and 100% (n=2/2) for LOS/NEC and severe LOS/NEC respectively with a similar high precision. Relative to a blinded clinical team the median time gain of the ML model was 3,4 (1,23-12,7) hours.Conclusion(s): A properly designed Machine Learning Model can detect LOS/NEC episodes on readily available clinical with a high sensitivity and precision. Bedside AI technology has the potential to improve decision making by leading to a faster clinical awareness (figure 3).
Precision - Recall Curve ML Model Sepsis/NECFigure 1: Precision-recall curve showing the performance of the ML model on the retrospective dataset (n=953): The amount of detected sepsis episodes are visualized on the x-axis, together with the model’s precision (y-axis) for the 4 different model probability output thresholds (black boxes).
Total number of alarm days per weekFigure 2: Boxplot representing median and interquartile range of the number of alarm days per week (y-axis) above a certain prediction threshold (x-axis) for patients in the retrospective dataset (n=953). An alarm day is defined as an increased probability output in a 24 hour time-period. The dashed line represents the threshold of 1 patient- alarm per week.