Developing a Machine Learning prediction model for bedside decision support by predicting readmission or death following discharge from the Intensive Care unit

Background Unexpected ICU readmission is associated with longer length of stay and an increase in mortality. Real time support systems could prevent untimely discharge from the ICU. We aim to develop a machine learning model for implementation at the bedside by predicting the risk of ICU readmission or death at time of potential discharge, showing feature importance and visualizing day-to-day changes in risk. Methods Data from adult patients, admitted to our mixed surgical-medical ICU between 2004 and 2016, were used in the analysis. Patient characteristics, clinical observations, (automated) physiological measurements, laboratory studies and treatment data were considered as model features. Different supervised learning algorithms were trained to predict ICU readmission and/or death, both within 7 days from ICU discharge, using 10-fold cross-validation. Feature importance was determined using SHapley Additive exPlanations. We constructed readmission probability-time curves to identify subgroups. Results Our dataset included 14,105 admissions. The combined readmission/mortality rate within seven days of ICU discharge was 5.3%. Using Gradient Boosting, the model achieved a Receiver Operating Characteristic AUC of 0.802 (95% CI 0.789-0.816) and a Precision-Recall AUC of 0.198 (95% CI 0.185-0.211). The most predictive features were well-known parameters, including physiological parameters, as well as less apparent features like nutritional support. Impact analysis using probability-time curves identified specific patients groups, that might lead to a change in discharge management with a relative risk reduction of 17%. Conclusions We developed a model that can accurately predict readmission and mortality after ICU discharge. Impact analysis showed that a relative risk reduction of 17% could be achievable. Given the large and increasing number of ICU admissions worldwide, this modest reduction may have significant impact for patients and society.