Prediction model of hospital mortality for patients treated in intensive care units: validation with focus on geographical and temporal transportability

Background: Prognostic models are key for quality and performance evaluations of intensive care units (ICUs). Lack of temporal and geographical transportability challenges the validity of prognostic models serving as monitoring and benchmarking tools for ICUs.Objective: To develop and validate an in-hospital mortality risk prediction model to facilitate benchmarking, quality assurance, and health economics evaluation.Study Design and Setting: We retrieved data from an international multicenter ICU cohort study from 2015-2017. We used a hierarchical logistic regression model that included age, a modified Simplified Acute Physiology Score-II, admission type, premorbid functional status, and diagnosis as the grouping variable.Results: We included 61,224 patients treated in the ICU (hospital mortality 10.6%). The developed prediction model had an area under the receiver operating characteristic curve 0.886, 95% confidence interval (CI) 0.882-0.890; a calibration slope 1.01, 95% CI (0.99-1.03); a mean calibration -0.004, 95% CI (-0.035-0.027). The model showed very good internal validity and geographical and temporal transportability.Conclusion: A novel framework evaluating the performance of our prediction model indicated good transportability properties in addition to traditional performance criteria. This approach could play a role in future evaluation of hospital outcome prediction models for other ICU patient populations.