Impact of missing values on the ability of the acute physiology and chronic health evaluation III and Japan risk of death models to predict mortality

Purpose: This study assessed model performance of the Acute Physiology and Chronic Health Evaluation (APACHE) III and Japan Risk of Death (JROD) when degraded by the number and category of missing variables. We also examined the impact of missing data on predicted mortality for facilities with missing physiological variables. Methods: We obtained data from the Japanese Intensive care PAtient Database (JIPAD). We calculated observed and predicted mortality rates using the APACHE III and JROD and the standardized mortality ratio (SMR) by the number and category of missing variables. Smoothed spline curves were calculated for the SMR to the missing proportion of the facility.Results: A total of 61,357 patients from 57 ICUs were included between April 2015 and March 2019. The APACHE III and JROD SMRs increased as the number of missing values increased. The SMR in the APACHE III model was elevated in facilities with a larger proportion of missing in each of the APS categories, arterial blood gas, albumin, glucose, and bilirubin. Facilities with a high proportion of missing albumin data preserved their SMRs in only the JROD model. Conclusion: An increased number of missing physiological variables resulted in falsely low predicted mortality rates and high SMRs.