Validating and Improving a Predictive Model to Identify Patients at the End of Life (RP316)

Outcomes 1. Participants will be able to understand key short comings of predictive model to identify patients in need of palliative care.2. Participants will be able to understand how clinicians can be involved via chart review in validating and improving a predictive model. Key Message We demonstrate how involving palliative care clinicians in validating and improving a predictive model to identify patients at the end of life can help to improve the clinical usefulness of a predictive model and can help to address some of its limitations. Importance Predictive models have been used to identify patients with poor prognosis to support timely provision of palliative care. Predictive models depend on the outcome to be measurable. Twelve-months prognosis has been a common, yet imperfect, proxy for requiring palliative care services. Objective(s) To use an iterative process of clinician chart review and model adjustments to improve the usefulness of a predictive model to support palliative care services planning and outreach. Scientific Methods Utilized Electronic health records data from Kaiser Permanente Southern California (KPSC) for 2018-2020, was analyzed with logistic regression using a Least Absolute Shrinkage and Selection Operator (LASSO) approach. The outcome was 12 months mortality risk among patients with serious illness (defined via adapted diagnoses codes from the Center to Advance Palliative Care). The final model with 47 variables, has an Area under the curve of 0.85. Risk strata are being refined and validated iteratively via physician chart review. Chart review included the 12 months surprise question and additional criteria including a need for pain/symptom control, lack of life care planning or family conflict. Results The initial high-risk group had a 24% mortality risk. Based on chart review of a random sample of 10 patients (6 high-, 4 low-risk) we found agreement on all 4 low-risk patients. Four out of 6 high-risk patients were confirmed as having high mortality risk. However, 5 out of 6 high-risk patients did not have an immediate SPC need (e.g. unmanaged pain/symptoms, family conflict). Probability cut-offs were restricted to produce a more conservative sample of high-risk patients (30% mortality risk). A second round of chart review (20 patients) is under way and will be augmented if necessary. Conclusion(s) Fine tuning and validating probability cut-offs of predictive models with physician chart review may improve the practical value of predictive models for palliative care. Impact Involving palliative care clinicians in validating and improving a predictive model via chart review can help to improve the usefulness of a predictive model and address a key limitation of predictive models. Keywords Innovative technologies/Communication and prognostication