Active deep learning to detect cognitive concerns in electronic health records

Background: Timely diagnosis of dementia is important to patients and their caregivers for advanced planning, yet dementia is under-diagnosed by healthcare professionals and under-coded in claims data. Sensitive and specific tools to detect cognitive concerns in diverse clinical settings could prompt referral for cognitive evaluation and specialist care. Method: We developed a deep learning natural language processing (NLP) method to detect cognitive concerns in unstructured clinician notes from electronic health records (EHR). We leveraged a gold-standard set of similar to 1000 patients sampled randomly from three strata: patients with diagnosis codes, patients with specialist visits but no code, or patients with neither. The physician performed a detailed chart review and adjudication of cognitive status, noting "cognitive concern" (i.e., any evidence of cognitive difficulties) and rating patients on a 5-point scale: normal, normal vs. MCI, MCI, MCI vs. dementia, and dementia. We used 10% of the labeled data as a test set and the remaining 90% for training and validation of the model to classify patients with any cognitive concerns (normal vs. other). We also built a web-based chart review annotation tool that facilitates labeling and enables an active learning loop to scale up labeling to thousands of charts. Result: In a random sample from the gold-standard dataset, 30 out of 80 patients with cognitive concerns had no diagnosis code or medication related to dementia; we hypothesized that our deep learning tool could leverage clinical text to improve detection of cognitive concerns. Indeed, a model with codes and medications had an area under the receiver operating characteristic (AUROC) curve of 0.79, sensitivity of 0.59, and specificity of 1.00 for the binary classification task. The deep learning model improved the AUROC to 0.90, increased sensitivity to 0.79, and maintained specificity of 0.98. Notes from primary care, specialties such as neurology and psychiatry, and social workers had the highest likelihood of containing information. Conclusion: The deep learning model was successful in detecting cases without a dementia-related diagnosis code or medication. Automatic processing of electronic medical records with a deep learning tool can be used for early detection of cognitive concern to optimize patient care and predict hospital readmission.