Leveraging Open-Source Large Language Models for encoding Social Determinants of Health using an Intelligent Router
CoRR(2024)
Abstract
Social Determinants of Health (SDOH) play a significant role in patient
health outcomes. The Center of Disease Control (CDC) introduced a subset of
ICD-10 codes called Z-codes in an attempt to officially recognize and measure
SDOH in the health care system. However, these codes are rarely annotated in a
patient's Electronic Health Record (EHR), and instead, in many cases, need to
be inferred from clinical notes. Previous research has shown that large
language models (LLMs) show promise on extracting unstructured data from EHRs.
However, with thousands of models to choose from with unique architectures and
training sets, it's difficult to choose one model that performs the best on
coding tasks. Further, clinical notes contain trusted health information making
the use of closed-source language models from commercial vendors difficult, so
the identification of open source LLMs that can be run within health
organizations and exhibits high performance on SDOH tasks is an urgent problem.
Here, we introduce an intelligent routing system for SDOH coding that uses a
language model router to direct medical record data to open source LLMs that
demonstrate optimal performance on specific SDOH codes. The intelligent routing
system exhibits state of the art performance of 97.4
5 codes, including homelessness and food insecurity, on par with closed models
such as GPT-4o. In order to train the routing system and validate models, we
also introduce a synthetic data generation and validation paradigm to increase
the scale of training data without needing privacy protected medical records.
Together, we demonstrate an architecture for intelligent routing of inputs to
task-optimal language models to achieve high performance across a set of
medical coding sub-tasks.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined