Development and Testing of a Novel Large Language Model-Based Clinical Decision Support Systems for Medication Safety in 12 Clinical Specialties
CoRR(2024)
摘要
Importance: We introduce a novel Retrieval Augmented Generation (RAG)-Large
Language Model (LLM) as a Clinical Decision Support System (CDSS) for safe
medication prescription. This model addresses the limitations of traditional
rule-based CDSS by providing relevant prescribing error alerts tailored to
patient context and institutional guidelines.
Objective: The study evaluates the efficacy of an LLM-based CDSS in
identifying medication errors across various medical and surgical case
vignettes, compared to a human expert panel. It also examines clinician
preferences among different CDSS integration modalities: junior pharmacist,
LLM-based CDSS alone, and a combination of both.
Design, Setting, and Participants: Utilizing a RAG model with GPT-4.0, the
study involved 61 prescribing error scenarios within 23 clinical vignettes
across 12 specialties. An expert panel assessed these cases using the PCNE
classification and NCC MERP index. Three junior pharmacists independently
reviewed each vignette under simulated conditions.
Main Outcomes and Measures: The study assesses the LLM-based CDSS's accuracy,
precision, recall, and F1 scores in identifying Drug-Related Problems (DRPs),
compared to junior pharmacists alone or in an assistive mode with the CDSS.
Results: The co-pilot mode of RAG-LLM significantly improved DRP
identification accuracy by 22
and F1 scores, indicating better detection of severe DRPs, despite a slight
decrease in precision. Accuracy varied across categories when pharmacists had
access to RAG-LLM responses.
Conclusions: The RAG-LLM based CDSS enhances medication error identification
accuracy when used with junior pharmacists, especially in detecting severe
DRPs.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要