SA-MDKIF: A Scalable and Adaptable Medical Domain Knowledge Injection Framework for Large Language Models
CoRR(2024)
摘要
Recent advances in large language models (LLMs) have demonstrated exceptional
performance in various natural language processing (NLP) tasks. However, their
effective application in the medical domain is hampered by a lack of medical
domain knowledge. In this study, we present SA-MDKIF, a scalable and adaptable
framework that aims to inject medical knowledge into general-purpose LLMs
through instruction tuning, thereby enabling adaptability for various
downstream tasks. SA-MDKIF consists of two stages: skill training and skill
adaptation. In the first stage, we define 12 basic medical skills and use
AdaLoRA to train these skills based on uniformly formatted instructional
datasets that we have constructed. In the next stage, we train the skill router
using task-specific downstream data and use this router to integrate the
acquired skills with LLMs during inference. Experimental results on 9 different
medical tasks show that SA-MDKIF improves performance by 10-20
original LLMs. Notably, this improvement is particularly pronounced for unseen
medical tasks, showing an improvement of up to 30
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要