An Early Investigation into the Utility of Multimodal Large Language Models in Medical Imaging
CoRR(2024)
摘要
Recent developments in multimodal large language models (MLLMs) have spurred
significant interest in their potential applications across various medical
imaging domains. On the one hand, there is a temptation to use these generative
models to synthesize realistic-looking medical image data, while on the other
hand, the ability to identify synthetic image data in a pool of data is also
significantly important. In this study, we explore the potential of the Gemini
(gemini-1.0-pro-vision-latest) and GPT-4V (gpt-4-vision-preview)
models for medical image analysis using two modalities of medical image data.
Utilizing synthetic and real imaging data, both Gemini AI and GPT-4V are first
used to classify real versus synthetic images, followed by an interpretation
and analysis of the input images. Experimental results demonstrate that both
Gemini and GPT-4 could perform some interpretation of the input images. In this
specific experiment, Gemini was able to perform slightly better than the GPT-4V
on the classification task. In contrast, responses associated with GPT-4V were
mostly generic in nature. Our early investigation presented in this work
provides insights into the potential of MLLMs to assist with the classification
and interpretation of retinal fundoscopy and lung X-ray images. We also
identify key limitations associated with the early investigation study on MLLMs
for specialized tasks in medical image analysis.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要