How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
arxiv(2024)
摘要
Automated segmentation is a fundamental medical image analysis task, which
enjoys significant advances due to the advent of deep learning. While
foundation models have been useful in natural language processing and some
vision tasks for some time, the foundation model developed with image
segmentation in mind - Segment Anything Model (SAM) - has been developed only
recently and has shown similar promise. However, there are still no systematic
analyses or “best-practice” guidelines for optimal fine-tuning of SAM for
medical image segmentation. This work summarizes existing fine-tuning
strategies with various backbone architectures, model components, and
fine-tuning algorithms across 18 combinations, and evaluates them on 17
datasets covering all common radiology modalities. Our study reveals that (1)
fine-tuning SAM leads to slightly better performance than previous segmentation
methods, (2) fine-tuning strategies that use parameter-efficient learning in
both the encoder and decoder are superior to other strategies, (3) network
architecture has a small impact on final performance, (4) further training SAM
with self-supervised learning can improve final model performance. We also
demonstrate the ineffectiveness of some methods popular in the literature and
further expand our experiments into few-shot and prompt-based settings. Lastly,
we released our code and MRI-specific fine-tuned weights, which consistently
obtained superior performance over the original SAM, at
https://github.com/mazurowski-lab/finetune-SAM.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要