Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision
arxiv(2024)
摘要
Humans effortlessly interpret images by parsing them into part-whole
hierarchies; deep learning excels in learning multi-level feature spaces, but
they often lack explicit coding of part-whole relations, a prominent property
of medical imaging. To overcome this limitation, we introduce Adam-v2, a new
self-supervised learning framework extending Adam [79] by explicitly
incorporating part-whole hierarchies into its learning objectives through three
key branches: (1) Localizability, acquiring discriminative representations to
distinguish different anatomical patterns; (2) Composability, learning each
anatomical structure in a parts-to-whole manner; and (3) Decomposability,
comprehending each anatomical structure in a whole-to-parts manner.
Experimental results across 10 tasks, compared to 11 baselines in zero-shot,
few-shot transfer, and full fine-tuning settings, showcase Adam-v2's superior
performance over large-scale medical models and existing SSL methods across
diverse downstream tasks. The higher generality and robustness of Adam-v2's
representations originate from its explicit construction of hierarchies for
distinct anatomical structures from unlabeled medical images. Adam-v2 preserves
a semantic balance of anatomical diversity and harmony in its embedding,
yielding representations that are both generic and semantically meaningful, yet
overlooked in existing SSL methods. All code and pretrained models are
available at https://github.com/JLiangLab/Eden.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要