Grounding Foundation Models through Federated Transfer Learning: A General Framework
CoRR(2023)
摘要
Foundation Models (FMs) such as GPT-4 encoded with vast knowledge and
powerful emergent abilities have achieved remarkable success in various natural
language processing and computer vision tasks. Grounding FMs by adapting them
to domain-specific tasks or augmenting them with domain-specific knowledge
enables us to exploit the full potential of FMs. However, grounding FMs faces
several challenges, stemming primarily from constrained computing resources,
data privacy, model heterogeneity, and model ownership. Federated Transfer
Learning (FTL), the combination of federated learning and transfer learning,
provides promising solutions to address these challenges. In recent years, the
need for grounding FMs leveraging FTL, coined FTL-FM, has arisen strongly in
both academia and industry. Motivated by the strong growth in FTL-FM research
and the potential impact of FTL-FM on industrial applications, we propose an
FTL-FM framework that formulates problems of grounding FMs in the federated
learning setting, construct a detailed taxonomy based on the FTL-FM framework
to categorize state-of-the-art FTL-FM works, and comprehensively overview
FTL-FM works based on the proposed taxonomy. We also establish correspondences
between FTL-FM and conventional phases of adapting FM so that FM practitioners
can align their research works with FTL-FM. In addition, we overview advanced
efficiency-improving and privacy-preserving techniques because efficiency and
privacy are critical concerns in FTL-FM. Last, we discuss opportunities and
future research directions of FTL-FM.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要