Neural Plasticity-Inspired Foundation Model for Observing the Earth Crossing Modalities
arxiv(2024)
摘要
The development of foundation models has revolutionized our ability to
interpret the Earth's surface using satellite observational data. Traditional
models have been siloed, tailored to specific sensors or data types like
optical, radar, and hyperspectral, each with its own unique characteristics.
This specialization hinders the potential for a holistic analysis that could
benefit from the combined strengths of these diverse data sources. Our novel
approach introduces the Dynamic One-For-All (DOFA) model, leveraging the
concept of neural plasticity in brain science to integrate various data
modalities into a single framework adaptively. This dynamic hypernetwork,
adjusting to different wavelengths, enables a single versatile Transformer
jointly trained on data from five sensors to excel across 12 distinct Earth
observation tasks, including sensors never seen during pretraining. DOFA's
innovative design offers a promising leap towards more accurate, efficient, and
unified Earth observation analysis, showcasing remarkable adaptability and
performance in harnessing the potential of multimodal Earth observation data.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要