Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimization
CoRR(2024)
摘要
Masked Autoencoder (MAE) is a notable method for self-supervised pretraining
in visual representation learning. It operates by randomly masking image
patches and reconstructing these masked patches using the unmasked ones. A key
limitation of MAE lies in its disregard for the varying informativeness of
different patches, as it uniformly selects patches to mask. To overcome this,
some approaches propose masking based on patch informativeness. However, these
methods often do not consider the specific requirements of downstream tasks,
potentially leading to suboptimal representations for these tasks. In response,
we introduce the Multi-level Optimized Mask Autoencoder (MLO-MAE), a novel
framework that leverages end-to-end feedback from downstream tasks to learn an
optimal masking strategy during pretraining. Our experimental findings
highlight MLO-MAE's significant advancements in visual representation learning.
Compared to existing methods, it demonstrates remarkable improvements across
diverse datasets and tasks, showcasing its adaptability and efficiency. Our
code is available at: https://github.com/Alexiland/MLOMAE
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要