Mimic before Reconstruct: Enhancing Masked Autoencoders with Feature Mimicking
INTERNATIONAL JOURNAL OF COMPUTER VISION(2023)
摘要
Masked Autoencoders (MAE) have been popular paradigms for large-scale vision representation pre-training. However, MAE solely reconstructs the low-level RGB signals after the decoder and lacks supervision upon high-level semantics for the encoder, thus suffering from sub-optimal learned representations and long pre-training epochs. To alleviate this, previous methods simply replace the pixel reconstruction targets of 75
更多查看译文
关键词
Masked autoencoders,Representation learning,Feature mimicking,Image classification
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要