Towards Robust Out-of-Distribution Generalization Bounds via Sharpness
ICLR 2024(2024)
摘要
Generalizing to out-of-distribution (OOD) data or unseen domain, termed OOD
generalization, still lacks appropriate theoretical guarantees. Canonical OOD
bounds focus on different distance measurements between source and target
domains but fail to consider the optimization property of the learned model. As
empirically shown in recent work, the sharpness of learned minima influences
OOD generalization. To bridge this gap between optimization and OOD
generalization, we study the effect of sharpness on how a model tolerates data
change in domain shift which is usually captured by "robustness" in
generalization. In this paper, we give a rigorous connection between sharpness
and robustness, which gives better OOD guarantees for robust algorithms. It
also provides a theoretical backing for "flat minima leads to better OOD
generalization". Overall, we propose a sharpness-based OOD generalization bound
by taking robustness into consideration, resulting in a tighter bound than
non-robust guarantees. Our findings are supported by the experiments on a ridge
regression model, as well as the experiments on deep learning classification
tasks.
更多查看译文
关键词
Out-of-Distribution generalization,Sharpness,Robustness
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要