Challenges in Training PINNs: A Loss Landscape Perspective
CoRR(2024)
摘要
This paper explores challenges in training Physics-Informed Neural Networks
(PINNs), emphasizing the role of the loss landscape in the training process. We
examine difficulties in minimizing the PINN loss function, particularly due to
ill-conditioning caused by differential operators in the residual term. We
compare gradient-based optimizers Adam, L-BFGS, and their combination
Adam+L-BFGS, showing the superiority of Adam+L-BFGS, and introduce a novel
second-order optimizer, NysNewton-CG (NNCG), which significantly improves PINN
performance. Theoretically, our work elucidates the connection between
ill-conditioned differential operators and ill-conditioning in the PINN loss
and shows the benefits of combining first- and second-order optimization
methods. Our work presents valuable insights and more powerful optimization
strategies for training PINNs, which could improve the utility of PINNs for
solving difficult partial differential equations.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要