Zero Coordinate Shift: Whetted Automatic Differentiation for Physics-informed Operator Learning
arxiv(2023)
摘要
Automatic differentiation (AD) is a critical step in physics-informed machine
learning, required for computing the high-order derivatives of network output
w.r.t. coordinates of collocation points. In this paper, we present a novel and
lightweight algorithm to conduct AD for physics-informed operator learning,
which we call the trick of Zero Coordinate Shift (ZCS). Instead of making all
sampled coordinates as leaf variables, ZCS introduces only one scalar-valued
leaf variable for each spatial or temporal dimension, simplifying the wanted
derivatives from "many-roots-many-leaves" to "one-root-many-leaves" whereby
reverse-mode AD becomes directly utilisable. It has led to an outstanding
performance leap by avoiding the duplication of the computational graph along
the dimension of functions (physical parameters). ZCS is easy to implement with
current deep learning libraries; our own implementation is achieved by
extending the DeepXDE package. We carry out a comprehensive benchmark analysis
and several case studies, training physics-informed DeepONets to solve partial
differential equations (PDEs) without data. The results show that ZCS has
persistently reduced GPU memory consumption and wall time for training by an
order of magnitude, and such reduction factor scales with the number of
functions. As a low-level optimisation technique, ZCS imposes no restrictions
on data, physics (PDE) or network architecture and does not compromise training
results from any aspect.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要