Adaptive multi-scale neural network with Resnet blocks for solving partial differential equations

In this paper, an adaptive multi-scale neural network with Resnet blocks (adaptive-MS-Resnet) architecture is constructed for solving the Poisson equation, Helmholtz equation, wave equation and Klein–Gordon equation, which exploits the idea of radial scaling in the frequency domain. The radial scaling achieves the conversion of high frequency to low one to capture the oscillatory part of the partial differential equations’ (PDEs’) solution. Moreover, a novel trigonometric activation function is proposed in the first hidden layer, such that the multi-scale residual neural network can be regarded as a channel for performing a Fourier transform-like on the input. In addition, an adaptive learning rate algorithm is introduced, which can automatically adjust the weights between different loss terms to alleviate the gradient imbalance. Finally, we conduct extensive experiments to demonstrate the effectiveness and accuracy of the proposed adaptive-MS-Resnet.