Optimizing the spectral bias in Fourier feature physics-informed neural networks
Please login to view abstract download link
Physics-Informed Neural Networks (PINNs) typically exhibit spectral bias, where high-frequency components of the solution converge significantly slower than low-frequency ones. In this work, we analyze the training dynamics of Fourier Feature PINNs in the Neural Tangent Kernel regime to address this limitation. We derive an explicit evolution equation for the prediction error in the frequency domain, demonstrating that the convergence rate of specific frequencies is primarily governed by the product of the differential operator's symbol and the spectral density of the initialization weights. Leveraging this theoretical insight, we propose an informative initialization strategy that tailors the initial weight distribution to the specific PDE being solved. With this method, we can counteract the operator-induced spectral bias, balancing the convergence speeds across the frequency spectrum. Numerical experiments on linear and non-linear partial differential equations confirm that this initialization strategy accelerates training and improves approximation accuracy for high-frequency features compared to standard initialization methods.
