Additive Nonlinear Preconditioning for Trust Region Optimization of Physics Informed Neural Networks
Please login to view abstract download link
We introduce an additive nonlinear preconditioning strategy for trust-region optimization of physics-informed neural networks. The preconditioner applies a domain decomposition strategy to the collocation data, defining subproblems over subsets of the training points, called subdomains. The resulting subdomain corrections can be computed in parallel and are combined additively to form a proposed step, which is evaluated on the full training objective before a final global trust-region step is performed. Both stages use matrix-free, L-BFGS-preconditioned Steihaug conjugate gradient with Hessian-vector products. We evaluate the approach on six PDE benchmarks against Adam, L-BFGS, and a standard trust-region method under the same wall-clock time budget. Across six random seeds, a nonlinearly preconditioned variant performs best on every benchmark, yielding the lowest mean final training loss on all six problems and the lowest mean relative L^2 error whenever a reference solution is available. The best-performing number of subdomains varies across benchmarks, and performance does not improve monotonically with further subdivision. These results indicate that the effectiveness of the nonlinear preconditioner depends on the chosen decomposition and motivate adaptive strategies for selecting the number of subdomains.
