On Large-Scale PDE Foundation Models
Please login to view abstract download link
Recent advances in data-driven operator learning have demonstrated the potential of foundation models for solving partial differential equations (PDEs) across a wide range of physics problems. In prior work, we introduced Poseidon, an efficient foundation model for PDEs, achieving strong sample efficiency and generalization on two-dimensional benchmark problems. While Poseidon was proof-of-concept work on two-dimensional (in space) benchmark datasets, applications of foundation models at large scale and for realistic problems are still to be shown, though advancements are being made. In this contribution, we revisit Poseidon and the concept of foundation models for physics and discuss its extension towards large-scale three-dimensional problems on unstructured domains for different physical domains including, but not limited to fluid dynamics and structural mechanics. We pay particular attention to challenges that arise in realistic settings at large problem-, model-, and data-scale. We investigate whether similar sample efficiency and accuracy gains (compared to standard neural surrogates that are trained from random initialization) can be observed in these large-scale models when compared to e.g. Poseidon, and discuss their limitations. These findings will motivate ongoing developments aimed at improving efficiency for data-driven simulations.
