Geometry-Enriched Token Representations for Scalable Neural Surrogates in Computational Mechanics

  • Alves Ribeiro, Bruno (Brown University)
  • Broggi, Guillaume (Delft University of Technology)
  • Alves Ribeiro, João (Massachusetts Institute of Technology)
  • A. Bessa, Miguel (Brown University)

Please login to view abstract download link

Neural surrogate models have emerged as a viable alternative to traditional numerical solvers to accelerate engineering simulations. Among recent architectures, transformer based approaches, such as Universal Physics Transformers [1] and their extensions [2] have demonstrated scalability to industrial-sized problems by operating on latent representations of mesh data. However, these methods have been developed primarily for computational fluid dynamics and rely on random subsampling to construct coarsened geometry representations, discarding spatial coherence and local geometric properties of the underlying discretization. Recent work has shown that incorporating geometric information into neural operators significantly improves predictive performance on complex three-dimensional domains [3]. In this work, we propose Geometry-Enriched Multi-branch Universal Physics Transformers (GEM-UPT), a geometry-aware token construction strategy for multi-branch physics transformers that replaces conventional random geometry coarsening with a structured encoding scheme preserving local shape information. The enriched geometry tokens enable the transformer backbone to capture meaningful geometric features without architectural modifications while maintaining computational scalability. Importantly, the proposed approach generalizes the transformer surrogate framework beyond fluid dynamics to broader computational mechanics applications. The method is validated on structural mechanics simulations involving thin-walled shell structures with varying geometries. Results show that geometry-enriched tokens yield approximately 50% lower prediction error compared to equivalent architectures with random geometry coarsening [2], while converging in over 20x fewer training epochs. Furthermore, the compact geometric representations reduce peak GPU memory by approximately 50% during training. All models are trained on a single GPU and produce predictions within seconds, confirming the practical viability of the approach for engineering simulation workflows.